[jira] [Commented] (HBASE-11234) FastDiffDeltaEncoder#getFirstKeyInBlock returns wrong result
[
https://issues.apache.org/jira/browse/HBASE-11234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008289#comment-14008289
]
Anoop Sam John commented on HBASE-11234:
prefix-tree part of fix is required in Trunk also.
For 94 and 96 patches no changes are needed. 94 is reverted I think. (Which we
can commit again)
I am fine with attached fix. Would like to get Matt Corgan's opinion also.
> FastDiffDeltaEncoder#getFirstKeyInBlock returns wrong result
>
>
> Key: HBASE-11234
> URL: https://issues.apache.org/jira/browse/HBASE-11234
> Project: HBase
> Issue Type: Bug
>Reporter: chunhui shen
>Assignee: chunhui shen
>Priority: Critical
> Fix For: 0.99.0, 0.96.3, 0.94.21, 0.98.4
>
> Attachments: 11234-94.patch, 11234-96.patch,
> 11234-98-with-prefix-tree.txt, HBASE-11234.patch
>
>
> As Ted found,
> With this change:
> {noformat}
> Index:
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
> ===
> ---
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
>(revision 1596579)
> +++
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
>(working copy)
> @@ -51,6 +51,7 @@
> import org.apache.hadoop.hbase.filter.FilterList.Operator;
> import org.apache.hadoop.hbase.filter.PageFilter;
> import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
> +import org.apache.hadoop.hbase.io.encoding.DataBlockEncoding;
> import org.apache.hadoop.hbase.io.hfile.CacheConfig;
> import org.apache.hadoop.hbase.io.hfile.HFileContext;
> import org.apache.hadoop.hbase.io.hfile.HFileContextBuilder;
> @@ -90,6 +91,7 @@
> CacheConfig cacheConf = new CacheConfig(TEST_UTIL.getConfiguration());
> HFileContextBuilder hcBuilder = new HFileContextBuilder();
> hcBuilder.withBlockSize(2 * 1024);
> +hcBuilder.withDataBlockEncoding(DataBlockEncoding.FAST_DIFF);
> HFileContext hFileContext = hcBuilder.build();
> StoreFile.Writer writer = new StoreFile.WriterBuilder(
> TEST_UTIL.getConfiguration(), cacheConf, fs).withOutputDir(
> {noformat}
> I got:
> java.lang.AssertionError:
> expected:
> but was:
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hbase.regionserver.TestReversibleScanners.seekTestOfReversibleKeyValueScanner(TestReversibleScanners.java:533)
> at
> org.apache.hadoop.hbase.regionserver.TestReversibleScanners.testReversibleStoreFileScanner(TestReversibleScanners.java:108)
> After debugging, it seems the method of
> FastDiffDeltaEncoder#getFirstKeyInBlock become broken. And it will cause
> hfilescanner#seekBefore returns wrong result.
> The solution is simple, see the patch.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Resolved] (HBASE-11246) Backport HBASE-11234 'FastDiffDeltaEncoder#getFirstKeyInBlock returns wrong result'
[ https://issues.apache.org/jira/browse/HBASE-11246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John resolved HBASE-11246. Resolution: Invalid HBASE-11234 is in reopened state now. No need for this Jira. So closing > Backport HBASE-11234 'FastDiffDeltaEncoder#getFirstKeyInBlock returns wrong > result' > --- > > Key: HBASE-11246 > URL: https://issues.apache.org/jira/browse/HBASE-11246 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Priority: Critical > Fix For: 0.98.4 > > > HBASE-11234 fixed bug in FastDiffDeltaEncoder. > This issue is to backport the fix to earlier releases. > For 0.96 / 0.98, the fix should also cover prefix tree encoding. This would > allow TestReversibleScanners#testReversibleStoreFileScanner to iterate > through all the data block encodings. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Reopened] (HBASE-11246) Backport HBASE-11234 'FastDiffDeltaEncoder#getFirstKeyInBlock returns wrong result'
[ https://issues.apache.org/jira/browse/HBASE-11246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu reopened HBASE-11246: > Backport HBASE-11234 'FastDiffDeltaEncoder#getFirstKeyInBlock returns wrong > result' > --- > > Key: HBASE-11246 > URL: https://issues.apache.org/jira/browse/HBASE-11246 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Priority: Critical > Fix For: 0.98.4 > > > HBASE-11234 fixed bug in FastDiffDeltaEncoder. > This issue is to backport the fix to earlier releases. > For 0.96 / 0.98, the fix should also cover prefix tree encoding. This would > allow TestReversibleScanners#testReversibleStoreFileScanner to iterate > through all the data block encodings. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-11246) Backport HBASE-11234 'FastDiffDeltaEncoder#getFirstKeyInBlock returns wrong result'
[ https://issues.apache.org/jira/browse/HBASE-11246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved HBASE-11246. Resolution: Duplicate > Backport HBASE-11234 'FastDiffDeltaEncoder#getFirstKeyInBlock returns wrong > result' > --- > > Key: HBASE-11246 > URL: https://issues.apache.org/jira/browse/HBASE-11246 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Priority: Critical > Fix For: 0.98.4 > > > HBASE-11234 fixed bug in FastDiffDeltaEncoder. > This issue is to backport the fix to earlier releases. > For 0.96 / 0.98, the fix should also cover prefix tree encoding. This would > allow TestReversibleScanners#testReversibleStoreFileScanner to iterate > through all the data block encodings. -- This message was sent by Atlassian JIRA (v6.2#6252)
Does anyone know how to read hbase hdfs file using java api?
Hi experts,
I create a table using this command: create 'test','cf1'.
then put some data there: put 'test','realRow','cf1','realvalue1'
now I am trying to read its data using hdfs java api instead of hbase api:
FileSystem fs = FileSystem.get(URI.create(uri), conf);
Path path = new Path(
"hdfs://localhost:9000/apps/hbase/data/data/default/test/69c16a187661b1ea8dd904851b9e3bb0/cf1/111c243d1953442f93b4d653690abe20"
);
FSDataInputStream in = fs.open(path);
String filename = "111c243d1953442f93b4d653690abe20";
BufferedOutputStream out = new BufferedOutputStream(
new FileOutputStream(new File(filename)));
byte[] b = new byte[1024];
int numBytes = 0;
while ((numBytes = in.read(b)) > 0) {
out.write(b, 0, numBytes);
System.out.println(Bytes.toString(b));
}
But data come like this:
[image: Inline image 1]
Is there something wrong with my decoding code?
Thanks
[jira] [Commented] (HBASE-11096) stop method of Master and RegionServer coprocessor is not invoked
[ https://issues.apache.org/jira/browse/HBASE-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008311#comment-14008311 ] Qiang Tian commented on HBASE-11096: Hi [~apurtell], it looks the shutdown method is not called..could you please check if the patch apply to the MasterCoprocessorHost.java correctly? I just run command "patch -p0< ../backup/HBASE-11096-trunk-v3.patch" to trunk, and the change is not applied correctly. ps when I firstly saw your update on 20/5, I merged the code with latest code using "git merge" command and the test running fine. today I used "git rebase" command to merge my local branch with the latest code (since you recommended rebase in mailing list), and I got the error ---after investigation the MasterCoprocessorHost.java was not merged correctly.. I did a search, it looks git rebase uses patch command to do the final merge. so if patch does not work correctly, the rebase will fail too... pps..this is my second time to see patch command fails...Did I use wrong option? > stop method of Master and RegionServer coprocessor is not invoked > -- > > Key: HBASE-11096 > URL: https://issues.apache.org/jira/browse/HBASE-11096 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.2, 0.98.1, 0.94.19 >Reporter: Qiang Tian >Assignee: Qiang Tian > Fix For: 0.99.0, 0.96.3, 0.94.21, 0.98.4 > > Attachments: HBASE-11096-0.94.patch, HBASE-11096-0.96.patch, > HBASE-11096-0.98.patch, HBASE-11096-trunk-v0.patch, > HBASE-11096-trunk-v0.patch, HBASE-11096-trunk-v1.patch, > HBASE-11096-trunk-v2.patch, HBASE-11096-trunk-v3.patch, > HBASE-11096-trunk-v3.patch, TestCoprocessorStop-failed-output.txt > > > the stop method of coprocessor specified by > "hbase.coprocessor.master.classes" and > "hbase.coprocessor.regionserver.classes" is not invoked. > If coprocessor allocates OS resources, it could cause master/regionserver > resource leak or hang during exit. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Does anyone know how to read hbase hdfs file using java api?
On Sat, May 24, 2014 at 11:23 PM, David Liu wrote:
> Hi experts,
>
> I create a table using this command: create 'test','cf1'.
> then put some data there: put 'test','realRow','cf1','realvalue1'
> now I am trying to read its data using hdfs java api instead of hbase api:
>
> FileSystem fs = FileSystem.get(URI.create(uri), conf);
>
> Path path = new Path(
>
>
> "hdfs://localhost:9000/apps/hbase/data/data/default/test/69c16a187661b1ea8dd904851b9e3bb0/cf1/111c243d1953442f93b4d653690abe20"
> );
>
> FSDataInputStream in = fs.open(path);
>
> String filename = "111c243d1953442f93b4d653690abe20";
>
> BufferedOutputStream out = new BufferedOutputStream(
>
> new FileOutputStream(new File(filename)));
>
> byte[] b = new byte[1024];
>
> int numBytes = 0;
>
> while ((numBytes = in.read(b)) > 0) {
>
> out.write(b, 0, numBytes);
>
> System.out.println(Bytes.toString(b));
>
> }
>
> But data come like this:
>
> [image: Inline image 1]
>
> Is there something wrong with my decoding code?
>
>
> Your image did not come across (they are suppressed on this mailing list).
Better to put up a link in future.
You just want to read raw bytes or you want interpreted view on the
contents?
HBase writes hfiles: see http://hbase.apache.org/book.html#hfilev2
For an example reading them, start at the main method here
http://hbase.apache.org/xref/org/apache/hadoop/hbase/io/hfile/HFile.html#891and
follow through to the pretty print class.
St.Ack
[jira] [Created] (HBASE-11249) Missing null check in finally block of HRegion#processRowsWithLocks()
Ted Yu created HBASE-11249:
--
Summary: Missing null check in finally block of
HRegion#processRowsWithLocks()
Key: HBASE-11249
URL: https://issues.apache.org/jira/browse/HBASE-11249
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
At line 4883:
{code}
Store store = getStore(kv);
if (store == null) {
checkFamily(CellUtil.cloneFamily(kv));
// unreachable
}
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11249) Missing null check in finally block of HRegion#processRowsWithLocks()
[
https://issues.apache.org/jira/browse/HBASE-11249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-11249:
---
Description:
At line 4883:
{code}
Store store = getStore(kv);
if (store == null) {
checkFamily(CellUtil.cloneFamily(kv));
// unreachable
}
{code}
Exception would be thrown from checkFamily() if store is null.
In the finally block:
{code}
} finally {
if (!mutations.isEmpty() && !walSyncSuccessful) {
LOG.warn("Wal sync failed. Roll back " + mutations.size() +
" memstore keyvalues for row(s):" + StringUtils.byteToHexString(
processor.getRowsToLock().iterator().next()) + "...");
for (KeyValue kv : mutations) {
getStore(kv).rollback(kv);
}
{code}
There is no corresponding null check for store above, potentially leading to
partially rolled back state in memstore.
was:
At line 4883:
{code}
Store store = getStore(kv);
if (store == null) {
checkFamily(CellUtil.cloneFamily(kv));
// unreachable
}
{code}
> Missing null check in finally block of HRegion#processRowsWithLocks()
> -
>
> Key: HBASE-11249
> URL: https://issues.apache.org/jira/browse/HBASE-11249
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>
> At line 4883:
> {code}
> Store store = getStore(kv);
> if (store == null) {
> checkFamily(CellUtil.cloneFamily(kv));
> // unreachable
> }
> {code}
> Exception would be thrown from checkFamily() if store is null.
> In the finally block:
> {code}
> } finally {
> if (!mutations.isEmpty() && !walSyncSuccessful) {
> LOG.warn("Wal sync failed. Roll back " + mutations.size() +
> " memstore keyvalues for row(s):" + StringUtils.byteToHexString(
> processor.getRowsToLock().iterator().next()) + "...");
> for (KeyValue kv : mutations) {
> getStore(kv).rollback(kv);
> }
> {code}
> There is no corresponding null check for store above, potentially leading to
> partially rolled back state in memstore.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11249) Missing null check in finally block of HRegion#processRowsWithLocks()
[
https://issues.apache.org/jira/browse/HBASE-11249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-11249:
---
Description:
At line 4883:
{code}
Store store = getStore(kv);
if (store == null) {
checkFamily(CellUtil.cloneFamily(kv));
// unreachable
}
{code}
Exception would be thrown from checkFamily() if store is null.
In the finally block:
{code}
} finally {
if (!mutations.isEmpty() && !walSyncSuccessful) {
LOG.warn("Wal sync failed. Roll back " + mutations.size() +
" memstore keyvalues for row(s):" + StringUtils.byteToHexString(
processor.getRowsToLock().iterator().next()) + "...");
for (KeyValue kv : mutations) {
getStore(kv).rollback(kv);
}
{code}
There is no corresponding null check for return value of getStore() above,
potentially leading to partially rolled back state in memstore.
was:
At line 4883:
{code}
Store store = getStore(kv);
if (store == null) {
checkFamily(CellUtil.cloneFamily(kv));
// unreachable
}
{code}
Exception would be thrown from checkFamily() if store is null.
In the finally block:
{code}
} finally {
if (!mutations.isEmpty() && !walSyncSuccessful) {
LOG.warn("Wal sync failed. Roll back " + mutations.size() +
" memstore keyvalues for row(s):" + StringUtils.byteToHexString(
processor.getRowsToLock().iterator().next()) + "...");
for (KeyValue kv : mutations) {
getStore(kv).rollback(kv);
}
{code}
There is no corresponding null check for store above, potentially leading to
partially rolled back state in memstore.
> Missing null check in finally block of HRegion#processRowsWithLocks()
> -
>
> Key: HBASE-11249
> URL: https://issues.apache.org/jira/browse/HBASE-11249
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>
> At line 4883:
> {code}
> Store store = getStore(kv);
> if (store == null) {
> checkFamily(CellUtil.cloneFamily(kv));
> // unreachable
> }
> {code}
> Exception would be thrown from checkFamily() if store is null.
> In the finally block:
> {code}
> } finally {
> if (!mutations.isEmpty() && !walSyncSuccessful) {
> LOG.warn("Wal sync failed. Roll back " + mutations.size() +
> " memstore keyvalues for row(s):" + StringUtils.byteToHexString(
> processor.getRowsToLock().iterator().next()) + "...");
> for (KeyValue kv : mutations) {
> getStore(kv).rollback(kv);
> }
> {code}
> There is no corresponding null check for return value of getStore() above,
> potentially leading to partially rolled back state in memstore.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11249) Missing null check in finally block of HRegion#processRowsWithLocks() may lead to partially rolled back state in memstore.
[
https://issues.apache.org/jira/browse/HBASE-11249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-11249:
---
Summary: Missing null check in finally block of
HRegion#processRowsWithLocks() may lead to partially rolled back state in
memstore. (was: Missing null check in finally block of
HRegion#processRowsWithLocks())
> Missing null check in finally block of HRegion#processRowsWithLocks() may
> lead to partially rolled back state in memstore.
> --
>
> Key: HBASE-11249
> URL: https://issues.apache.org/jira/browse/HBASE-11249
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>
> At line 4883:
> {code}
> Store store = getStore(kv);
> if (store == null) {
> checkFamily(CellUtil.cloneFamily(kv));
> // unreachable
> }
> {code}
> Exception would be thrown from checkFamily() if store is null.
> In the finally block:
> {code}
> } finally {
> if (!mutations.isEmpty() && !walSyncSuccessful) {
> LOG.warn("Wal sync failed. Roll back " + mutations.size() +
> " memstore keyvalues for row(s):" + StringUtils.byteToHexString(
> processor.getRowsToLock().iterator().next()) + "...");
> for (KeyValue kv : mutations) {
> getStore(kv).rollback(kv);
> }
> {code}
> There is no corresponding null check for return value of getStore() above,
> potentially leading to partially rolled back state in memstore.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11234) FastDiffDeltaEncoder#getFirstKeyInBlock returns wrong result
[
https://issues.apache.org/jira/browse/HBASE-11234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-11234:
---
Attachment: 11234-trunk.addendum
Addendum for trunk which touches PrefixTreeArrayScanner.java
> FastDiffDeltaEncoder#getFirstKeyInBlock returns wrong result
>
>
> Key: HBASE-11234
> URL: https://issues.apache.org/jira/browse/HBASE-11234
> Project: HBase
> Issue Type: Bug
>Reporter: chunhui shen
>Assignee: chunhui shen
>Priority: Critical
> Fix For: 0.99.0, 0.96.3, 0.94.21, 0.98.4
>
> Attachments: 11234-94.patch, 11234-96.patch,
> 11234-98-with-prefix-tree.txt, 11234-trunk.addendum, HBASE-11234.patch
>
>
> As Ted found,
> With this change:
> {noformat}
> Index:
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
> ===
> ---
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
>(revision 1596579)
> +++
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
>(working copy)
> @@ -51,6 +51,7 @@
> import org.apache.hadoop.hbase.filter.FilterList.Operator;
> import org.apache.hadoop.hbase.filter.PageFilter;
> import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
> +import org.apache.hadoop.hbase.io.encoding.DataBlockEncoding;
> import org.apache.hadoop.hbase.io.hfile.CacheConfig;
> import org.apache.hadoop.hbase.io.hfile.HFileContext;
> import org.apache.hadoop.hbase.io.hfile.HFileContextBuilder;
> @@ -90,6 +91,7 @@
> CacheConfig cacheConf = new CacheConfig(TEST_UTIL.getConfiguration());
> HFileContextBuilder hcBuilder = new HFileContextBuilder();
> hcBuilder.withBlockSize(2 * 1024);
> +hcBuilder.withDataBlockEncoding(DataBlockEncoding.FAST_DIFF);
> HFileContext hFileContext = hcBuilder.build();
> StoreFile.Writer writer = new StoreFile.WriterBuilder(
> TEST_UTIL.getConfiguration(), cacheConf, fs).withOutputDir(
> {noformat}
> I got:
> java.lang.AssertionError:
> expected:
> but was:
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hbase.regionserver.TestReversibleScanners.seekTestOfReversibleKeyValueScanner(TestReversibleScanners.java:533)
> at
> org.apache.hadoop.hbase.regionserver.TestReversibleScanners.testReversibleStoreFileScanner(TestReversibleScanners.java:108)
> After debugging, it seems the method of
> FastDiffDeltaEncoder#getFirstKeyInBlock become broken. And it will cause
> hfilescanner#seekBefore returns wrong result.
> The solution is simple, see the patch.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11161) Provide example of POJO encoding with protobuf
[ https://issues.apache.org/jira/browse/HBASE-11161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008447#comment-14008447 ] Hudson commented on HBASE-11161: SUCCESS: Integrated in HBase-0.98 #314 (See [https://builds.apache.org/job/HBase-0.98/314/]) HBASE-11161 Provide example of POJO encoding with protobuf (Nick Dimiduk) (apurtell: rev d73711c800415250e5543cf22db486074c8ce815) * hbase-examples/src/main/java/org/apache/hadoop/hbase/types/PBCell.java * hbase-common/src/main/java/org/apache/hadoop/hbase/types/PBType.java * hbase-examples/src/test/java/org/apache/hadoop/hbase/types/TestPBCell.java Amend HBASE-11161 Provide example of POJO encoding with protobuf; Add explicit dependency on protobuf-java to hbase-common for Hadoop 1 (apurtell: rev 503709fdf67d335fcb96f9a60a4b971de01479e1) * hbase-common/pom.xml > Provide example of POJO encoding with protobuf > -- > > Key: HBASE-11161 > URL: https://issues.apache.org/jira/browse/HBASE-11161 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.99.0, 0.98.3 > > Attachments: HBASE-11161-addendum.patch, HBASE-11161.01-0.98.patch, > HBASE-11161.01.patch, pbtype_example.patch > > > It would be nice to see how to use the DataType API with some out-of-the-box > data serialization tools. This ticket is to provide such an example using > Protobuf, since we already ship that dependency. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11137) Add mapred.TableSnapshotInputFormat
[ https://issues.apache.org/jira/browse/HBASE-11137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008443#comment-14008443 ] Hudson commented on HBASE-11137: SUCCESS: Integrated in HBase-0.98 #314 (See [https://builds.apache.org/job/HBase-0.98/314/]) Amend HBASE-11137 Add mapred.TableSnapshotInputFormat; add missing license headers on branch 0.98 (apurtell: rev 568bfe416c1c880c84dbb4a11cdbe61aea9717a2) * hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/mapred/TableSnapshotInputFormat.java > Add mapred.TableSnapshotInputFormat > --- > > Key: HBASE-11137 > URL: https://issues.apache.org/jira/browse/HBASE-11137 > Project: HBase > Issue Type: Improvement > Components: mapreduce, Performance >Affects Versions: 0.98.0, 0.96.2 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.99.0, 0.98.3 > > Attachments: HBASE-11137.00-0.98.patch, HBASE-11137.00.patch, > HBASE-11137.01.patch, HBASE-11137.01_rerun.patch, HBASE-11137.02-0.98.patch, > HBASE-11137.02.patch > > > We should have feature parity between mapreduce and mapred implementations. > This is important for Hive. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11227) Mention 8- and 16-bit fixed-with encodings in OrderedBytes docstring
[ https://issues.apache.org/jira/browse/HBASE-11227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008445#comment-14008445 ] Hudson commented on HBASE-11227: SUCCESS: Integrated in HBase-0.98 #314 (See [https://builds.apache.org/job/HBase-0.98/314/]) HBASE-11227 Mention 8- and 16-bit fixed-with encodings in OrderedBytes docstring (Nick Dimiduk) (apurtell: rev 2aed26ae6f9f3b7e961d0d92aebad41ecd2b200c) * hbase-common/src/main/java/org/apache/hadoop/hbase/util/OrderedBytes.java > Mention 8- and 16-bit fixed-with encodings in OrderedBytes docstring > > > Key: HBASE-11227 > URL: https://issues.apache.org/jira/browse/HBASE-11227 > Project: HBase > Issue Type: Task > Components: documentation >Affects Versions: 0.95.2 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Trivial > Fix For: 0.99.0, 0.98.3 > > Attachments: HBASE-11227.00.patch > > > Per title. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11149) Wire encryption is broken
[ https://issues.apache.org/jira/browse/HBASE-11149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008444#comment-14008444 ] Hudson commented on HBASE-11149: SUCCESS: Integrated in HBase-0.98 #314 (See [https://builds.apache.org/job/HBase-0.98/314/]) HBASE-11149. Addendum for fixing unit test (apurtell: rev 52dd30727a35b1b56ef2c474065ae206811d4a09) * hbase-server/src/test/java/org/apache/hadoop/hbase/security/TestHBaseSaslRpcClient.java > Wire encryption is broken > - > > Key: HBASE-11149 > URL: https://issues.apache.org/jira/browse/HBASE-11149 > Project: HBase > Issue Type: Bug > Components: IPC/RPC >Reporter: Devaraj Das >Assignee: Devaraj Das > Fix For: 0.99.0, 0.96.3, 0.98.3 > > Attachments: 11149-1.txt, 11149-2-addendum.txt, 11149-2.txt > > > Upon some testing with the QOP configuration (hbase.rpc.protection), > discovered that RPC doesn't work with "integrity" and "privacy" values for > the configuration key. I was using 0.98.x for testing but I believe the issue > is there in trunk as well (haven't checked 0.96 and 0.94). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-9857) Blockcache prefetch option
[ https://issues.apache.org/jira/browse/HBASE-9857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008442#comment-14008442 ] Hudson commented on HBASE-9857: --- SUCCESS: Integrated in HBase-0.98 #314 (See [https://builds.apache.org/job/HBase-0.98/314/]) Amend HBASE-9857 Blockcache prefetch option; add missing license header to correct file this time (apurtell: rev a260c862a71c6433213e6619c4de004250468c83) * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/PrefetchExecutor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java > Blockcache prefetch option > -- > > Key: HBASE-9857 > URL: https://issues.apache.org/jira/browse/HBASE-9857 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 0.99.0, 0.98.3 > > Attachments: 9857.patch, 9857.patch, HBASE-9857-0.98.patch, > HBASE-9857-trunk.patch, HBASE-9857-trunk.patch > > > Attached patch implements a prefetching function for HFile (v3) blocks, if > indicated by a column family or regionserver property. The purpose of this > change is to as rapidly after region open as reasonable warm the blockcache > with all the data and index blocks of (presumably also in-memory) table data, > without counting those block loads as cache misses. Great for fast reads and > keeping the cache hit ratio high. Can tune the IO impact versus time until > all data blocks are in cache. Works a bit like CompactSplitThread. Makes some > effort not to stampede. > I have been using this for setting up various experiments and thought I'd > polish it up a bit and throw it out there. If the data to be preloaded will > not fit in blockcache, or if as a percentage of blockcache it is large, this > is not a good idea, will just blow out the cache and trigger a lot of useless > GC activity. Might be useful as an expert tuning option though. Or not. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11234) FastDiffDeltaEncoder#getFirstKeyInBlock returns wrong result
[
https://issues.apache.org/jira/browse/HBASE-11234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008446#comment-14008446
]
Hudson commented on HBASE-11234:
SUCCESS: Integrated in HBase-0.98 #314 (See
[https://builds.apache.org/job/HBase-0.98/314/])
Revert "HBASE-11234 FastDiffDeltaEncoder#getFirstKeyInBlock returns wrong
result" (apurtell: rev b985e4fd0ebdd920b5c3b33dd5549713d82fa16a)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
*
hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/PrefixKeyDeltaEncoder.java
*
hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/FastDiffDeltaEncoder.java
> FastDiffDeltaEncoder#getFirstKeyInBlock returns wrong result
>
>
> Key: HBASE-11234
> URL: https://issues.apache.org/jira/browse/HBASE-11234
> Project: HBase
> Issue Type: Bug
>Reporter: chunhui shen
>Assignee: chunhui shen
>Priority: Critical
> Fix For: 0.99.0, 0.96.3, 0.94.21, 0.98.4
>
> Attachments: 11234-94.patch, 11234-96.patch,
> 11234-98-with-prefix-tree.txt, 11234-trunk.addendum, HBASE-11234.patch
>
>
> As Ted found,
> With this change:
> {noformat}
> Index:
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
> ===
> ---
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
>(revision 1596579)
> +++
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
>(working copy)
> @@ -51,6 +51,7 @@
> import org.apache.hadoop.hbase.filter.FilterList.Operator;
> import org.apache.hadoop.hbase.filter.PageFilter;
> import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
> +import org.apache.hadoop.hbase.io.encoding.DataBlockEncoding;
> import org.apache.hadoop.hbase.io.hfile.CacheConfig;
> import org.apache.hadoop.hbase.io.hfile.HFileContext;
> import org.apache.hadoop.hbase.io.hfile.HFileContextBuilder;
> @@ -90,6 +91,7 @@
> CacheConfig cacheConf = new CacheConfig(TEST_UTIL.getConfiguration());
> HFileContextBuilder hcBuilder = new HFileContextBuilder();
> hcBuilder.withBlockSize(2 * 1024);
> +hcBuilder.withDataBlockEncoding(DataBlockEncoding.FAST_DIFF);
> HFileContext hFileContext = hcBuilder.build();
> StoreFile.Writer writer = new StoreFile.WriterBuilder(
> TEST_UTIL.getConfiguration(), cacheConf, fs).withOutputDir(
> {noformat}
> I got:
> java.lang.AssertionError:
> expected:
> but was:
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hbase.regionserver.TestReversibleScanners.seekTestOfReversibleKeyValueScanner(TestReversibleScanners.java:533)
> at
> org.apache.hadoop.hbase.regionserver.TestReversibleScanners.testReversibleStoreFileScanner(TestReversibleScanners.java:108)
> After debugging, it seems the method of
> FastDiffDeltaEncoder#getFirstKeyInBlock become broken. And it will cause
> hfilescanner#seekBefore returns wrong result.
> The solution is simple, see the patch.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11149) Wire encryption is broken
[ https://issues.apache.org/jira/browse/HBASE-11149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008454#comment-14008454 ] Hudson commented on HBASE-11149: SUCCESS: Integrated in HBase-TRUNK #5140 (See [https://builds.apache.org/job/HBase-TRUNK/5140/]) HBASE-11149. Addendum for fixing unit test (ddas: rev e9017586ca16d428d2d596bbe816fbaf2467e20c) * hbase-server/src/test/java/org/apache/hadoop/hbase/security/TestHBaseSaslRpcClient.java > Wire encryption is broken > - > > Key: HBASE-11149 > URL: https://issues.apache.org/jira/browse/HBASE-11149 > Project: HBase > Issue Type: Bug > Components: IPC/RPC >Reporter: Devaraj Das >Assignee: Devaraj Das > Fix For: 0.99.0, 0.96.3, 0.98.3 > > Attachments: 11149-1.txt, 11149-2-addendum.txt, 11149-2.txt > > > Upon some testing with the QOP configuration (hbase.rpc.protection), > discovered that RPC doesn't work with "integrity" and "privacy" values for > the configuration key. I was using 0.98.x for testing but I believe the issue > is there in trunk as well (haven't checked 0.96 and 0.94). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11161) Provide example of POJO encoding with protobuf
[ https://issues.apache.org/jira/browse/HBASE-11161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008456#comment-14008456 ] Hudson commented on HBASE-11161: SUCCESS: Integrated in HBase-TRUNK #5140 (See [https://builds.apache.org/job/HBase-TRUNK/5140/]) HBASE-11161 Provide example of POJO encoding with protobuf (Nick Dimiduk) (apurtell: rev 8b145419edac5d5d29e0e7d5718f60635e355500) * hbase-examples/src/test/java/org/apache/hadoop/hbase/types/TestPBCell.java * hbase-examples/src/main/java/org/apache/hadoop/hbase/types/PBCell.java * hbase-common/src/main/java/org/apache/hadoop/hbase/types/PBType.java Amend HBASE-11161 Provide example of POJO encoding with protobuf; Add explicit dependency on protobuf-java to hbase-common for Hadoop 1 (apurtell: rev b85cbbdea920525f9c019c0ac9187590943fcc27) * hbase-common/pom.xml > Provide example of POJO encoding with protobuf > -- > > Key: HBASE-11161 > URL: https://issues.apache.org/jira/browse/HBASE-11161 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.99.0, 0.98.3 > > Attachments: HBASE-11161-addendum.patch, HBASE-11161.01-0.98.patch, > HBASE-11161.01.patch, pbtype_example.patch > > > It would be nice to see how to use the DataType API with some out-of-the-box > data serialization tools. This ticket is to provide such an example using > Protobuf, since we already ship that dependency. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-9857) Blockcache prefetch option
[ https://issues.apache.org/jira/browse/HBASE-9857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008453#comment-14008453 ] Hudson commented on HBASE-9857: --- SUCCESS: Integrated in HBase-TRUNK #5140 (See [https://builds.apache.org/job/HBase-TRUNK/5140/]) Amend HBASE-9857 Blockcache prefetch option; add missing license header to correct file this time (apurtell: rev be85f89cd4508de5337820a2694d5262c6d69092) * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/PrefetchExecutor.java > Blockcache prefetch option > -- > > Key: HBASE-9857 > URL: https://issues.apache.org/jira/browse/HBASE-9857 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 0.99.0, 0.98.3 > > Attachments: 9857.patch, 9857.patch, HBASE-9857-0.98.patch, > HBASE-9857-trunk.patch, HBASE-9857-trunk.patch > > > Attached patch implements a prefetching function for HFile (v3) blocks, if > indicated by a column family or regionserver property. The purpose of this > change is to as rapidly after region open as reasonable warm the blockcache > with all the data and index blocks of (presumably also in-memory) table data, > without counting those block loads as cache misses. Great for fast reads and > keeping the cache hit ratio high. Can tune the IO impact versus time until > all data blocks are in cache. Works a bit like CompactSplitThread. Makes some > effort not to stampede. > I have been using this for setting up various experiments and thought I'd > polish it up a bit and throw it out there. If the data to be preloaded will > not fit in blockcache, or if as a percentage of blockcache it is large, this > is not a good idea, will just blow out the cache and trigger a lot of useless > GC activity. Might be useful as an expert tuning option though. Or not. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11227) Mention 8- and 16-bit fixed-with encodings in OrderedBytes docstring
[ https://issues.apache.org/jira/browse/HBASE-11227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008455#comment-14008455 ] Hudson commented on HBASE-11227: SUCCESS: Integrated in HBase-TRUNK #5140 (See [https://builds.apache.org/job/HBase-TRUNK/5140/]) HBASE-11227 Mention 8- and 16-bit fixed-with encodings in OrderedBytes docstring (Nick Dimiduk) (apurtell: rev 30aab8b5ea667b97f7a161893cf98b82d7f34fc6) * hbase-common/src/main/java/org/apache/hadoop/hbase/util/OrderedBytes.java > Mention 8- and 16-bit fixed-with encodings in OrderedBytes docstring > > > Key: HBASE-11227 > URL: https://issues.apache.org/jira/browse/HBASE-11227 > Project: HBase > Issue Type: Task > Components: documentation >Affects Versions: 0.95.2 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Trivial > Fix For: 0.99.0, 0.98.3 > > Attachments: HBASE-11227.00.patch > > > Per title. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11238) Add info about SlabCache and BucketCache to Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008457#comment-14008457 ] Misty Stanley-Jones commented on HBASE-11238: - My thinking was that this is a config setting, not a development setting. I wouldn't normally look in Javadoc for a config setting. But maybe it's different in HBase-land? If you want to keep it in Javadoc, do you like my version or have I lost some of the meaning in the rewrite? > Add info about SlabCache and BucketCache to Ref Guide > - > > Key: HBASE-11238 > URL: https://issues.apache.org/jira/browse/HBASE-11238 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 0.98.2 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11238.patch > > > Upstream issues: HBASE-11171 and HBASE-11098. Could back port some of what is > in these issues, the package-info.java class for instance. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (HBASE-11250) Document: CLONE - Fix jersey serialization/deserialization of json objects
[ https://issues.apache.org/jira/browse/HBASE-11250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones reassigned HBASE-11250: --- Assignee: Misty Stanley-Jones (was: Francis Liu) > Document: CLONE - Fix jersey serialization/deserialization of json objects > -- > > Key: HBASE-11250 > URL: https://issues.apache.org/jira/browse/HBASE-11250 > Project: HBase > Issue Type: Bug > Components: documentation, REST >Affects Versions: 0.92.2 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Blocker > > Stargate uses the default json marshaller/unmarshaller in natural mode. In > this mode the unmarshaller has trouble unmarshalling json instances. > This patch fixes this issue by using jackson as the marshaller/unmarshaller > instead. > I've also updated all the model unit tests to test json > serialization/deserialization. Backwards compatibilty can be verified by > modify the test base class to use the original marshaller/unmarshaller and > see that model tests pass. > The patch is backward compatible except for StorageClusterStatusModel, which > is broken anyway. It only shows one node in the liveNodes field. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11250) Document: CLONE - Fix jersey serialization/deserialization of json objects
Misty Stanley-Jones created HBASE-11250: --- Summary: Document: CLONE - Fix jersey serialization/deserialization of json objects Key: HBASE-11250 URL: https://issues.apache.org/jira/browse/HBASE-11250 Project: HBase Issue Type: Bug Components: REST Reporter: Misty Stanley-Jones Assignee: Francis Liu Priority: Blocker Fix For: 0.98.0, 0.96.0 Stargate uses the default json marshaller/unmarshaller in natural mode. In this mode the unmarshaller has trouble unmarshalling json instances. This patch fixes this issue by using jackson as the marshaller/unmarshaller instead. I've also updated all the model unit tests to test json serialization/deserialization. Backwards compatibilty can be verified by modify the test base class to use the original marshaller/unmarshaller and see that model tests pass. The patch is backward compatible except for StorageClusterStatusModel, which is broken anyway. It only shows one node in the liveNodes field. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11250) Document: CLONE - Fix jersey serialization/deserialization of json objects
[ https://issues.apache.org/jira/browse/HBASE-11250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11250: Fix Version/s: (was: 0.96.0) (was: 0.98.0) > Document: CLONE - Fix jersey serialization/deserialization of json objects > -- > > Key: HBASE-11250 > URL: https://issues.apache.org/jira/browse/HBASE-11250 > Project: HBase > Issue Type: Bug > Components: documentation, REST >Affects Versions: 0.92.2 >Reporter: Misty Stanley-Jones >Assignee: Francis Liu >Priority: Blocker > > Stargate uses the default json marshaller/unmarshaller in natural mode. In > this mode the unmarshaller has trouble unmarshalling json instances. > This patch fixes this issue by using jackson as the marshaller/unmarshaller > instead. > I've also updated all the model unit tests to test json > serialization/deserialization. Backwards compatibilty can be verified by > modify the test base class to use the original marshaller/unmarshaller and > see that model tests pass. > The patch is backward compatible except for StorageClusterStatusModel, which > is broken anyway. It only shows one node in the liveNodes field. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11250) Document: CLONE - Fix jersey serialization/deserialization of json objects
[ https://issues.apache.org/jira/browse/HBASE-11250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11250: Affects Version/s: 0.92.2 > Document: CLONE - Fix jersey serialization/deserialization of json objects > -- > > Key: HBASE-11250 > URL: https://issues.apache.org/jira/browse/HBASE-11250 > Project: HBase > Issue Type: Bug > Components: documentation, REST >Affects Versions: 0.92.2 >Reporter: Misty Stanley-Jones >Assignee: Francis Liu >Priority: Blocker > > Stargate uses the default json marshaller/unmarshaller in natural mode. In > this mode the unmarshaller has trouble unmarshalling json instances. > This patch fixes this issue by using jackson as the marshaller/unmarshaller > instead. > I've also updated all the model unit tests to test json > serialization/deserialization. Backwards compatibilty can be verified by > modify the test base class to use the original marshaller/unmarshaller and > see that model tests pass. > The patch is backward compatible except for StorageClusterStatusModel, which > is broken anyway. It only shows one node in the liveNodes field. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11250) Document: CLONE - Fix jersey serialization/deserialization of json objects
[ https://issues.apache.org/jira/browse/HBASE-11250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11250: Component/s: documentation > Document: CLONE - Fix jersey serialization/deserialization of json objects > -- > > Key: HBASE-11250 > URL: https://issues.apache.org/jira/browse/HBASE-11250 > Project: HBase > Issue Type: Bug > Components: documentation, REST >Affects Versions: 0.92.2 >Reporter: Misty Stanley-Jones >Assignee: Francis Liu >Priority: Blocker > > Stargate uses the default json marshaller/unmarshaller in natural mode. In > this mode the unmarshaller has trouble unmarshalling json instances. > This patch fixes this issue by using jackson as the marshaller/unmarshaller > instead. > I've also updated all the model unit tests to test json > serialization/deserialization. Backwards compatibilty can be verified by > modify the test base class to use the original marshaller/unmarshaller and > see that model tests pass. > The patch is backward compatible except for StorageClusterStatusModel, which > is broken anyway. It only shows one node in the liveNodes field. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11226) Document and increase the default value for hbase.hstore.flusher.count
[ https://issues.apache.org/jira/browse/HBASE-11226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008467#comment-14008467 ] Hudson commented on HBASE-11226: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #295 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/295/]) HBASE-11226 Document and increase the default value for hbase.hstore.flusher.count (nkeywal: rev 996705c61de1c7b533024dcb41cc571531ebfbb9) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java * hbase-common/src/main/resources/hbase-default.xml > Document and increase the default value for hbase.hstore.flusher.count > -- > > Key: HBASE-11226 > URL: https://issues.apache.org/jira/browse/HBASE-11226 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.99.0, 0.98.2 >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon > Fix For: 0.99.0, 0.98.3 > > Attachments: 11226.v1.patch > > > HBASE-6466 add the possibility to have multiple threads for the flusher. > The default value is 1, but this should be incremented to 2 reasons: > - I've observed that the flush of a region can be delayed because another is > in progress. During a write load, this leads to an increased latency because > the memstore size increases and then block the client > - if, by accident, a flusher hits a slow or bad datanode, all the flush will > have to wait until the timeouts expires. With 2 or more flushers and some > luck the other regions will be flushed by the second thread. > Lastly this setting is important enough to be documented in the standard > hbase site imho. > I think it's important enough to go in the .98 branch as well > There will be an impact: as the flush won't be queued (or less queued) we may > have more compactions... -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11161) Provide example of POJO encoding with protobuf
[ https://issues.apache.org/jira/browse/HBASE-11161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008471#comment-14008471 ] Hudson commented on HBASE-11161: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #295 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/295/]) HBASE-11161 Provide example of POJO encoding with protobuf (Nick Dimiduk) (apurtell: rev d73711c800415250e5543cf22db486074c8ce815) * hbase-common/src/main/java/org/apache/hadoop/hbase/types/PBType.java * hbase-examples/src/main/java/org/apache/hadoop/hbase/types/PBCell.java * hbase-examples/src/test/java/org/apache/hadoop/hbase/types/TestPBCell.java Amend HBASE-11161 Provide example of POJO encoding with protobuf; Add explicit dependency on protobuf-java to hbase-common for Hadoop 1 (apurtell: rev 503709fdf67d335fcb96f9a60a4b971de01479e1) * hbase-common/pom.xml > Provide example of POJO encoding with protobuf > -- > > Key: HBASE-11161 > URL: https://issues.apache.org/jira/browse/HBASE-11161 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.99.0, 0.98.3 > > Attachments: HBASE-11161-addendum.patch, HBASE-11161.01-0.98.patch, > HBASE-11161.01.patch, pbtype_example.patch > > > It would be nice to see how to use the DataType API with some out-of-the-box > data serialization tools. This ticket is to provide such an example using > Protobuf, since we already ship that dependency. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11137) Add mapred.TableSnapshotInputFormat
[ https://issues.apache.org/jira/browse/HBASE-11137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008466#comment-14008466 ] Hudson commented on HBASE-11137: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #295 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/295/]) Amend HBASE-11137 Add mapred.TableSnapshotInputFormat; add missing license headers on branch 0.98 (apurtell: rev 568bfe416c1c880c84dbb4a11cdbe61aea9717a2) * hbase-server/src/main/java/org/apache/hadoop/hbase/mapred/TableSnapshotInputFormat.java * hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java > Add mapred.TableSnapshotInputFormat > --- > > Key: HBASE-11137 > URL: https://issues.apache.org/jira/browse/HBASE-11137 > Project: HBase > Issue Type: Improvement > Components: mapreduce, Performance >Affects Versions: 0.98.0, 0.96.2 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.99.0, 0.98.3 > > Attachments: HBASE-11137.00-0.98.patch, HBASE-11137.00.patch, > HBASE-11137.01.patch, HBASE-11137.01_rerun.patch, HBASE-11137.02-0.98.patch, > HBASE-11137.02.patch > > > We should have feature parity between mapreduce and mapred implementations. > This is important for Hive. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11149) Wire encryption is broken
[ https://issues.apache.org/jira/browse/HBASE-11149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008468#comment-14008468 ] Hudson commented on HBASE-11149: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #295 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/295/]) HBASE-11149. Addendum for fixing unit test (apurtell: rev 52dd30727a35b1b56ef2c474065ae206811d4a09) * hbase-server/src/test/java/org/apache/hadoop/hbase/security/TestHBaseSaslRpcClient.java > Wire encryption is broken > - > > Key: HBASE-11149 > URL: https://issues.apache.org/jira/browse/HBASE-11149 > Project: HBase > Issue Type: Bug > Components: IPC/RPC >Reporter: Devaraj Das >Assignee: Devaraj Das > Fix For: 0.99.0, 0.96.3, 0.98.3 > > Attachments: 11149-1.txt, 11149-2-addendum.txt, 11149-2.txt > > > Upon some testing with the QOP configuration (hbase.rpc.protection), > discovered that RPC doesn't work with "integrity" and "privacy" values for > the configuration key. I was using 0.98.x for testing but I believe the issue > is there in trunk as well (haven't checked 0.96 and 0.94). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-9857) Blockcache prefetch option
[ https://issues.apache.org/jira/browse/HBASE-9857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008465#comment-14008465 ] Hudson commented on HBASE-9857: --- SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #295 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/295/]) Amend HBASE-9857 Blockcache prefetch option; add missing license header to correct file this time (apurtell: rev a260c862a71c6433213e6619c4de004250468c83) * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/PrefetchExecutor.java > Blockcache prefetch option > -- > > Key: HBASE-9857 > URL: https://issues.apache.org/jira/browse/HBASE-9857 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 0.99.0, 0.98.3 > > Attachments: 9857.patch, 9857.patch, HBASE-9857-0.98.patch, > HBASE-9857-trunk.patch, HBASE-9857-trunk.patch > > > Attached patch implements a prefetching function for HFile (v3) blocks, if > indicated by a column family or regionserver property. The purpose of this > change is to as rapidly after region open as reasonable warm the blockcache > with all the data and index blocks of (presumably also in-memory) table data, > without counting those block loads as cache misses. Great for fast reads and > keeping the cache hit ratio high. Can tune the IO impact versus time until > all data blocks are in cache. Works a bit like CompactSplitThread. Makes some > effort not to stampede. > I have been using this for setting up various experiments and thought I'd > polish it up a bit and throw it out there. If the data to be preloaded will > not fit in blockcache, or if as a percentage of blockcache it is large, this > is not a good idea, will just blow out the cache and trigger a lot of useless > GC activity. Might be useful as an expert tuning option though. Or not. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11227) Mention 8- and 16-bit fixed-with encodings in OrderedBytes docstring
[ https://issues.apache.org/jira/browse/HBASE-11227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008469#comment-14008469 ] Hudson commented on HBASE-11227: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #295 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/295/]) HBASE-11227 Mention 8- and 16-bit fixed-with encodings in OrderedBytes docstring (Nick Dimiduk) (apurtell: rev 2aed26ae6f9f3b7e961d0d92aebad41ecd2b200c) * hbase-common/src/main/java/org/apache/hadoop/hbase/util/OrderedBytes.java > Mention 8- and 16-bit fixed-with encodings in OrderedBytes docstring > > > Key: HBASE-11227 > URL: https://issues.apache.org/jira/browse/HBASE-11227 > Project: HBase > Issue Type: Task > Components: documentation >Affects Versions: 0.95.2 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Trivial > Fix For: 0.99.0, 0.98.3 > > Attachments: HBASE-11227.00.patch > > > Per title. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11209) Increase the default value for hbase.hregion.memstore.block.multipler from 2 to 4
[ https://issues.apache.org/jira/browse/HBASE-11209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008464#comment-14008464 ] Hudson commented on HBASE-11209: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #295 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/295/]) HBASE-11209 Increase the default value for hbase.hregion.memstore.block.multipler from 2 to 4 (nkeywal: rev 5521dfe6f05c8f49f2ebfa1c54b80169f2296a83) * hbase-common/src/main/resources/hbase-default.xml > Increase the default value for hbase.hregion.memstore.block.multipler from 2 > to 4 > - > > Key: HBASE-11209 > URL: https://issues.apache.org/jira/browse/HBASE-11209 > Project: HBase > Issue Type: Brainstorming > Components: regionserver >Affects Versions: 0.99.0, 0.98.2 >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon > Fix For: 0.99.0, 0.98.3 > > Attachments: HBASE-11209.patch > > > On a YCSB test, I saw a 33% performance increase, both on the max latency and > on the throughput. I'm convinced enough that this value is better that I > think it makes sense to change it on 0.98 as well. > More fundamentally, but outside of the scope of this patch, I think this > parameter should be changed to something at the region server level: today, > we have: > - global memstore check: if we're other 40%, we flush the biggest memstore > - local: no more than 2 (proposed: 4) memstore size per region. > But if we have enough memory and a spike on a region, there is no reason for > not taking the write. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11234) FastDiffDeltaEncoder#getFirstKeyInBlock returns wrong result
[
https://issues.apache.org/jira/browse/HBASE-11234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008470#comment-14008470
]
Hudson commented on HBASE-11234:
SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #295 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/295/])
Revert "HBASE-11234 FastDiffDeltaEncoder#getFirstKeyInBlock returns wrong
result" (apurtell: rev b985e4fd0ebdd920b5c3b33dd5549713d82fa16a)
*
hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/FastDiffDeltaEncoder.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
*
hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/PrefixKeyDeltaEncoder.java
> FastDiffDeltaEncoder#getFirstKeyInBlock returns wrong result
>
>
> Key: HBASE-11234
> URL: https://issues.apache.org/jira/browse/HBASE-11234
> Project: HBase
> Issue Type: Bug
>Reporter: chunhui shen
>Assignee: chunhui shen
>Priority: Critical
> Fix For: 0.99.0, 0.96.3, 0.94.21, 0.98.4
>
> Attachments: 11234-94.patch, 11234-96.patch,
> 11234-98-with-prefix-tree.txt, 11234-trunk.addendum, HBASE-11234.patch
>
>
> As Ted found,
> With this change:
> {noformat}
> Index:
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
> ===
> ---
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
>(revision 1596579)
> +++
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
>(working copy)
> @@ -51,6 +51,7 @@
> import org.apache.hadoop.hbase.filter.FilterList.Operator;
> import org.apache.hadoop.hbase.filter.PageFilter;
> import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
> +import org.apache.hadoop.hbase.io.encoding.DataBlockEncoding;
> import org.apache.hadoop.hbase.io.hfile.CacheConfig;
> import org.apache.hadoop.hbase.io.hfile.HFileContext;
> import org.apache.hadoop.hbase.io.hfile.HFileContextBuilder;
> @@ -90,6 +91,7 @@
> CacheConfig cacheConf = new CacheConfig(TEST_UTIL.getConfiguration());
> HFileContextBuilder hcBuilder = new HFileContextBuilder();
> hcBuilder.withBlockSize(2 * 1024);
> +hcBuilder.withDataBlockEncoding(DataBlockEncoding.FAST_DIFF);
> HFileContext hFileContext = hcBuilder.build();
> StoreFile.Writer writer = new StoreFile.WriterBuilder(
> TEST_UTIL.getConfiguration(), cacheConf, fs).withOutputDir(
> {noformat}
> I got:
> java.lang.AssertionError:
> expected:
> but was:
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hbase.regionserver.TestReversibleScanners.seekTestOfReversibleKeyValueScanner(TestReversibleScanners.java:533)
> at
> org.apache.hadoop.hbase.regionserver.TestReversibleScanners.testReversibleStoreFileScanner(TestReversibleScanners.java:108)
> After debugging, it seems the method of
> FastDiffDeltaEncoder#getFirstKeyInBlock become broken. And it will cause
> hfilescanner#seekBefore returns wrong result.
> The solution is simple, see the patch.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11250) Document: CLONE - Fix jersey serialization/deserialization of json objects
[ https://issues.apache.org/jira/browse/HBASE-11250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008473#comment-14008473 ] Misty Stanley-Jones commented on HBASE-11250: - This should be updated at the Wiki: http://wiki.apache.org/hadoop/Hbase/Stargate > Document: CLONE - Fix jersey serialization/deserialization of json objects > -- > > Key: HBASE-11250 > URL: https://issues.apache.org/jira/browse/HBASE-11250 > Project: HBase > Issue Type: Bug > Components: documentation, REST >Affects Versions: 0.92.2 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Blocker > > Stargate uses the default json marshaller/unmarshaller in natural mode. In > this mode the unmarshaller has trouble unmarshalling json instances. > This patch fixes this issue by using jackson as the marshaller/unmarshaller > instead. > I've also updated all the model unit tests to test json > serialization/deserialization. Backwards compatibilty can be verified by > modify the test base class to use the original marshaller/unmarshaller and > see that model tests pass. > The patch is backward compatible except for StorageClusterStatusModel, which > is broken anyway. It only shows one node in the liveNodes field. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11218) Data loss in HBase standalone mode
[ https://issues.apache.org/jira/browse/HBASE-11218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008526#comment-14008526 ] Liu Shaohui commented on HBASE-11218: - [~nkeywal] The comparison done in my dev machine is following ||case||trunk||HBASE-11218|| |small tests|6m32.725s|6m34.124s| |medium tests|91m23.238s|93m6.703s| |large tests|154m58.474s|150m42.585s| The test time increase is very little. > Data loss in HBase standalone mode > -- > > Key: HBASE-11218 > URL: https://issues.apache.org/jira/browse/HBASE-11218 > Project: HBase > Issue Type: Bug >Reporter: Liu Shaohui >Assignee: Liu Shaohui > Fix For: 0.99.0 > > Attachments: HBASE-11218-trunk-v1.diff > > > Data loss in HBase standalone mode. > *How to produce it* > # Start HBase standalone mode. > # Create a table using hbase shell. > # Scan '.META.' and you will find data in meta table > # Kill the HBase process with -9 option > # Start the HBase agaion > # Scan '.META.' and you will find nothing in meta table. > *There are three main reasons.* > # FSDataOutputStream.sync should call flush() if the underlying wrapped > stream is not Syncable. See HADOOP-8861 > # writeChecksum is ture in default LocalFileSystem and the > ChecksumFSOutputSummer will buffer the data, which make the waledits are not > written to os's filesystem with sync method immediately, and those edits will > be lost in regionserver's failover. > # The MiniZooKeeperCluster deletes the old zk data at startup which maye > cause data loss in meta table. The failover procedure is: split pre root > regionserver's hlog -> assign root -> split pre meta regionserver's hlog -> > assign meta -> split all other regionservers' hlogs -> assign other regions. > If there is no data in zookeeper, we will get null for root regionserver and > then assign root table. Some data in root table maybe be lost for some root's > WalEdits have not been splited and replayed. So does the Meta table. > I finished the patch for 0.94 and am working on the patch for trunk. > Suggestions are welcomed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11218) Data loss in HBase standalone mode
[
https://issues.apache.org/jira/browse/HBASE-11218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008527#comment-14008527
]
Liu Shaohui commented on HBASE-11218:
-
[~enis]
{quote}
in pseudo distributed mode, where there is an actual RS, but still using the
local fs, there still will be data loss with this patch from my understanding
{quote}
Yes. but I think the data loss in pseudo distributed mode caused by hardware
failure is accepted, data loss caused by the HBase's implement is not accepted
and we should fix it.
> Data loss in HBase standalone mode
> --
>
> Key: HBASE-11218
> URL: https://issues.apache.org/jira/browse/HBASE-11218
> Project: HBase
> Issue Type: Bug
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
> Fix For: 0.99.0
>
> Attachments: HBASE-11218-trunk-v1.diff
>
>
> Data loss in HBase standalone mode.
> *How to produce it*
> # Start HBase standalone mode.
> # Create a table using hbase shell.
> # Scan '.META.' and you will find data in meta table
> # Kill the HBase process with -9 option
> # Start the HBase agaion
> # Scan '.META.' and you will find nothing in meta table.
> *There are three main reasons.*
> # FSDataOutputStream.sync should call flush() if the underlying wrapped
> stream is not Syncable. See HADOOP-8861
> # writeChecksum is ture in default LocalFileSystem and the
> ChecksumFSOutputSummer will buffer the data, which make the waledits are not
> written to os's filesystem with sync method immediately, and those edits will
> be lost in regionserver's failover.
> # The MiniZooKeeperCluster deletes the old zk data at startup which maye
> cause data loss in meta table. The failover procedure is: split pre root
> regionserver's hlog -> assign root -> split pre meta regionserver's hlog ->
> assign meta -> split all other regionservers' hlogs -> assign other regions.
> If there is no data in zookeeper, we will get null for root regionserver and
> then assign root table. Some data in root table maybe be lost for some root's
> WalEdits have not been splited and replayed. So does the Meta table.
> I finished the patch for 0.94 and am working on the patch for trunk.
> Suggestions are welcomed.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-10771) Primitive type put/get APIs in ByteRange
[ https://issues.apache.org/jira/browse/HBASE-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-10771: --- Attachment: HBASE-10771_V4.patch Andy helped me to run a test for checking the inlining of put/getxxx() APIs. As per that org.apache.hadoop.hbase.util.Bytes#explainWrongLengthOrOffset () is too big for inlining. The patch changes the getxxx() API to not to use Bytes#putxxx API. That code is copy pasted to SimpleByteRange without the sanity checking. Also addressed comment about adding a test where lots of KV are written and read back from a PBR. > Primitive type put/get APIs in ByteRange > - > > Key: HBASE-10771 > URL: https://issues.apache.org/jira/browse/HBASE-10771 > Project: HBase > Issue Type: Improvement >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 0.99.0 > > Attachments: HBASE-10771.patch, HBASE-10771_V2.patch, > HBASE-10771_V3.patch, HBASE-10771_V4.patch > > > While doing HBASE-10713 I came across the need to write int/long (and read > also) from a ByteRange. CellBlocks are backed by ByteRange. So we can add > such APIs. > Also as per HBASE-10750 we return a ByteRange from MSLAB and also discussion > under HBASE-10191 suggest we can have BR backed HFileBlocks etc. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11251) LoadTestTool should grant READ permission for the users that are given READ access
ramkrishna.s.vasudevan created HBASE-11251:
--
Summary: LoadTestTool should grant READ permission for the users
that are given READ access
Key: HBASE-11251
URL: https://issues.apache.org/jira/browse/HBASE-11251
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.2
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.98.3
In 0.98.2 onwards the AccessControlFilter
{code}
case CHECK_CELL_FIRST: {
LOG.info("Am coming here for cell first strategy");
if (authManager.authorize(user, table, cell, Permission.Action.READ) &&
authManager.authorize(user, table, family, qualifier,
Permission.Action.READ)) {
LOG.info("Returning include");
return ReturnCode.INCLUDE;
}
{code}
expects a READ permission on the table for those Users that are granted READ
permission on the cell level.
In 0.98.1
{code}
if (authManager.authorize(user, table, cell, cellFirstStrategy,
Permission.Action.READ)) {
return ReturnCode.INCLUDE;
}
{code}
So from 0.98.2 onwards IntegrationTestIngestWithACL was failing. Hence this
JIRA is targeted to correct the behaviour and make the IT work again.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11251) LoadTestTool should grant READ permission for the users that are given READ access for specific cells
[
https://issues.apache.org/jira/browse/HBASE-11251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-11251:
---
Summary: LoadTestTool should grant READ permission for the users that are
given READ access for specific cells (was: LoadTestTool should grant READ
permission for the users that are given READ access)
> LoadTestTool should grant READ permission for the users that are given READ
> access for specific cells
> -
>
> Key: HBASE-11251
> URL: https://issues.apache.org/jira/browse/HBASE-11251
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.2
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.98.3
>
>
> In 0.98.2 onwards the AccessControlFilter
> {code}
> case CHECK_CELL_FIRST: {
> LOG.info("Am coming here for cell first strategy");
> if (authManager.authorize(user, table, cell, Permission.Action.READ)
> &&
> authManager.authorize(user, table, family, qualifier,
> Permission.Action.READ)) {
> LOG.info("Returning include");
> return ReturnCode.INCLUDE;
> }
> {code}
> expects a READ permission on the table for those Users that are granted READ
> permission on the cell level.
> In 0.98.1
> {code}
> if (authManager.authorize(user, table, cell, cellFirstStrategy,
> Permission.Action.READ)) {
> return ReturnCode.INCLUDE;
> }
> {code}
> So from 0.98.2 onwards IntegrationTestIngestWithACL was failing. Hence this
> JIRA is targeted to correct the behaviour and make the IT work again.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11251) LoadTestTool should grant READ permission for the users that are given READ access for specific cells
[
https://issues.apache.org/jira/browse/HBASE-11251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-11251:
---
Fix Version/s: 0.99.0
> LoadTestTool should grant READ permission for the users that are given READ
> access for specific cells
> -
>
> Key: HBASE-11251
> URL: https://issues.apache.org/jira/browse/HBASE-11251
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.2
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0, 0.98.3
>
>
> In 0.98.2 onwards the AccessControlFilter
> {code}
> case CHECK_CELL_FIRST: {
> LOG.info("Am coming here for cell first strategy");
> if (authManager.authorize(user, table, cell, Permission.Action.READ)
> &&
> authManager.authorize(user, table, family, qualifier,
> Permission.Action.READ)) {
> LOG.info("Returning include");
> return ReturnCode.INCLUDE;
> }
> {code}
> expects a READ permission on the table for those Users that are granted READ
> permission on the cell level.
> In 0.98.1
> {code}
> if (authManager.authorize(user, table, cell, cellFirstStrategy,
> Permission.Action.READ)) {
> return ReturnCode.INCLUDE;
> }
> {code}
> So from 0.98.2 onwards IntegrationTestIngestWithACL was failing. Hence this
> JIRA is targeted to correct the behaviour and make the IT work again.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11251) LoadTestTool should grant READ permission for the users that are given READ access for specific cells
[
https://issues.apache.org/jira/browse/HBASE-11251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-11251:
---
Status: Patch Available (was: Open)
> LoadTestTool should grant READ permission for the users that are given READ
> access for specific cells
> -
>
> Key: HBASE-11251
> URL: https://issues.apache.org/jira/browse/HBASE-11251
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.2
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0, 0.98.3
>
> Attachments: HBASE-11251_0.98.patch, HBASE-11251_trunk.patch
>
>
> In 0.98.2 onwards the AccessControlFilter
> {code}
> case CHECK_CELL_FIRST: {
> LOG.info("Am coming here for cell first strategy");
> if (authManager.authorize(user, table, cell, Permission.Action.READ)
> &&
> authManager.authorize(user, table, family, qualifier,
> Permission.Action.READ)) {
> LOG.info("Returning include");
> return ReturnCode.INCLUDE;
> }
> {code}
> expects a READ permission on the table for those Users that are granted READ
> permission on the cell level.
> In 0.98.1
> {code}
> if (authManager.authorize(user, table, cell, cellFirstStrategy,
> Permission.Action.READ)) {
> return ReturnCode.INCLUDE;
> }
> {code}
> So from 0.98.2 onwards IntegrationTestIngestWithACL was failing. Hence this
> JIRA is targeted to correct the behaviour and make the IT work again.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11251) LoadTestTool should grant READ permission for the users that are given READ access for specific cells
[
https://issues.apache.org/jira/browse/HBASE-11251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-11251:
---
Attachment: HBASE-11251_trunk.patch
> LoadTestTool should grant READ permission for the users that are given READ
> access for specific cells
> -
>
> Key: HBASE-11251
> URL: https://issues.apache.org/jira/browse/HBASE-11251
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.2
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0, 0.98.3
>
> Attachments: HBASE-11251_0.98.patch, HBASE-11251_trunk.patch
>
>
> In 0.98.2 onwards the AccessControlFilter
> {code}
> case CHECK_CELL_FIRST: {
> LOG.info("Am coming here for cell first strategy");
> if (authManager.authorize(user, table, cell, Permission.Action.READ)
> &&
> authManager.authorize(user, table, family, qualifier,
> Permission.Action.READ)) {
> LOG.info("Returning include");
> return ReturnCode.INCLUDE;
> }
> {code}
> expects a READ permission on the table for those Users that are granted READ
> permission on the cell level.
> In 0.98.1
> {code}
> if (authManager.authorize(user, table, cell, cellFirstStrategy,
> Permission.Action.READ)) {
> return ReturnCode.INCLUDE;
> }
> {code}
> So from 0.98.2 onwards IntegrationTestIngestWithACL was failing. Hence this
> JIRA is targeted to correct the behaviour and make the IT work again.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11251) LoadTestTool should grant READ permission for the users that are given READ access for specific cells
[
https://issues.apache.org/jira/browse/HBASE-11251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-11251:
---
Attachment: HBASE-11251_0.98.patch
> LoadTestTool should grant READ permission for the users that are given READ
> access for specific cells
> -
>
> Key: HBASE-11251
> URL: https://issues.apache.org/jira/browse/HBASE-11251
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.2
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0, 0.98.3
>
> Attachments: HBASE-11251_0.98.patch, HBASE-11251_trunk.patch
>
>
> In 0.98.2 onwards the AccessControlFilter
> {code}
> case CHECK_CELL_FIRST: {
> LOG.info("Am coming here for cell first strategy");
> if (authManager.authorize(user, table, cell, Permission.Action.READ)
> &&
> authManager.authorize(user, table, family, qualifier,
> Permission.Action.READ)) {
> LOG.info("Returning include");
> return ReturnCode.INCLUDE;
> }
> {code}
> expects a READ permission on the table for those Users that are granted READ
> permission on the cell level.
> In 0.98.1
> {code}
> if (authManager.authorize(user, table, cell, cellFirstStrategy,
> Permission.Action.READ)) {
> return ReturnCode.INCLUDE;
> }
> {code}
> So from 0.98.2 onwards IntegrationTestIngestWithACL was failing. Hence this
> JIRA is targeted to correct the behaviour and make the IT work again.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11251) LoadTestTool should grant READ permission for the users that are given READ access for specific cells
[
https://issues.apache.org/jira/browse/HBASE-11251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008574#comment-14008574
]
Andrew Purtell commented on HBASE-11251:
+1
This change was made so the cell first strategy (which is not the default) can
be useful for the use case it is intended for - denying/overriding access. The
cell ACL IT was not working at the time so it was missed.
> LoadTestTool should grant READ permission for the users that are given READ
> access for specific cells
> -
>
> Key: HBASE-11251
> URL: https://issues.apache.org/jira/browse/HBASE-11251
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.2
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0, 0.98.3
>
> Attachments: HBASE-11251_0.98.patch, HBASE-11251_trunk.patch
>
>
> In 0.98.2 onwards the AccessControlFilter
> {code}
> case CHECK_CELL_FIRST: {
> LOG.info("Am coming here for cell first strategy");
> if (authManager.authorize(user, table, cell, Permission.Action.READ)
> &&
> authManager.authorize(user, table, family, qualifier,
> Permission.Action.READ)) {
> LOG.info("Returning include");
> return ReturnCode.INCLUDE;
> }
> {code}
> expects a READ permission on the table for those Users that are granted READ
> permission on the cell level.
> In 0.98.1
> {code}
> if (authManager.authorize(user, table, cell, cellFirstStrategy,
> Permission.Action.READ)) {
> return ReturnCode.INCLUDE;
> }
> {code}
> So from 0.98.2 onwards IntegrationTestIngestWithACL was failing. Hence this
> JIRA is targeted to correct the behaviour and make the IT work again.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11234) FastDiffDeltaEncoder#getFirstKeyInBlock returns wrong result
[
https://issues.apache.org/jira/browse/HBASE-11234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-11234:
---
Fix Version/s: (was: 0.98.4)
0.98.3
> FastDiffDeltaEncoder#getFirstKeyInBlock returns wrong result
>
>
> Key: HBASE-11234
> URL: https://issues.apache.org/jira/browse/HBASE-11234
> Project: HBase
> Issue Type: Bug
>Reporter: chunhui shen
>Assignee: chunhui shen
>Priority: Critical
> Fix For: 0.99.0, 0.96.3, 0.98.3, 0.94.21
>
> Attachments: 11234-94.patch, 11234-96.patch,
> 11234-98-with-prefix-tree.txt, 11234-trunk.addendum, HBASE-11234.patch
>
>
> As Ted found,
> With this change:
> {noformat}
> Index:
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
> ===
> ---
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
>(revision 1596579)
> +++
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
>(working copy)
> @@ -51,6 +51,7 @@
> import org.apache.hadoop.hbase.filter.FilterList.Operator;
> import org.apache.hadoop.hbase.filter.PageFilter;
> import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
> +import org.apache.hadoop.hbase.io.encoding.DataBlockEncoding;
> import org.apache.hadoop.hbase.io.hfile.CacheConfig;
> import org.apache.hadoop.hbase.io.hfile.HFileContext;
> import org.apache.hadoop.hbase.io.hfile.HFileContextBuilder;
> @@ -90,6 +91,7 @@
> CacheConfig cacheConf = new CacheConfig(TEST_UTIL.getConfiguration());
> HFileContextBuilder hcBuilder = new HFileContextBuilder();
> hcBuilder.withBlockSize(2 * 1024);
> +hcBuilder.withDataBlockEncoding(DataBlockEncoding.FAST_DIFF);
> HFileContext hFileContext = hcBuilder.build();
> StoreFile.Writer writer = new StoreFile.WriterBuilder(
> TEST_UTIL.getConfiguration(), cacheConf, fs).withOutputDir(
> {noformat}
> I got:
> java.lang.AssertionError:
> expected:
> but was:
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hbase.regionserver.TestReversibleScanners.seekTestOfReversibleKeyValueScanner(TestReversibleScanners.java:533)
> at
> org.apache.hadoop.hbase.regionserver.TestReversibleScanners.testReversibleStoreFileScanner(TestReversibleScanners.java:108)
> After debugging, it seems the method of
> FastDiffDeltaEncoder#getFirstKeyInBlock become broken. And it will cause
> hfilescanner#seekBefore returns wrong result.
> The solution is simple, see the patch.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11251) LoadTestTool should grant READ permission for the users that are given READ access for specific cells
[
https://issues.apache.org/jira/browse/HBASE-11251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008594#comment-14008594
]
Anoop Sam John commented on HBASE-11251:
+ HTable table = new HTable(conf, tableName);
The table is just used to get the tabelName which we already have. We can
avoid this HTable creation and so the close in finally. This is there in 2
places in the patch.
{code}
-AccessControlClient.grant(conf, tableName, userOwner.getShortName(),
COLUMN_FAMILY,
+try {
+ AccessControlClient.grant(conf, table.getName(),
userOwner.getShortName(), null,
{code}
Any reason why the specific CF grant is changed in this patch?
> LoadTestTool should grant READ permission for the users that are given READ
> access for specific cells
> -
>
> Key: HBASE-11251
> URL: https://issues.apache.org/jira/browse/HBASE-11251
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.2
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0, 0.98.3
>
> Attachments: HBASE-11251_0.98.patch, HBASE-11251_trunk.patch
>
>
> In 0.98.2 onwards the AccessControlFilter
> {code}
> case CHECK_CELL_FIRST: {
> LOG.info("Am coming here for cell first strategy");
> if (authManager.authorize(user, table, cell, Permission.Action.READ)
> &&
> authManager.authorize(user, table, family, qualifier,
> Permission.Action.READ)) {
> LOG.info("Returning include");
> return ReturnCode.INCLUDE;
> }
> {code}
> expects a READ permission on the table for those Users that are granted READ
> permission on the cell level.
> In 0.98.1
> {code}
> if (authManager.authorize(user, table, cell, cellFirstStrategy,
> Permission.Action.READ)) {
> return ReturnCode.INCLUDE;
> }
> {code}
> So from 0.98.2 onwards IntegrationTestIngestWithACL was failing. Hence this
> JIRA is targeted to correct the behaviour and make the IT work again.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11251) LoadTestTool should grant READ permission for the users that are given READ access for specific cells
[
https://issues.apache.org/jira/browse/HBASE-11251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008596#comment-14008596
]
ramkrishna.s.vasudevan commented on HBASE-11251:
bq.Any reason why the specific CF grant is changed in this patch?
CF grant permission is not needed here. It was done previously. But while
testing I found specifically we need not grant permission on the CF level.
Also there is only one CF in the test.
bq.The table is just used to get the tabelName which we already have. We can
avoid this HTable creation and so the close in finally. This is there in 2
places in the patch.
Ok. I just ensured the close had to be done which was not done in the existing
code.
> LoadTestTool should grant READ permission for the users that are given READ
> access for specific cells
> -
>
> Key: HBASE-11251
> URL: https://issues.apache.org/jira/browse/HBASE-11251
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.2
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0, 0.98.3
>
> Attachments: HBASE-11251_0.98.patch, HBASE-11251_trunk.patch
>
>
> In 0.98.2 onwards the AccessControlFilter
> {code}
> case CHECK_CELL_FIRST: {
> LOG.info("Am coming here for cell first strategy");
> if (authManager.authorize(user, table, cell, Permission.Action.READ)
> &&
> authManager.authorize(user, table, family, qualifier,
> Permission.Action.READ)) {
> LOG.info("Returning include");
> return ReturnCode.INCLUDE;
> }
> {code}
> expects a READ permission on the table for those Users that are granted READ
> permission on the cell level.
> In 0.98.1
> {code}
> if (authManager.authorize(user, table, cell, cellFirstStrategy,
> Permission.Action.READ)) {
> return ReturnCode.INCLUDE;
> }
> {code}
> So from 0.98.2 onwards IntegrationTestIngestWithACL was failing. Hence this
> JIRA is targeted to correct the behaviour and make the IT work again.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11251) LoadTestTool should grant READ permission for the users that are given READ access for specific cells
[
https://issues.apache.org/jira/browse/HBASE-11251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008598#comment-14008598
]
Anoop Sam John commented on HBASE-11251:
bq. But while testing I found specifically we need not grant permission on the
CF level. Also there is only one CF in the test.
Fine. NP.
+1 for commit (you can remove the HTable create code on commit)
> LoadTestTool should grant READ permission for the users that are given READ
> access for specific cells
> -
>
> Key: HBASE-11251
> URL: https://issues.apache.org/jira/browse/HBASE-11251
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.2
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0, 0.98.3
>
> Attachments: HBASE-11251_0.98.patch, HBASE-11251_trunk.patch
>
>
> In 0.98.2 onwards the AccessControlFilter
> {code}
> case CHECK_CELL_FIRST: {
> LOG.info("Am coming here for cell first strategy");
> if (authManager.authorize(user, table, cell, Permission.Action.READ)
> &&
> authManager.authorize(user, table, family, qualifier,
> Permission.Action.READ)) {
> LOG.info("Returning include");
> return ReturnCode.INCLUDE;
> }
> {code}
> expects a READ permission on the table for those Users that are granted READ
> permission on the cell level.
> In 0.98.1
> {code}
> if (authManager.authorize(user, table, cell, cellFirstStrategy,
> Permission.Action.READ)) {
> return ReturnCode.INCLUDE;
> }
> {code}
> So from 0.98.2 onwards IntegrationTestIngestWithACL was failing. Hence this
> JIRA is targeted to correct the behaviour and make the IT work again.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10771) Primitive type put/get APIs in ByteRange
[
https://issues.apache.org/jira/browse/HBASE-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008599#comment-14008599
]
Hadoop QA commented on HBASE-10771:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12646734/HBASE-10771_V4.patch
against trunk revision .
ATTACHMENT ID: 12646734
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 8 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 3
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:red}-1 release audit{color}. The applied patch generated 31 release
audit warnings (more than the trunk's current 0 warnings).
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9583//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9583//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9583//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9583//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9583//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9583//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9583//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9583//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9583//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9583//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9583//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9583//console
This message is automatically generated.
> Primitive type put/get APIs in ByteRange
> -
>
> Key: HBASE-10771
> URL: https://issues.apache.org/jira/browse/HBASE-10771
> Project: HBase
> Issue Type: Improvement
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Fix For: 0.99.0
>
> Attachments: HBASE-10771.patch, HBASE-10771_V2.patch,
> HBASE-10771_V3.patch, HBASE-10771_V4.patch
>
>
> While doing HBASE-10713 I came across the need to write int/long (and read
> also) from a ByteRange. CellBlocks are backed by ByteRange. So we can add
> such APIs.
> Also as per HBASE-10750 we return a ByteRange from MSLAB and also discussion
> under HBASE-10191 suggest we can have BR backed HFileBlocks etc.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11251) LoadTestTool should grant READ permission for the users that are given READ access for specific cells
[
https://issues.apache.org/jira/browse/HBASE-11251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-11251:
---
Attachment: HBASE-11251_trunk_1.patch
HBASE-11251_0.98_1.patch
> LoadTestTool should grant READ permission for the users that are given READ
> access for specific cells
> -
>
> Key: HBASE-11251
> URL: https://issues.apache.org/jira/browse/HBASE-11251
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.2
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0, 0.98.3
>
> Attachments: HBASE-11251_0.98.patch, HBASE-11251_0.98_1.patch,
> HBASE-11251_trunk.patch, HBASE-11251_trunk_1.patch
>
>
> In 0.98.2 onwards the AccessControlFilter
> {code}
> case CHECK_CELL_FIRST: {
> LOG.info("Am coming here for cell first strategy");
> if (authManager.authorize(user, table, cell, Permission.Action.READ)
> &&
> authManager.authorize(user, table, family, qualifier,
> Permission.Action.READ)) {
> LOG.info("Returning include");
> return ReturnCode.INCLUDE;
> }
> {code}
> expects a READ permission on the table for those Users that are granted READ
> permission on the cell level.
> In 0.98.1
> {code}
> if (authManager.authorize(user, table, cell, cellFirstStrategy,
> Permission.Action.READ)) {
> return ReturnCode.INCLUDE;
> }
> {code}
> So from 0.98.2 onwards IntegrationTestIngestWithACL was failing. Hence this
> JIRA is targeted to correct the behaviour and make the IT work again.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11251) LoadTestTool should grant READ permission for the users that are given READ access for specific cells
[
https://issues.apache.org/jira/browse/HBASE-11251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008602#comment-14008602
]
ramkrishna.s.vasudevan commented on HBASE-11251:
Why the HTable got added and my compiler showed that the HTable had to closed
was in 0.98 the HTable creation was not removed. I first tested the patch in
0.98 and applied on trunk.
So that code was applied back to trunk and hence I thought the table had to be
closed.
> LoadTestTool should grant READ permission for the users that are given READ
> access for specific cells
> -
>
> Key: HBASE-11251
> URL: https://issues.apache.org/jira/browse/HBASE-11251
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.2
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0, 0.98.3
>
> Attachments: HBASE-11251_0.98.patch, HBASE-11251_0.98_1.patch,
> HBASE-11251_trunk.patch, HBASE-11251_trunk_1.patch
>
>
> In 0.98.2 onwards the AccessControlFilter
> {code}
> case CHECK_CELL_FIRST: {
> LOG.info("Am coming here for cell first strategy");
> if (authManager.authorize(user, table, cell, Permission.Action.READ)
> &&
> authManager.authorize(user, table, family, qualifier,
> Permission.Action.READ)) {
> LOG.info("Returning include");
> return ReturnCode.INCLUDE;
> }
> {code}
> expects a READ permission on the table for those Users that are granted READ
> permission on the cell level.
> In 0.98.1
> {code}
> if (authManager.authorize(user, table, cell, cellFirstStrategy,
> Permission.Action.READ)) {
> return ReturnCode.INCLUDE;
> }
> {code}
> So from 0.98.2 onwards IntegrationTestIngestWithACL was failing. Hence this
> JIRA is targeted to correct the behaviour and make the IT work again.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11251) LoadTestTool should grant READ permission for the users that are given READ access for specific cells
[
https://issues.apache.org/jira/browse/HBASE-11251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-11251:
---
Status: Open (was: Patch Available)
> LoadTestTool should grant READ permission for the users that are given READ
> access for specific cells
> -
>
> Key: HBASE-11251
> URL: https://issues.apache.org/jira/browse/HBASE-11251
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.2
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0, 0.98.3
>
> Attachments: HBASE-11251_0.98.patch, HBASE-11251_0.98_1.patch,
> HBASE-11251_trunk.patch, HBASE-11251_trunk_1.patch
>
>
> In 0.98.2 onwards the AccessControlFilter
> {code}
> case CHECK_CELL_FIRST: {
> LOG.info("Am coming here for cell first strategy");
> if (authManager.authorize(user, table, cell, Permission.Action.READ)
> &&
> authManager.authorize(user, table, family, qualifier,
> Permission.Action.READ)) {
> LOG.info("Returning include");
> return ReturnCode.INCLUDE;
> }
> {code}
> expects a READ permission on the table for those Users that are granted READ
> permission on the cell level.
> In 0.98.1
> {code}
> if (authManager.authorize(user, table, cell, cellFirstStrategy,
> Permission.Action.READ)) {
> return ReturnCode.INCLUDE;
> }
> {code}
> So from 0.98.2 onwards IntegrationTestIngestWithACL was failing. Hence this
> JIRA is targeted to correct the behaviour and make the IT work again.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Resolved] (HBASE-11251) LoadTestTool should grant READ permission for the users that are given READ access for specific cells
[
https://issues.apache.org/jira/browse/HBASE-11251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan resolved HBASE-11251.
Resolution: Fixed
Hadoop Flags: Reviewed
Committed to trunk and 0.98.
Thanks for the reviews.
> LoadTestTool should grant READ permission for the users that are given READ
> access for specific cells
> -
>
> Key: HBASE-11251
> URL: https://issues.apache.org/jira/browse/HBASE-11251
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.2
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0, 0.98.3
>
> Attachments: HBASE-11251_0.98.patch, HBASE-11251_0.98_1.patch,
> HBASE-11251_trunk.patch, HBASE-11251_trunk_1.patch
>
>
> In 0.98.2 onwards the AccessControlFilter
> {code}
> case CHECK_CELL_FIRST: {
> LOG.info("Am coming here for cell first strategy");
> if (authManager.authorize(user, table, cell, Permission.Action.READ)
> &&
> authManager.authorize(user, table, family, qualifier,
> Permission.Action.READ)) {
> LOG.info("Returning include");
> return ReturnCode.INCLUDE;
> }
> {code}
> expects a READ permission on the table for those Users that are granted READ
> permission on the cell level.
> In 0.98.1
> {code}
> if (authManager.authorize(user, table, cell, cellFirstStrategy,
> Permission.Action.READ)) {
> return ReturnCode.INCLUDE;
> }
> {code}
> So from 0.98.2 onwards IntegrationTestIngestWithACL was failing. Hence this
> JIRA is targeted to correct the behaviour and make the IT work again.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10771) Primitive type put/get APIs in ByteRange
[ https://issues.apache.org/jira/browse/HBASE-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008607#comment-14008607 ] Anoop Sam John commented on HBASE-10771: The javadoc warnings and release audit warnings are not related to this patch > Primitive type put/get APIs in ByteRange > - > > Key: HBASE-10771 > URL: https://issues.apache.org/jira/browse/HBASE-10771 > Project: HBase > Issue Type: Improvement >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 0.99.0 > > Attachments: HBASE-10771.patch, HBASE-10771_V2.patch, > HBASE-10771_V3.patch, HBASE-10771_V4.patch > > > While doing HBASE-10713 I came across the need to write int/long (and read > also) from a ByteRange. CellBlocks are backed by ByteRange. So we can add > such APIs. > Also as per HBASE-10750 we return a ByteRange from MSLAB and also discussion > under HBASE-10191 suggest we can have BR backed HFileBlocks etc. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11251) LoadTestTool should grant READ permission for the users that are given READ access for specific cells
[
https://issues.apache.org/jira/browse/HBASE-11251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008617#comment-14008617
]
Hadoop QA commented on HBASE-11251:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12646740/HBASE-11251_trunk.patch
against trunk revision .
ATTACHMENT ID: 12646740
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 3
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:red}-1 release audit{color}. The applied patch generated 32 release
audit warnings (more than the trunk's current 0 warnings).
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestTableLockManager
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9584//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9584//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9584//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9584//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9584//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9584//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9584//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9584//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9584//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9584//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9584//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9584//console
This message is automatically generated.
> LoadTestTool should grant READ permission for the users that are given READ
> access for specific cells
> -
>
> Key: HBASE-11251
> URL: https://issues.apache.org/jira/browse/HBASE-11251
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.2
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0, 0.98.3
>
> Attachments: HBASE-11251_0.98.patch, HBASE-11251_0.98_1.patch,
> HBASE-11251_trunk.patch, HBASE-11251_trunk_1.patch
>
>
> In 0.98.2 onwards the AccessControlFilter
> {code}
> case CHECK_CELL_FIRST: {
> LOG.info("Am coming here for cell first strategy");
> if (authManager.authorize(user, table, cell, Permission.Action.READ)
> &&
> authManager.authorize(user, table, family, qualifier,
> Permission.Action.READ)) {
> LOG.info("Returning include");
> return ReturnCode.INCLUDE;
> }
> {code}
> expects a READ permission on the table for those Users that are granted READ
> permission on the cell level.
> In 0.98.1
> {code}
> if (authManager.authorize(user, table, cell, cellFirstStrategy,
> Permission.Action.READ)) {
> return ReturnCode.INCLUDE;
> }
> {code}
> So from 0.98.2 onwards IntegrationTestIngestWithACL was failing. Hence this
> JIRA is targeted to correct the behaviour and make the IT work again.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
