date:20131201

[jira] [Commented] (HBASE-10060) Unsynchronized scanning

2013-12-01 Thread Hadoop QA (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-10060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835984#comment-13835984
]

Hadoop QA commented on HBASE-10060:
---

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12616464/10060-trunk-v2.txt
against trunk revision .

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.

{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.

{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.

{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.

{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.

{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.

{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100

{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.

{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestStoreScanner
org.apache.hadoop.hbase.regionserver.TestMemStore
org.apache.hadoop.hbase.regionserver.TestHRegionBusyWait
org.apache.hadoop.hbase.regionserver.TestStore

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8035//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8035//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8035//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8035//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8035//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8035//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8035//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8035//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8035//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8035//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8035//console

This message is automatically generated.

Unsynchronized scanning
---

Key: HBASE-10060
URL: https://issues.apache.org/jira/browse/HBASE-10060
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Priority: Minor
Attachments: 10060-trunk-v2.txt, 10060-trunk.txt

HBASE-10015 has some lengthy discussion. The solution there ended up
replacing synchronized with ReentrantLock, which - somewhat surprisingly -
yielded a non-trivial improvement for tall tables.
The goal should be to avoid locking in StoreScanner at all. StoreScanner is
only accessed by a single thread *except* when we have a concurrent flush or
a compaction, which is rare (we'd acquire and release the lock millions of
times per second, and compact/flush a few time an hour at the most).

--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Issue Comment Deleted] (HBASE-9489) Add cp hooks in online merge before and after setting PONR

2013-12-01 Thread Anoop Sam John (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-9489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anoop Sam John updated HBASE-9489:
--

Comment: was deleted

(was: +1
nit:  Few more occurrence of white spaces in RegionMergeTransaction. Pls remove 
them while committing.)

 Add cp hooks in online merge before and after setting PONR
 --

 Key: HBASE-9489
 URL: https://issues.apache.org/jira/browse/HBASE-9489
 Project: HBase
  Issue Type: Sub-task
Reporter: rajeshbabu
Assignee: rajeshbabu
 Attachments: HBASE-9489.patch, HBASE-9489_v2.patch, 
 HBASE-9489_v3.patch, HBASE-9489_v4.patch


 As we need to merge index region along with user region we need the hooks 
 before and after setting PONR in region merge transtion.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HBASE-9489) Add cp hooks in online merge before and after setting PONR

2013-12-01 Thread Anoop Sam John (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-9489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13836003#comment-13836003
 ] 

Anoop Sam John commented on HBASE-9489:
---

+1
nit:  Few more occurrence of white spaces in RegionMergeTransaction. Pls remove 
them while committing.

 Add cp hooks in online merge before and after setting PONR
 --

 Key: HBASE-9489
 URL: https://issues.apache.org/jira/browse/HBASE-9489
 Project: HBase
  Issue Type: Sub-task
Reporter: rajeshbabu
Assignee: rajeshbabu
 Attachments: HBASE-9489.patch, HBASE-9489_v2.patch, 
 HBASE-9489_v3.patch, HBASE-9489_v4.patch


 As we need to merge index region along with user region we need the hooks 
 before and after setting PONR in region merge transtion.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HBASE-9489) Add cp hooks in online merge before and after setting PONR

2013-12-01 Thread Anoop Sam John (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-9489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13836002#comment-13836002
 ] 

Anoop Sam John commented on HBASE-9489:
---

+1
nit:  Few more occurrence of white spaces in RegionMergeTransaction. Pls remove 
them while committing.

 Add cp hooks in online merge before and after setting PONR
 --

 Key: HBASE-9489
 URL: https://issues.apache.org/jira/browse/HBASE-9489
 Project: HBase
  Issue Type: Sub-task
Reporter: rajeshbabu
Assignee: rajeshbabu
 Attachments: HBASE-9489.patch, HBASE-9489_v2.patch, 
 HBASE-9489_v3.patch, HBASE-9489_v4.patch


 As we need to merge index region along with user region we need the hooks 
 before and after setting PONR in region merge transtion.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HBASE-9535) Try a pool of direct byte buffers handling incoming ipc requests

2013-12-01 Thread Nicolas Liochon (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13836022#comment-13836022
]

Nicolas Liochon commented on HBASE-9535:

bq. You have diff of GC use?
Not really. I've looked at the gc through the profile, it was around 4 minor
gc/s with CMS, 1/s with G1, w/ or w/o the patch. I don't think the patch makes
a difference in this scenario, as with YCSB puts we have a few large
allocations, instead of ismultiple tiny ones. So for this scenario I think the
benefit comes from the array copy suppressed.

Try a pool of direct byte buffers handling incoming ipc requests

Key: HBASE-9535
URL: https://issues.apache.org/jira/browse/HBASE-9535
Project: HBase
Issue Type: Brainstorming
Reporter: stack
Assignee: Nicolas Liochon
Attachments: 9535.v1.patch, 9535.v2-trunk.patch, 9535.v2.patch,
9535.v3.patch

ipc takes in a query by allocating a ByteBuffer of the size of the request
and then reading off the socket into this on-heap BB.
Experiment with keeping a pool of BBs so we have some buffer reuse to cut on
garbage generated. Could checkout from pool in RpcServer#Reader. Could
check back into the pool when Handler is done just before it queues the
response on the Responder's queue. We should be good since, at least for
now, kvs get copied up into MSLAB (not references) when data gets stuffed
into MemStore; this should make it so no references left over when we check
the BB back into the pool for use next time around.
If on-heap BBs work, we could then try direct BBs (Allocation of DBBs takes
time so if already allocated, should be good. GC of DBBs is a pain but if in
a pool, we shouldn't be wanting this to happen). The copy from socket to the
DBB will be off-heap (should be fast).
Could start w/ the HDFS DirectBufferPool. It is unbounded and keeps items by
size (we might want to bypass the pool if an object is size N).
DBBs for this task would contend w/ offheap BBs used in BlockReadLocal when
short-circuit reading. It'd be a bummer if we had to allocate big objects
on-heap. Would still be an improvement.

--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Created] (HBASE-10061) TableMapReduceUtil.findOrCreateJar calls updateMap(null, ) resulting in thrown NPE

2013-12-01 Thread Amit Sela (JIRA)

Amit Sela created HBASE-10061:
-

 Summary: TableMapReduceUtil.findOrCreateJar calls updateMap(null, 
) resulting in thrown NPE
 Key: HBASE-10061
 URL: https://issues.apache.org/jira/browse/HBASE-10061
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 0.94.12
Reporter: Amit Sela


TableMapReduceUtil.findOrCreateJar line 596:
jar = getJar(my_class);
updateMap(jar, packagedClasses);
In case getJar returns null, updateMap will throw NPE.
Should check null==jar before calling updateMap.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HBASE-10061) TableMapReduceUtil.findOrCreateJar calls updateMap(null, ) resulting in thrown NPE

2013-12-01 Thread Amit Sela (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-10061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13836055#comment-13836055
 ] 

Amit Sela commented on HBASE-10061:
---

Currently, findOrCreateJar throws IOException if jar path is null or empty.
Ted suggested to add a configuration parameter that will allow to choose 
between logging a missing jar (like it used to be in previous versions, i.e., 
0.94.2) or throwing Exception (as it is now).

 TableMapReduceUtil.findOrCreateJar calls updateMap(null, ) resulting in 
 thrown NPE
 --

 Key: HBASE-10061
 URL: https://issues.apache.org/jira/browse/HBASE-10061
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 0.94.12
Reporter: Amit Sela

 TableMapReduceUtil.findOrCreateJar line 596:
 jar = getJar(my_class);
 updateMap(jar, packagedClasses);
 In case getJar returns null, updateMap will throw NPE.
 Should check null==jar before calling updateMap.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HBASE-10048) Add hlog number metric in regionserver

2013-12-01 Thread Himanshu Vashishtha (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-10048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13836076#comment-13836076
 ] 

Himanshu Vashishtha commented on HBASE-10048:
-

+1 to the idea of exposing log numbers, and thanks for the patch.
(I'd avoid doing the numLogs +1 stuff, as it looks brittle). IMO, it is OKAY 
to show the rolled WALs and leave out the current one. And, name the metrics 
accordingly. What do you think [~lshmouse]?


[~zjushch]: Pardon for chiming in late; but couldn't just we make an 
intelligent guess about log size rather than creating a separate metrics for 
the same.
https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java#L360

 Add hlog number metric in regionserver
 --

 Key: HBASE-10048
 URL: https://issues.apache.org/jira/browse/HBASE-10048
 Project: HBase
  Issue Type: Improvement
Reporter: Liu Shaohui
Assignee: Liu Shaohui
Priority: Minor
 Attachments: HBASE-10048-0.94-v1.diff, HBASE-10048-0.94-v2.diff, 
 HBASE-10048-trunk-v1.diff, HBASE-10048-trunk-v2.diff, 
 HBASE-10048-trunk-v3.diff


 Add hlog number metric in regionserver. 
 We can use this metric to alert about memstore flush because of too many 
 hlogs.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (HBASE-9955) Make hadoop2 the default and deprecate hadoop1

2013-12-01 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-9955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-9955:
-

Attachment: 9955v2.txt

Patch had trash in it.

 Make hadoop2 the default and deprecate hadoop1
 --

 Key: HBASE-9955
 URL: https://issues.apache.org/jira/browse/HBASE-9955
 Project: HBase
  Issue Type: Task
Reporter: stack
Assignee: stack
 Fix For: 0.98.0

 Attachments: 9955.txt, 9955v2.txt


 See Hadoop version trunk dependency? on the dev mailing ilst.  Consensus 
 seems to be forming to do the subject line (Recheck the mail thread before 
 going ahead).



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HBASE-9955) Make hadoop2 the default and deprecate hadoop1

2013-12-01 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-9955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13836118#comment-13836118
 ] 

Hadoop QA commented on HBASE-9955:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12616486/9955v2.txt
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 hadoop1.0{color}.  The patch compiles against the hadoop 
1.0 profile.

{color:red}-1 hadoop2.0{color}.  The patch failed to compile against the 
hadoop 2.0 profile.

Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/8036//console

This message is automatically generated.

 Make hadoop2 the default and deprecate hadoop1
 --

 Key: HBASE-9955
 URL: https://issues.apache.org/jira/browse/HBASE-9955
 Project: HBase
  Issue Type: Task
Reporter: stack
Assignee: stack
 Fix For: 0.98.0

 Attachments: 9955.txt, 9955v2.txt


 See Hadoop version trunk dependency? on the dev mailing ilst.  Consensus 
 seems to be forming to do the subject line (Recheck the mail thread before 
 going ahead).



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HBASE-7091) support custom GC options in hbase-env.sh

2013-12-01 Thread Jesse Yates (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-7091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13836126#comment-13836126
]

Jesse Yates commented on HBASE-7091:

Is this in trunk or 0.94 that you have a concern Nicolas? I don't think it
erases it - it just doesn't apply it multiple times.

bin/hbase does this kind of thing:
{code}
HBASE_OPTS=$HBASE_OPTS $CLIENT_GC_OPTS
{code}

So it doesn't erase it - just copies it.

support custom GC options in hbase-env.sh
-

Key: HBASE-7091
URL: https://issues.apache.org/jira/browse/HBASE-7091
Project: HBase
Issue Type: Bug
Components: scripts
Affects Versions: 0.94.4
Reporter: Jesse Yates
Assignee: Jesse Yates
Labels: newbie
Fix For: 0.94.4, 0.95.0

Attachments: hbase-7091-v1.patch

When running things like bin/start-hbase and bin/hbase-daemon.sh start
[master|regionserver|etc] we end up setting HBASE_OPTS property a couple
times via calling hbase-env.sh. This is generally not a problem for most
cases, but when you want to set your own GC log properties, one would think
you should set HBASE_GC_OPTS, which get added to HBASE_OPTS.
NOPE! That would make too much sense.
Running bin/hbase-daemons.sh will run bin/hbase-daemon.sh with the daemons it
needs to start. Each time through hbase-daemon.sh we also call bin/hbase.
This isn't a big deal except for each call to hbase-daemon.sh, we also source
hbase-env.sh twice (once in the script and once in bin/hbase). This is
important for my next point.
Note that to turn on GC logging, you uncomment:
{code}
# export HBASE_OPTS=$HBASE_OPTS -verbose:gc -XX:+PrintGCDetails
-XX:+PrintGCDateStamps $HBASE_GC_OPTS
{code}
and then to log to a gc file for each server, you then uncomment:
{code}
# export HBASE_USE_GC_LOGFILE=true
{code}
in hbase-env.sh
On the first pass through hbase-daemon.sh, HBASE_GC_OPTS isn't set, so
HBASE_OPTS doesn't get anything funky, but we set HBASE_USE_GC_LOGFILE, which
then sets HBASE_GC_OPTS to the log file (-Xloggc:...). Then in bin/hbase we
again run hbase-env.sh, which now hs HBASE_GC_OPTS set, adding the GC file.
This isn't a general problem because HBASE_OPTS is set without prefixing the
existing HBASE_OPTS (eg. HBASE_OPTS=$HBASE_OPTS ...), allowing easy
updating. However, GC OPTS don't work the same and this is really odd
behavior when you want to set your own GC opts, which can include turning on
GC log rolling (yes, yes, they really are jvm opts, but they ought to support
their own param, to help minimize clutter).
The simple version of this patch will just add an idempotent GC option to
hbase-env.sh and some comments that uncommenting
{code}
# export HBASE_USE_GC_LOGFILE=true
{code}
will lead to a custom gc log file per server (along with an example name), so
you don't need to set -Xloggc.
The more complex solution does the above and also solves the multiple calls
to hbase-env.sh so we can be sane about how all this works. Note that to fix
this, hbase-daemon.sh just needs to read in HBASE_USE_GC_LOGFILE after
sourcing hbase-env.sh and then update HBASE_OPTS. Oh and also not source
hbase-env.sh in bin/hbase.
Even further, we might want to consider adding options just for cases where
we don't need gc logging - i.e. the shell, the config reading tool, hcbk,
etc. This is the hardest version to handle since the first couple will
willy-nilly apply the gc options.

--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HBASE-10048) Add hlog number metric in regionserver

2013-12-01 Thread chunhui shen (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-10048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13836177#comment-13836177
 ] 

chunhui shen commented on HBASE-10048:
--

bq,couldn't just we make an intelligent guess about log size rather than 
creating a separate metrics for the same.
I know the hlog size is about 95% of block size in normal case, 
but we have seen the hlog file size much bigger than block size in abnormal 
case.

I think such a metric could show the unhealthy scenario timely,  avoiding the 
hlog size become very huge.

Maybe we only need the metric for the max size of current hlog.
[~liushaohui] Any thought about the log size metric?

 Add hlog number metric in regionserver
 --

 Key: HBASE-10048
 URL: https://issues.apache.org/jira/browse/HBASE-10048
 Project: HBase
  Issue Type: Improvement
Reporter: Liu Shaohui
Assignee: Liu Shaohui
Priority: Minor
 Attachments: HBASE-10048-0.94-v1.diff, HBASE-10048-0.94-v2.diff, 
 HBASE-10048-trunk-v1.diff, HBASE-10048-trunk-v2.diff, 
 HBASE-10048-trunk-v3.diff


 Add hlog number metric in regionserver. 
 We can use this metric to alert about memstore flush because of too many 
 hlogs.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (HBASE-4811) Support reverse Scan

2013-12-01 Thread chunhui shen (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chunhui shen updated HBASE-4811:


Description: 
Reversed scan means scan the rows backward. 
And StartRow bigger than StopRow in a reversed scan.

For example, for the following rows:

aaa/c1:q1/value1
aaa/c1:q2/value2
bbb/c1:q1/value1
bbb/c1:q2/value2
ccc/c1:q1/value1
ccc/c1:q2/value2
ddd/c1:q1/value1
ddd/c1:q2/value2
eee/c1:q1/value1
eee/c1:q2/value2

you could do a reversed scan from 'ddd' to 'bbb'(exclude) like this:

Scan scan = new Scan();
scan.setStartRow('ddd');
scan.setStopRow('bbb');
scan.setReversed(true);
for(Result result:htable.getScanner(scan)){
 System.out.println(result);
}

Aslo you could do the reversed scan with shell like this:
hbase scan 'table',{REVERSED = true,STARTROW='ddd', STOPROW='bbb'}


And the output is:
ddd/c1:q1/value1
ddd/c1:q2/value2
ccc/c1:q1/value1
ccc/c1:q2/value2


All the documentation I find about HBase says that if you want forward and 
reverse scans you should just build 2 tables and one be ascending and one 
descending.  Is there a fundamental reason that HBase only supports forward 
Scan?  It seems like a lot of extra space overhead and coding overhead (to keep 
them in sync) to support 2 tables.  

I am assuming this has been discussed before, but I can't find the discussions 
anywhere about it or why it would be infeasible.


  was:
Reversed scan means scan the rows backward. 
And StartRow bigger than StopRow in a reversed scan.

For example, for the following rows:

aaa/c1:q1/value1
aaa/c1:q2/value2
bbb/c1:q1/value1
bbb/c1:q2/value2
ccc/c1:q1/value1
ccc/c1:q2/value2
ddd/c1:q1/value1
ddd/c1:q2/value2
eee/c1:q1/value1
eee/c1:q2/value2

you could do a reversed scan from 'ddd' to 'bbb'(exclude) like this:

Scan scan = new Scan();
scan.setStartRow('ddd');
scan.setStopRow('bbb');
scan.setReversed(true);
for(Result result:htable.getScanner(scan)){
 System.out.println(result);
}

Aslo you could do the reversed scan with shell like this:
hbase scan 'table',{REVERSED = true,STARTROW='ddd', STOPROW='bbb'}


And the output is:
ddd/c1:q1/value1
ddd/c1:q2/value2
ccc/c1:q1/value1
ccc/c1:q2/value2


NOTE: when setting reversed as true for a client scan, you must set the start 
row, else will throw exception. Through {@link 
Scan#createBiggestByteArray(int)},you could get a big enough byte array as the 
start row


All the documentation I find about HBase says that if you want forward and 
reverse scans you should just build 2 tables and one be ascending and one 
descending.  Is there a fundamental reason that HBase only supports forward 
Scan?  It seems like a lot of extra space overhead and coding overhead (to keep 
them in sync) to support 2 tables.  

I am assuming this has been discussed before, but I can't find the discussions 
anywhere about it or why it would be infeasible.



 Support reverse Scan
 

 Key: HBASE-4811
 URL: https://issues.apache.org/jira/browse/HBASE-4811
 Project: HBase
  Issue Type: New Feature
  Components: Client
Affects Versions: 0.20.6, 0.94.7
Reporter: John Carrino
Assignee: chunhui shen
 Fix For: 0.98.0, 0.96.1, 0.94.15

 Attachments: 4811-0.94-v22.txt, 4811-0.94-v23.txt, 4811-0.94-v3.txt, 
 4811-trunk-v10.txt, 4811-trunk-v29.patch, 4811-trunk-v5.patch, 
 HBase-4811-0.94-v2.txt, HBase-4811-0.94.3modified.txt, hbase-4811-0.94 
 v21.patch, hbase-4811-0.94-v24.patch, hbase-4811-trunkv1.patch, 
 hbase-4811-trunkv11.patch, hbase-4811-trunkv12.patch, 
 hbase-4811-trunkv13.patch, hbase-4811-trunkv14.patch, 
 hbase-4811-trunkv15.patch, hbase-4811-trunkv16.patch, 
 hbase-4811-trunkv17.patch, hbase-4811-trunkv18.patch, 
 hbase-4811-trunkv19.patch, hbase-4811-trunkv20.patch, 
 hbase-4811-trunkv21.patch, hbase-4811-trunkv24.patch, 
 hbase-4811-trunkv24.patch, hbase-4811-trunkv25.patch, 
 hbase-4811-trunkv26.patch, hbase-4811-trunkv27.patch, 
 hbase-4811-trunkv28.patch, hbase-4811-trunkv4.patch, 
 hbase-4811-trunkv6.patch, hbase-4811-trunkv7.patch, hbase-4811-trunkv8.patch, 
 hbase-4811-trunkv9.patch


 Reversed scan means scan the rows backward. 
 And StartRow bigger than StopRow in a reversed scan.
 For example, for the following rows:
 aaa/c1:q1/value1
 aaa/c1:q2/value2
 bbb/c1:q1/value1
 bbb/c1:q2/value2
 ccc/c1:q1/value1
 ccc/c1:q2/value2
 ddd/c1:q1/value1
 ddd/c1:q2/value2
 eee/c1:q1/value1
 eee/c1:q2/value2
 you could do a reversed scan from 'ddd' to 'bbb'(exclude) like this:
 Scan scan = new Scan();
 scan.setStartRow('ddd');
 scan.setStopRow('bbb');
 scan.setReversed(true);
 for(Result result:htable.getScanner(scan)){
  System.out.println(result);
 }
 Aslo you could do the reversed scan with shell like this:
 hbase scan 'table',{REVERSED = true,STARTROW='ddd', STOPROW='bbb'}
 And the output is:
 ddd/c1:q1/value1
 ddd/c1:q2/value2
 ccc/c1:q1/value1

[jira] [Commented] (HBASE-4811) Support reverse Scan

2013-12-01 Thread chunhui shen (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13836180#comment-13836180
 ] 

chunhui shen commented on HBASE-4811:
-

Commited to trunk,
Thanks very much for the review, Ted, Lars,Jean-Marc,Andrew and others

 Support reverse Scan
 

 Key: HBASE-4811
 URL: https://issues.apache.org/jira/browse/HBASE-4811
 Project: HBase
  Issue Type: New Feature
  Components: Client
Affects Versions: 0.20.6, 0.94.7
Reporter: John Carrino
Assignee: chunhui shen
 Fix For: 0.98.0, 0.96.1, 0.94.15

 Attachments: 4811-0.94-v22.txt, 4811-0.94-v23.txt, 4811-0.94-v3.txt, 
 4811-trunk-v10.txt, 4811-trunk-v29.patch, 4811-trunk-v5.patch, 
 HBase-4811-0.94-v2.txt, HBase-4811-0.94.3modified.txt, hbase-4811-0.94 
 v21.patch, hbase-4811-0.94-v24.patch, hbase-4811-trunkv1.patch, 
 hbase-4811-trunkv11.patch, hbase-4811-trunkv12.patch, 
 hbase-4811-trunkv13.patch, hbase-4811-trunkv14.patch, 
 hbase-4811-trunkv15.patch, hbase-4811-trunkv16.patch, 
 hbase-4811-trunkv17.patch, hbase-4811-trunkv18.patch, 
 hbase-4811-trunkv19.patch, hbase-4811-trunkv20.patch, 
 hbase-4811-trunkv21.patch, hbase-4811-trunkv24.patch, 
 hbase-4811-trunkv24.patch, hbase-4811-trunkv25.patch, 
 hbase-4811-trunkv26.patch, hbase-4811-trunkv27.patch, 
 hbase-4811-trunkv28.patch, hbase-4811-trunkv4.patch, 
 hbase-4811-trunkv6.patch, hbase-4811-trunkv7.patch, hbase-4811-trunkv8.patch, 
 hbase-4811-trunkv9.patch


 Reversed scan means scan the rows backward. 
 And StartRow bigger than StopRow in a reversed scan.
 For example, for the following rows:
 aaa/c1:q1/value1
 aaa/c1:q2/value2
 bbb/c1:q1/value1
 bbb/c1:q2/value2
 ccc/c1:q1/value1
 ccc/c1:q2/value2
 ddd/c1:q1/value1
 ddd/c1:q2/value2
 eee/c1:q1/value1
 eee/c1:q2/value2
 you could do a reversed scan from 'ddd' to 'bbb'(exclude) like this:
 Scan scan = new Scan();
 scan.setStartRow('ddd');
 scan.setStopRow('bbb');
 scan.setReversed(true);
 for(Result result:htable.getScanner(scan)){
  System.out.println(result);
 }
 Aslo you could do the reversed scan with shell like this:
 hbase scan 'table',{REVERSED = true,STARTROW='ddd', STOPROW='bbb'}
 And the output is:
 ddd/c1:q1/value1
 ddd/c1:q2/value2
 ccc/c1:q1/value1
 ccc/c1:q2/value2
 All the documentation I find about HBase says that if you want forward and 
 reverse scans you should just build 2 tables and one be ascending and one 
 descending.  Is there a fundamental reason that HBase only supports forward 
 Scan?  It seems like a lot of extra space overhead and coding overhead (to 
 keep them in sync) to support 2 tables.  
 I am assuming this has been discussed before, but I can't find the 
 discussions anywhere about it or why it would be infeasible.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (HBASE-4811) Support reverse Scan

2013-12-01 Thread chunhui shen (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chunhui shen updated HBASE-4811:


   Resolution: Fixed
Fix Version/s: (was: 0.94.15)
   (was: 0.96.1)
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Moving out of 0.94.15, wait lars to confirm when backport to 0.94

 Support reverse Scan
 

 Key: HBASE-4811
 URL: https://issues.apache.org/jira/browse/HBASE-4811
 Project: HBase
  Issue Type: New Feature
  Components: Client
Affects Versions: 0.20.6, 0.94.7
Reporter: John Carrino
Assignee: chunhui shen
 Fix For: 0.98.0

 Attachments: 4811-0.94-v22.txt, 4811-0.94-v23.txt, 4811-0.94-v3.txt, 
 4811-trunk-v10.txt, 4811-trunk-v29.patch, 4811-trunk-v5.patch, 
 HBase-4811-0.94-v2.txt, HBase-4811-0.94.3modified.txt, hbase-4811-0.94 
 v21.patch, hbase-4811-0.94-v24.patch, hbase-4811-trunkv1.patch, 
 hbase-4811-trunkv11.patch, hbase-4811-trunkv12.patch, 
 hbase-4811-trunkv13.patch, hbase-4811-trunkv14.patch, 
 hbase-4811-trunkv15.patch, hbase-4811-trunkv16.patch, 
 hbase-4811-trunkv17.patch, hbase-4811-trunkv18.patch, 
 hbase-4811-trunkv19.patch, hbase-4811-trunkv20.patch, 
 hbase-4811-trunkv21.patch, hbase-4811-trunkv24.patch, 
 hbase-4811-trunkv24.patch, hbase-4811-trunkv25.patch, 
 hbase-4811-trunkv26.patch, hbase-4811-trunkv27.patch, 
 hbase-4811-trunkv28.patch, hbase-4811-trunkv4.patch, 
 hbase-4811-trunkv6.patch, hbase-4811-trunkv7.patch, hbase-4811-trunkv8.patch, 
 hbase-4811-trunkv9.patch


 Reversed scan means scan the rows backward. 
 And StartRow bigger than StopRow in a reversed scan.
 For example, for the following rows:
 aaa/c1:q1/value1
 aaa/c1:q2/value2
 bbb/c1:q1/value1
 bbb/c1:q2/value2
 ccc/c1:q1/value1
 ccc/c1:q2/value2
 ddd/c1:q1/value1
 ddd/c1:q2/value2
 eee/c1:q1/value1
 eee/c1:q2/value2
 you could do a reversed scan from 'ddd' to 'bbb'(exclude) like this:
 Scan scan = new Scan();
 scan.setStartRow('ddd');
 scan.setStopRow('bbb');
 scan.setReversed(true);
 for(Result result:htable.getScanner(scan)){
  System.out.println(result);
 }
 Aslo you could do the reversed scan with shell like this:
 hbase scan 'table',{REVERSED = true,STARTROW='ddd', STOPROW='bbb'}
 And the output is:
 ddd/c1:q1/value1
 ddd/c1:q2/value2
 ccc/c1:q1/value1
 ccc/c1:q2/value2
 All the documentation I find about HBase says that if you want forward and 
 reverse scans you should just build 2 tables and one be ascending and one 
 descending.  Is there a fundamental reason that HBase only supports forward 
 Scan?  It seems like a lot of extra space overhead and coding overhead (to 
 keep them in sync) to support 2 tables.  
 I am assuming this has been discussed before, but I can't find the 
 discussions anywhere about it or why it would be infeasible.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

77 matches

Mail list logo