[jira] [Commented] (HBASE-4071) Data GC: Remove all versions TTL EXCEPT the last written version
[ https://issues.apache.org/jira/browse/HBASE-4071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13084792#comment-13084792 ] Andrew Purtell commented on HBASE-4071: --- TTLs were introduced to conveniently expire transient data and garbage collect it. Setting a TTL on a column family defines that column family as transient. Whatever changes are contemplated to be hard coded, this use case must be maintained. I want to be able to set a TTL and versions to 1 or 3 or 10 or whatever and have ALL of the versions discarded when their timestamps indicate expiry. Data GC: Remove all versions TTL EXCEPT the last written version -- Key: HBASE-4071 URL: https://issues.apache.org/jira/browse/HBASE-4071 Project: HBase Issue Type: New Feature Reporter: stack We were chatting today about our backup cluster. What we want is to be able to restore the dataset from any point of time but only within a limited timeframe -- say one week. Thereafter, if the versions are older than one week, rather than as we do with TTL where we let go of all versions older than TTL, instead, let go of all versions EXCEPT the last one written. So, its like versions==1 when TTL one week. We want to allow that if an error is caught within a week of its happening -- user mistakenly removes a critical table -- then we'll be able to restore up the the moment just before catastrophe hit otherwise, we keep one version only. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4071) Data GC: Remove all versions TTL EXCEPT the last written version
[ https://issues.apache.org/jira/browse/HBASE-4071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13084800#comment-13084800 ] Lars Hofhansl commented on HBASE-4071: -- Of course you're right Andrew. We shouldn't mix usecases together. What do you think of the minversions idea? As long as minversions for a CF is 0 the behavior would be identical to the current behavior. Also maybe what we have in mind is not a common scenario after all, and we shouldn't complicate the core codebase with that. Data GC: Remove all versions TTL EXCEPT the last written version -- Key: HBASE-4071 URL: https://issues.apache.org/jira/browse/HBASE-4071 Project: HBase Issue Type: New Feature Reporter: stack We were chatting today about our backup cluster. What we want is to be able to restore the dataset from any point of time but only within a limited timeframe -- say one week. Thereafter, if the versions are older than one week, rather than as we do with TTL where we let go of all versions older than TTL, instead, let go of all versions EXCEPT the last one written. So, its like versions==1 when TTL one week. We want to allow that if an error is caught within a week of its happening -- user mistakenly removes a critical table -- then we'll be able to restore up the the moment just before catastrophe hit otherwise, we keep one version only. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4200) blockCache summary - web UI
blockCache summary - web UI --- Key: HBASE-4200 URL: https://issues.apache.org/jira/browse/HBASE-4200 Project: HBase Issue Type: Sub-task Reporter: Doug Meil Priority: Minor Web UI for block cache summary report. This will most likely be a new web-page linked from the RegionServer detail page. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
[
https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13084856#comment-13084856
]
Ted Yu commented on HBASE-4199:
---
It would be easier for other developers to make review comments if review board
was used.
{code}
+/**
+ * This class provides tests for the {@link HTableUtil} class
+ *
+ */
+public class TestBlockCacheSummary {
{code}
Please update javadoc with brief description of what is tested in the above
class.
{code}
+ /**
+ *
+ */
+ public BlockCacheSummaryEntry() {
+
+ }
{code}
The above can be made package private. Please also fill in javadoc.
Earlier in the same class:
{code}
+ private String table;
+ private String columnFamily;
{code}
Would it be more general if the above two fields were declared as byte[] ?
{code}
+ * @return heapSize
+ */
+ public int getBlocks() {
+return blocks;
{code}
Please correct javadoc.
blockCache summary - backend
Key: HBASE-4199
URL: https://issues.apache.org/jira/browse/HBASE-4199
Project: HBase
Issue Type: Sub-task
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
Attachments: java_HBASE_4199.patch
This is the backend work for the blockCache summary. Change to BlockCache
interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition
to HRegionInterface, and HRegionServer.
This will NOT include any of the web UI or anything else like that. That is
for another sub-task.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13084864#comment-13084864 ] Doug Meil commented on HBASE-4199: -- BlockCacheSummaryEntry can't be made package private because it's also the class that is eventually returned by the RegionServer API. I'll fix the javadoc. How do you use the review board? I don't see anything on More Actions or Workflow in Jira. blockCache summary - backend Key: HBASE-4199 URL: https://issues.apache.org/jira/browse/HBASE-4199 Project: HBase Issue Type: Sub-task Reporter: Doug Meil Assignee: Doug Meil Priority: Minor Attachments: java_HBASE_4199.patch This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer. This will NOT include any of the web UI or anything else like that. That is for another sub-task. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4071) Data GC: Remove all versions TTL EXCEPT the last written version
[ https://issues.apache.org/jira/browse/HBASE-4071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13084872#comment-13084872 ] Andrew Purtell commented on HBASE-4071: --- bq. What do you think of the minversions idea? I think a new minversions in core is a fine idea. We can document how minversions, maxversions, and TTL settings interact with each other simply enough. Data GC: Remove all versions TTL EXCEPT the last written version -- Key: HBASE-4071 URL: https://issues.apache.org/jira/browse/HBASE-4071 Project: HBase Issue Type: New Feature Reporter: stack We were chatting today about our backup cluster. What we want is to be able to restore the dataset from any point of time but only within a limited timeframe -- say one week. Thereafter, if the versions are older than one week, rather than as we do with TTL where we let go of all versions older than TTL, instead, let go of all versions EXCEPT the last one written. So, its like versions==1 when TTL one week. We want to allow that if an error is caught within a week of its happening -- user mistakenly removes a critical table -- then we'll be able to restore up the the moment just before catastrophe hit otherwise, we keep one version only. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4197) RegionServer expects all scanner to be subclasses of HRegion.RegionScanner
[
https://issues.apache.org/jira/browse/HBASE-4197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13084880#comment-13084880
]
jirapos...@reviews.apache.org commented on HBASE-4197:
--
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1496/
---
(Updated 2011-08-14 18:39:53.875778)
Review request for hbase, Ted Yu and Mingjie Lai.
Summary
---
1. Don't require custom scanners created by conprocessors to be subclasses of
HRegion.RegionScanner (see HBASE-4197).
2. Simplify the interfaces for Scanners in HRegion, HRegionServer, and
RegionObserver. This avoids a bunch instanceof checks and casts to
HRegion.RegionScanner.
(Sorry HBase-git would not accept my patch)
This addresses bug HBASE-4197.
https://issues.apache.org/jira/browse/HBASE-4197
Diffs (updated)
-
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java
1157388
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java
1157388
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
1157388
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
1157388
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java
1157388
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java
PRE-CREATION
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java
1157388
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
1157388
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestWideScanner.java
1157388
Diff: https://reviews.apache.org/r/1496/diff
Testing
---
Manual test attached to the bug.
Thanks,
Lars
RegionServer expects all scanner to be subclasses of HRegion.RegionScanner
--
Key: HBASE-4197
URL: https://issues.apache.org/jira/browse/HBASE-4197
Project: HBase
Issue Type: Bug
Components: coprocessors
Affects Versions: 0.92.0
Reporter: Lars Hofhansl
Attachments: 4197-bigger.txt, 4197-v2.txt, 4197.txt, ScannerTest.java
Returning just an InternalScanner from RegionObsever.{pre|post}OpenScanner
leads to the following exception when using the scanner.
java.io.IOException: InternalScanner implementation is expected to be
HRegion.RegionScanner.
at
org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2023)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:314)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1225)
The problem is in HRegionServer.next(...):
{code}
InternalScanner s = this.scanners.get(scannerName);
...
// Call coprocessor. Get region info from scanner.
HRegion region = null;
if (s instanceof HRegion.RegionScanner) {
HRegion.RegionScanner rs = (HRegion.RegionScanner) s;
region = getRegion(rs.getRegionName().getRegionName());
} else {
throw new IOException(InternalScanner implementation is expected +
to be HRegion.RegionScanner.);
}
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4199) blockCache summary - backend
[ https://issues.apache.org/jira/browse/HBASE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13084881#comment-13084881 ] Ted Yu commented on HBASE-4199: --- I was talking about the default ctor for BlockCacheSummaryEntry. Throughout the patch, after BlockCacheSummaryEntry() is called, table and columnFamily are filled in. So I think BlockCacheSummaryEntry(String table, String columnFamily) should be used. Please see Mingjie's comment in HBASE-4197 about using review board. Thanks blockCache summary - backend Key: HBASE-4199 URL: https://issues.apache.org/jira/browse/HBASE-4199 Project: HBase Issue Type: Sub-task Reporter: Doug Meil Assignee: Doug Meil Priority: Minor Attachments: java_HBASE_4199.patch This is the backend work for the blockCache summary. Change to BlockCache interface, Summarization in LruBlockCache, BlockCacheSummaryEntry, addition to HRegionInterface, and HRegionServer. This will NOT include any of the web UI or anything else like that. That is for another sub-task. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4197) RegionServer expects all scanner to be subclasses of HRegion.RegionScanner
[
https://issues.apache.org/jira/browse/HBASE-4197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13084891#comment-13084891
]
jirapos...@reviews.apache.org commented on HBASE-4197:
--
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1496/#review1449
---
Ship it!
- Ted
On 2011-08-14 18:39:53, Lars Hofhansl wrote:
bq.
bq. ---
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1496/
bq. ---
bq.
bq. (Updated 2011-08-14 18:39:53)
bq.
bq.
bq. Review request for hbase, Ted Yu and Mingjie Lai.
bq.
bq.
bq. Summary
bq. ---
bq.
bq. 1. Don't require custom scanners created by conprocessors to be subclasses
of HRegion.RegionScanner (see HBASE-4197).
bq. 2. Simplify the interfaces for Scanners in HRegion, HRegionServer, and
RegionObserver. This avoids a bunch instanceof checks and casts to
HRegion.RegionScanner.
bq.
bq. (Sorry HBase-git would not accept my patch)
bq.
bq.
bq. This addresses bug HBASE-4197.
bq. https://issues.apache.org/jira/browse/HBASE-4197
bq.
bq.
bq. Diffs
bq. -
bq.
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java
1157388
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java
1157388
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
1157388
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
1157388
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java
1157388
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java
PRE-CREATION
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java
1157388
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
1157388
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestWideScanner.java
1157388
bq.
bq. Diff: https://reviews.apache.org/r/1496/diff
bq.
bq.
bq. Testing
bq. ---
bq.
bq. Manual test attached to the bug.
bq.
bq.
bq. Thanks,
bq.
bq. Lars
bq.
bq.
RegionServer expects all scanner to be subclasses of HRegion.RegionScanner
--
Key: HBASE-4197
URL: https://issues.apache.org/jira/browse/HBASE-4197
Project: HBase
Issue Type: Bug
Components: coprocessors
Affects Versions: 0.92.0
Reporter: Lars Hofhansl
Attachments: 4197-bigger.txt, 4197-v2.txt, 4197.txt, ScannerTest.java
Returning just an InternalScanner from RegionObsever.{pre|post}OpenScanner
leads to the following exception when using the scanner.
java.io.IOException: InternalScanner implementation is expected to be
HRegion.RegionScanner.
at
org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2023)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:314)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1225)
The problem is in HRegionServer.next(...):
{code}
InternalScanner s = this.scanners.get(scannerName);
...
// Call coprocessor. Get region info from scanner.
HRegion region = null;
if (s instanceof HRegion.RegionScanner) {
HRegion.RegionScanner rs = (HRegion.RegionScanner) s;
region = getRegion(rs.getRegionName().getRegionName());
} else {
throw new IOException(InternalScanner implementation is expected +
to be HRegion.RegionScanner.);
}
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3933) Hmaster throw NullPointerException
[
https://issues.apache.org/jira/browse/HBASE-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13084957#comment-13084957
]
gaojinchao commented on HBASE-3933:
---
Hi all. I have a new idea for this issue. why don't we get the regionserver
list from zk when it is failover?
we can avoid this case that the hlog is splited but region server is servering.
Hmaster throw NullPointerException
--
Key: HBASE-3933
URL: https://issues.apache.org/jira/browse/HBASE-3933
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.90.3
Reporter: gaojinchao
Assignee: gaojinchao
Attachments: Hmastersetup0.90
NullPointerException while hmaster starting.
{code}
java.lang.NullPointerException
at java.util.TreeMap.getEntry(TreeMap.java:324)
at java.util.TreeMap.get(TreeMap.java:255)
at
org.apache.hadoop.hbase.master.AssignmentManager.addToServers(AssignmentManager.java:1512)
at
org.apache.hadoop.hbase.master.AssignmentManager.regionOnline(AssignmentManager.java:606)
at
org.apache.hadoop.hbase.master.AssignmentManager.processFailover(AssignmentManager.java:214)
at
org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:402)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:283)
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4197) RegionServer expects all scanner to be subclasses of HRegion.RegionScanner
[
https://issues.apache.org/jira/browse/HBASE-4197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13084964#comment-13084964
]
Lars Hofhansl commented on HBASE-4197:
--
Yeah! :)
@Ted: Will you commit the patch attached to the review?
Note that this changes the externally visible interface of Coprocessors
(RegionScanner instead of InternalScanner)... Just calling this out, so that
there are no surprises later.
RegionServer expects all scanner to be subclasses of HRegion.RegionScanner
--
Key: HBASE-4197
URL: https://issues.apache.org/jira/browse/HBASE-4197
Project: HBase
Issue Type: Bug
Components: coprocessors
Affects Versions: 0.92.0
Reporter: Lars Hofhansl
Attachments: 4197-bigger.txt, 4197-v2.txt, 4197.txt, ScannerTest.java
Returning just an InternalScanner from RegionObsever.{pre|post}OpenScanner
leads to the following exception when using the scanner.
java.io.IOException: InternalScanner implementation is expected to be
HRegion.RegionScanner.
at
org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2023)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:314)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1225)
The problem is in HRegionServer.next(...):
{code}
InternalScanner s = this.scanners.get(scannerName);
...
// Call coprocessor. Get region info from scanner.
HRegion region = null;
if (s instanceof HRegion.RegionScanner) {
HRegion.RegionScanner rs = (HRegion.RegionScanner) s;
region = getRegion(rs.getRegionName().getRegionName());
} else {
throw new IOException(InternalScanner implementation is expected +
to be HRegion.RegionScanner.);
}
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
