[jira] [Updated] (HBASE-15302) Reenable the other tests disabled by HBASE-14678
[ https://issues.apache.org/jira/browse/HBASE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-15302: -- Attachment: HBASE-15302-branch-1-v1.patch rebuild on QA since branch-1 changed these days. > Reenable the other tests disabled by HBASE-14678 > > > Key: HBASE-15302 > URL: https://issues.apache.org/jira/browse/HBASE-15302 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0, 1.3.0, 1.2.1 > > Attachments: HBASE-15302-branch-1-v1.patch, > HBASE-15302-branch-1-v1.txt, HBASE-15302-v1.txt, HBASE-15302-v1.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15325) ResultScanner allowing partial result will miss the rest of the row if the region is moved between two rpc requests
[ https://issues.apache.org/jira/browse/HBASE-15325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-15325: -- Attachment: (was: HBASE-15325-v11.patch) > ResultScanner allowing partial result will miss the rest of the row if the > region is moved between two rpc requests > --- > > Key: HBASE-15325 > URL: https://issues.apache.org/jira/browse/HBASE-15325 > Project: HBase > Issue Type: Bug > Components: dataloss, Scanners >Affects Versions: 1.2.0, 1.1.3 >Reporter: Phil Yang >Assignee: Phil Yang >Priority: Critical > Fix For: 2.0.0, 1.3.0, 1.2.1, 1.1.4, 1.4.0 > > Attachments: 15325-test.txt, HBASE-15325-v1.txt, > HBASE-15325-v10.patch, HBASE-15325-v11.patch, HBASE-15325-v2.txt, > HBASE-15325-v3.txt, HBASE-15325-v5.txt, HBASE-15325-v6.1.txt, > HBASE-15325-v6.2.txt, HBASE-15325-v6.3.txt, HBASE-15325-v6.4.txt, > HBASE-15325-v6.5.txt, HBASE-15325-v6.txt, HBASE-15325-v7.patch, > HBASE-15325-v8.patch, HBASE-15325-v9.patch > > > HBASE-11544 allow scan rpc return partial of a row to reduce memory usage for > one rpc request. And client can setAllowPartial or setBatch to get several > cells in a row instead of the whole row. > However, the status of the scanner is saved on server and we need this to get > the next part if there is a partial result before. If we move the region to > another RS, client will get a NotServingRegionException and open a new > scanner to the new RS which will be regarded as a new scan from the end of > this row. So the rest cells of the row of last result will be missing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15481) Add pre/post roll to WALObserver
[ https://issues.apache.org/jira/browse/HBASE-15481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202528#comment-15202528 ] Sean Busbey commented on HBASE-15481: - backup and replication can get that via the WALActionListener, since they're internal. I'm -0 on exposing the specific wal files to external monitoring in a way that ties us to compatibility promises (even LimitedPrivate ones). > Add pre/post roll to WALObserver > > > Key: HBASE-15481 > URL: https://issues.apache.org/jira/browse/HBASE-15481 > Project: HBase > Issue Type: New Feature >Affects Versions: 1.3.0 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15481-v0.patch, HBASE-15481-v1.patch, > HBASE-15481-v1.patch > > > currently the WALObserver has only a pre/post Write. It will be useful to > have a pre/post Roll too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15464) Flush / Compaction metrics revisited
[ https://issues.apache.org/jira/browse/HBASE-15464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-15464: -- Attachment: hbase-15464_v3.patch > Flush / Compaction metrics revisited > > > Key: HBASE-15464 > URL: https://issues.apache.org/jira/browse/HBASE-15464 > Project: HBase > Issue Type: Sub-task > Components: metrics >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: hbase-15464_v1.patch, hbase-15464_v2.patch, > hbase-15464_v3.patch, hbase-15464_v3.patch > > > We can add a couple of metrics related to flushes and compactions: > - flush memstore and output file size histogram: This will allow seeing > whether we are flushing too early due to memory pressure, too many regions, > etc. Tracking flush memstore size vs output file size is useful in > understanding the block encoding compression benefits. > - total flushed output bytes: This will allow to monitor the IO / throughput > from flushers. You can use this to set num flushers, flush throttle, etc. > - smallCompactionQueueLength / large...: This is tracked, but not emitted > anymore due to a bug. > - compaction time histogram: similar to flush time histogram, how long > compactions are taking. > - compaction input num files / output num files histogram: How many files on > average we are compacting. Stripe compaction / date tiered compaction can use > the num output files metric. > - compaction input / output data sizes histogram: How much data on average > we are compacting. > - compaction input / output total bytes: Measure compaction IO / throughput. > measure write amplification, enables to set compaction throttle. > - Breakdown for above for major compactions -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15487) Deletions done via BulkDeleteEndpoint make past data re-appear
Mathias Herberts created HBASE-15487:
Summary: Deletions done via BulkDeleteEndpoint make past data
re-appear
Key: HBASE-15487
URL: https://issues.apache.org/jira/browse/HBASE-15487
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.3
Reporter: Mathias Herberts
The Warp10 (www.warp10.io) time series database uses HBase as its underlying
data store. The deletion of ranges of cells is performed using the
BulkDeleteEndpoint.
In the following scenario the deletion does not appear to be working properly:
The table 't' is created with a single version using:
create 't', {NAME => 'v', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER =>
'NONE', REPLICATION_SCOPE => '0', VERSIONS=> '1', MIN_VERSIONS => '0', TTL =>
'2147483647', KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '65536', IN_MEMORY
=>'false', BLOCKCACHE => 'true'}
We write a cell at row '0x00', colfam 'v', colq '', value 0x0
We write the same cell again with value 0x1
A scan will return a single value 0x1
We then perform a delete using the BulkDeleteEndpoint and a Scan with a
DeleteType of 'VERSION'
The reported number of deleted versions is 1 (which is coherent given the table
was created with MAX_VERSIONS=1)
The same scan as the one performed before the delete returns a single value 0x0.
This seems to happen when all operations are performed against the memstore.
A regular delete will remove the cell and a later scan won't show it.
I'll attach a test which demonstrates the problem.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15487) Deletions done via BulkDeleteEndpoint make past data re-appear
[
https://issues.apache.org/jira/browse/HBASE-15487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202254#comment-15202254
]
Mathias Herberts commented on HBASE-15487:
--
Forgot to mention that HBase has to be configured with the BulkDeleteEndpoint
enabled via
hbase.coprocessor.region.classes
org.apache.hadoop.hbase.coprocessor.example.BulkDeleteEndpoint
> Deletions done via BulkDeleteEndpoint make past data re-appear
> --
>
> Key: HBASE-15487
> URL: https://issues.apache.org/jira/browse/HBASE-15487
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.0.3
>Reporter: Mathias Herberts
> Attachments: HBaseTest.java
>
>
> The Warp10 (www.warp10.io) time series database uses HBase as its underlying
> data store. The deletion of ranges of cells is performed using the
> BulkDeleteEndpoint.
> In the following scenario the deletion does not appear to be working properly:
> The table 't' is created with a single version using:
> create 't', {NAME => 'v', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER =>
> 'NONE', REPLICATION_SCOPE => '0', VERSIONS=> '1', MIN_VERSIONS => '0', TTL =>
> '2147483647', KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '65536', IN_MEMORY
> =>'false', BLOCKCACHE => 'true'}
> We write a cell at row '0x00', colfam 'v', colq '', value 0x0
> We write the same cell again with value 0x1
> A scan will return a single value 0x1
> We then perform a delete using the BulkDeleteEndpoint and a Scan with a
> DeleteType of 'VERSION'
> The reported number of deleted versions is 1 (which is coherent given the
> table was created with MAX_VERSIONS=1)
> The same scan as the one performed before the delete returns a single value
> 0x0.
> This seems to happen when all operations are performed against the memstore.
> A regular delete will remove the cell and a later scan won't show it.
> I'll attach a test which demonstrates the problem.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15302) Reenable the other tests disabled by HBASE-14678
[ https://issues.apache.org/jira/browse/HBASE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202507#comment-15202507 ] Hudson commented on HBASE-15302: FAILURE: Integrated in HBase-1.4 #31 (See [https://builds.apache.org/job/HBase-1.4/31/]) HBASE-15302 Reenable the other tests disabled by HBASE-14678 (Phil Yang) (tedyu: rev d942d4e5a1fe5644a51b5775f3a3e27dd219ad1d) * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestMasterFailoverWithProcedures.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotCloneIndependence.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer2.java * hbase-shell/src/test/java/org/apache/hadoop/hbase/client/TestReplicationShell.java * hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestWALSplitCompressed.java * hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestWALSplit.java * hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/TestPartialResultsFromClientSide.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java > Reenable the other tests disabled by HBASE-14678 > > > Key: HBASE-15302 > URL: https://issues.apache.org/jira/browse/HBASE-15302 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0, 1.3.0, 1.2.1 > > Attachments: HBASE-15302-branch-1-v1.patch, HBASE-15302-v1.txt, > HBASE-15302-v1.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15488) Add ACL for setting split merge switch
[ https://issues.apache.org/jira/browse/HBASE-15488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15488: --- Attachment: HBASE-15488.v1.patch > Add ACL for setting split merge switch > -- > > Key: HBASE-15488 > URL: https://issues.apache.org/jira/browse/HBASE-15488 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-15488.v1.patch > > > Currently there is no access control for the split merge switch setter in > MasterRpcServices. > This JIRA adds necessary coprocessor hook along with enforcing permission > check in AccessController through the new hook. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15481) Add pre/post roll to WALObserver
[
https://issues.apache.org/jira/browse/HBASE-15481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202434#comment-15202434
]
Hadoop QA commented on HBASE-15481:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
21s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 26s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 56s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 6m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
25s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 5s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 35s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 4s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 4s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 6m 47s
{color} | {color:red} hbase-server: patch generated 2 new + 78 unchanged - 0
fixed = 80 total (was 78) {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
50m 14s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 9s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 13s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 97m 34s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 187m 14s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.balancer.TestStochasticLoadBalancer2
|
| Timed out junit tests |
org.apache.hadoop.hbase.regionserver.TestCorruptedRegionStoreFile |
| | org.apache.hadoop.hbase.regionserver.TestSplitLogWorker |
| | org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy
|
| | org.apache.hadoop.hbase.regionserver.TestCompaction |
| | org.apache.hadoop.hbase.snapshot.TestSnapshotClientRetries |
| | org.apache.hadoop.hbase.coprocessor.TestRegionObserverScannerOpenHook |
| | org.apache.hadoop.hbase.wal.TestWALFiltering |
| | org.apache.hadoop.hbase.coprocessor.TestRegionServerObserver |
| | org.apache.hadoop.hbase.wal.TestDefaultWALProviderWithHLogKey |
| | org.apache.hadoop.hbase.TestZooKeeper |
| |
[jira] [Updated] (HBASE-15477) Do not save 'next block header' when we cache hfileblocks
[ https://issues.apache.org/jira/browse/HBASE-15477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15477: -- Attachment: 15477v3.patch Fix compile error > Do not save 'next block header' when we cache hfileblocks > - > > Key: HBASE-15477 > URL: https://issues.apache.org/jira/browse/HBASE-15477 > Project: HBase > Issue Type: Sub-task > Components: BlockCache, Performance >Reporter: stack >Assignee: stack > Attachments: 15477.patch, 15477v2.patch, 15477v3.patch > > > When we read from HDFS, we overread to pick up the next blocks header. > Doing this saves a seek as we move through the hfile; we save having to > do an explicit seek just to read the block header every time we need to > read the body. We used to read in the next header as part of the > current blocks buffer. This buffer was then what got persisted to > blockcache; so we were over-persisting wrtiting out our block plus the > next blocks' header (overpersisting 33 bytes). Parse of HFileBlock > complicated by this extra tail. Fix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15406) Split / merge switch left disabled after early termination of hbck
[
https://issues.apache.org/jira/browse/HBASE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201469#comment-15201469
]
Ted Yu commented on HBASE-15406:
{code}
4283 ZKUtil.createSetData(zkw, splitOrMergeLock,
4284ProtobufUtil.prependPBMagic(builder.build().toByteArray()));
{code}
How about calling createEphemeralNodeAndWatch() in the above ?
Protection against changing the switch when hbck is running can be achieved by
checking presence of the lock znode (not its contents).
If hbck dies unexpectedly, the lock znode would disappear.
> Split / merge switch left disabled after early termination of hbck
> --
>
> Key: HBASE-15406
> URL: https://issues.apache.org/jira/browse/HBASE-15406
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15406.patch, HBASE-15406.v1.patch,
> HBASE-15406_v1.patch, test.patch, wip.patch
>
>
> This was what I did on cluster with 1.4.0-SNAPSHOT built Thursday:
> Run 'hbase hbck -disableSplitAndMerge' on gateway node of the cluster
> Terminate hbck early
> Enter hbase shell where I observed:
> {code}
> hbase(main):001:0> splitormerge_enabled 'SPLIT'
> false
> 0 row(s) in 0.3280 seconds
> hbase(main):002:0> splitormerge_enabled 'MERGE'
> false
> 0 row(s) in 0.0070 seconds
> {code}
> Expectation is that the split / merge switches should be restored to default
> value after hbck exits.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14845) hbase-server leaks jdk.tools dependency to mapreduce consumers
[ https://issues.apache.org/jira/browse/HBASE-14845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-14845: --- Fix Version/s: (was: 0.98.18) 0.98.19 > hbase-server leaks jdk.tools dependency to mapreduce consumers > -- > > Key: HBASE-14845 > URL: https://issues.apache.org/jira/browse/HBASE-14845 > Project: HBase > Issue Type: Bug > Components: build, dependencies >Affects Versions: 2.0.0, 0.98.14, 1.2.0, 1.1.2, 1.3.0, 1.0.3 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 2.0.0, 1.3.0, 1.2.1, 1.0.4, 0.98.19, 1.1.5 > > Attachments: HBASE-14845.1.patch > > > HBASE-13963 / HBASE-14844 take care of removing leaks of our dependency on > jdk-tools. > Until we move the mapreduce support classes out of hbase-server > (HBASE-11843), we need to also avoid leaking the dependency from that module. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15488) Add ACL for setting split merge switch
[ https://issues.apache.org/jira/browse/HBASE-15488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15488: --- Fix Version/s: (was: 1.3.0) 1.4.0 Status: Patch Available (was: Open) > Add ACL for setting split merge switch > -- > > Key: HBASE-15488 > URL: https://issues.apache.org/jira/browse/HBASE-15488 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-15488.v1.patch > > > Currently there is no access control for the split merge switch setter in > MasterRpcServices. > This JIRA adds necessary coprocessor hook along with enforcing permission > check in AccessController through the new hook. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14870) Backport namespace permissions to 98 branch
[ https://issues.apache.org/jira/browse/HBASE-14870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-14870: --- Fix Version/s: (was: 0.98.18) 0.98.19 > Backport namespace permissions to 98 branch > --- > > Key: HBASE-14870 > URL: https://issues.apache.org/jira/browse/HBASE-14870 > Project: HBase > Issue Type: Task >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.98.19 > > > Backport namespace permissions to 0.98. The new permission checks will be > disabled by default for behavioral compatibility with previous releases, like > what we did when we introduced enforcement of the EXEC permission. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15406) Split / merge switch left disabled after early termination of hbck
[
https://issues.apache.org/jira/browse/HBASE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201495#comment-15201495
]
Heng Chen commented on HBASE-15406:
---
Think about this situation:
1. before the first hbck run, splitAndMerge is true
2. hbck abort, splitAndMerge is false, we will keep the state on zk as
original state, and rerun will restore this state firstly.
3. user uses command set splitAndMerge to be true
4. hbck rerun, first it restore the original state left by first hbck to be
false, then do normal operation (acquire the lock and change the switch). At
the end of hbck, the switch state will be false. But user has set it to be
true.
> Split / merge switch left disabled after early termination of hbck
> --
>
> Key: HBASE-15406
> URL: https://issues.apache.org/jira/browse/HBASE-15406
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15406.patch, HBASE-15406.v1.patch,
> HBASE-15406_v1.patch, test.patch, wip.patch
>
>
> This was what I did on cluster with 1.4.0-SNAPSHOT built Thursday:
> Run 'hbase hbck -disableSplitAndMerge' on gateway node of the cluster
> Terminate hbck early
> Enter hbase shell where I observed:
> {code}
> hbase(main):001:0> splitormerge_enabled 'SPLIT'
> false
> 0 row(s) in 0.3280 seconds
> hbase(main):002:0> splitormerge_enabled 'MERGE'
> false
> 0 row(s) in 0.0070 seconds
> {code}
> Expectation is that the split / merge switches should be restored to default
> value after hbck exits.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15481) Add pre/post roll to WALObserver
[ https://issues.apache.org/jira/browse/HBASE-15481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201977#comment-15201977 ] Matteo Bertozzi commented on HBASE-15481: - no you add extra stuff that does not make sense. Since instead of just subscribing to a listener you have to Implement from a coprocessor and add yourself in the coprocessor list. You can't probably use the base coproc class because you may already inherit from something so you end up with lots of methods that you don't need. (aside the extra cp perf overhead) > Add pre/post roll to WALObserver > > > Key: HBASE-15481 > URL: https://issues.apache.org/jira/browse/HBASE-15481 > Project: HBase > Issue Type: New Feature >Affects Versions: 1.3.0 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15481-v0.patch, HBASE-15481-v1.patch > > > currently the WALObserver has only a pre/post Write. It will be useful to > have a pre/post Roll too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15474) Exception in HConnectionImplementation's constructor cause Zookeeper connnections leak
[
https://issues.apache.org/jira/browse/HBASE-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202341#comment-15202341
]
Ted Yu commented on HBASE-15474:
You can name your patch HBASE-15474-branch-1.v2.patch
> Exception in HConnectionImplementation's constructor cause Zookeeper
> connnections leak
> ---
>
> Key: HBASE-15474
> URL: https://issues.apache.org/jira/browse/HBASE-15474
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Allan Yang
>Assignee: Allan Yang
> Labels: patch
> Fix For: 1.1.0
>
> Attachments: HBASE-15474v2.patch
>
>
> HConnectionImplementation creates a ZooKeeperKeepAliveConnection during
> construction, but if the constructor throw a exception, the zookeeper
> connection is not properly closed.
> {code}
> HConnectionImplementation(Configuration conf, boolean managed,
> ExecutorService pool, User user) throws IOException {
> this(conf);
> this.user = user;
> this.batchPool = pool;
> this.managed = managed;
> this.registry = setupRegistry();
> retrieveClusterId(); //here is the zookeeper connection created
> this.rpcClient = RpcClientFactory.createClient(this.conf,
> this.clusterId);// In our case, the exception happens here, so the zookeeper
> connection never closes
> this.rpcControllerFactory = RpcControllerFactory.instantiate(conf);
> ..
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15482) Provide an option to skip calculating block locations for SnapshotInputFormat
[ https://issues.apache.org/jira/browse/HBASE-15482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202170#comment-15202170 ] Dave Latham commented on HBASE-15482: - Liyin, you mentioned it being outside of the scope of this jira, but we use hdfs snapshots rather than hbase snapshots for reasons like that, and have an adapted input format for reading from them. If there is interest we could look at trying to share that better, can see what shape it is in. > Provide an option to skip calculating block locations for SnapshotInputFormat > - > > Key: HBASE-15482 > URL: https://issues.apache.org/jira/browse/HBASE-15482 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Reporter: Liyin Tang >Priority: Minor > > When a MR job is reading from SnapshotInputFormat, it needs to calculate the > splits based on the block locations in order to get best locality. However, > this process may take a long time for large snapshots. > In some setup, the computing layer, Spark, Hive or Presto could run out side > of HBase cluster. In these scenarios, the block locality doesn't matter. > Therefore, it will be great to have an option to skip calculating the block > locations for every job. That will super useful for the Hive/Presto/Spark > connectors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
