[jira] [Created] (HBASE-17389) Convert all internal usages from ReplicationAdmin to Admin
Guanghao Zhang created HBASE-17389: -- Summary: Convert all internal usages from ReplicationAdmin to Admin Key: HBASE-17389 URL: https://issues.apache.org/jira/browse/HBASE-17389 Project: HBase Issue Type: Sub-task Affects Versions: 2.0.0 Reporter: Guanghao Zhang Fix For: 2.0.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-17388) Move ReplicationPeer and other replication related PB messages to the replication.proto
Guanghao Zhang created HBASE-17388: -- Summary: Move ReplicationPeer and other replication related PB messages to the replication.proto Key: HBASE-17388 URL: https://issues.apache.org/jira/browse/HBASE-17388 Project: HBase Issue Type: Sub-task Components: Replication Affects Versions: 2.0.0 Reporter: Guanghao Zhang Assignee: Guanghao Zhang Fix For: 2.0.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-17387) Reduce the overhead of exception report in RegionActionResult for multi()
Ted Yu created HBASE-17387:
--
Summary: Reduce the overhead of exception report in
RegionActionResult for multi()
Key: HBASE-17387
URL: https://issues.apache.org/jira/browse/HBASE-17387
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
For RSRpcServices#doNonAtomicRegionMutation() :
{code}
for (ClientProtos.Action action: actions.getActionList()) {
...
} catch (IOException ie) {
rpcServer.getMetrics().exception(ie);
resultOrExceptionBuilder = ResultOrException.newBuilder().
setException(ResponseConverter.buildException(ie));
}
if (resultOrExceptionBuilder != null) {
// Propagate index.
resultOrExceptionBuilder.setIndex(action.getIndex());
builder.addResultOrException(resultOrExceptionBuilder.build());
}
{code}
The exceptions are added to builder in the for loop.
The ClientProtos.ResultOrException.Builder instance is created within the for
loop.
For large multi call, this may incur non-trivial overhead for garbage collector
if there're many exceptions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-17386) Adding FN documentation to reference guide
Thiruvel Thirumoolan created HBASE-17386: Summary: Adding FN documentation to reference guide Key: HBASE-17386 URL: https://issues.apache.org/jira/browse/HBASE-17386 Project: HBase Issue Type: Sub-task Reporter: Thiruvel Thirumoolan Assignee: Thiruvel Thirumoolan -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-17385) Change usage documentation from bin/hbase to hbase in various tools
Enis Soztutar created HBASE-17385:
-
Summary: Change usage documentation from bin/hbase to hbase in
various tools
Key: HBASE-17385
URL: https://issues.apache.org/jira/browse/HBASE-17385
Project: HBase
Issue Type: Bug
Reporter: Enis Soztutar
Some tools print bin/hbase in their usage documentation.{{bin/hbase}} is only
applicable to development environments. Typical deployments always should just
refer to {{hbase}}.
For example, CopyTable usage is like:
{code}
Usage: CopyTable [general options] [--starttime=X] [--endtime=Y]
[--new.name=NEW] [--peer.adr=ADR]
Options:
rs.class hbase.regionserver.class of the peer cluster
specify if different from current cluster
rs.impl hbase.regionserver.impl of the peer cluster
startrow the start row
stoprow the stop row
starttimebeginning of the time range (unixtime in millis)
without endtime means from starttime to forever
endtime end of the time range. Ignored if no starttime specified.
versions number of cell versions to copy
new.name new table's name
peer.adr Address of the peer cluster given in the format
hbase.zookeeper.quorum:hbase.zookeeper.client.port:zookeeper.znode.parent
families comma-separated list of families to copy
To copy from cf1 to cf2, give sourceCfName:destCfName.
To keep the same name, just give "cfName"
all.cellsalso copy delete markers and deleted cells
bulkload Write input into HFiles and bulk load to the destination table
Args:
tablenameName of the table to copy
Examples:
To copy 'TestTable' to a cluster that uses replication for a 1 hour window:
$ bin/hbase org.apache.hadoop.hbase.mapreduce.CopyTable
--starttime=1265875194289 --endtime=1265878794289
--peer.adr=server1,server2,server3:2181:/hbase
--families=myOldCf:myNewCf,cf2,cf3 TestTable
For performance consider the following general option:
It is recommended that you set the following to >=100. A higher value uses
more memory but
decreases the round trip time to the server and may increase performance.
-Dhbase.client.scanner.caching=100
The following should always be set to false, to prevent writing data twice,
which may produce
inaccurate results.
-Dmapreduce.map.speculative=false
{code}
in above, it should be:
{code}
To copy 'TestTable' to a cluster that uses replication for a 1 hour window:
$ hbase org.apache.hadoop.hbase.mapreduce.CopyTable --starttime=1265875194289
--endtime=1265878794289 --peer.adr=server1,server2,server3:2181:/hbase --
families=myOldCf:myNewCf,cf2,cf3 TestTable
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (HBASE-17149) Procedure V2 - Fix nonce submission to avoid unnecessary calling coprocessor multiple times
[ https://issues.apache.org/jira/browse/HBASE-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Yuan Jiang reopened HBASE-17149: > Procedure V2 - Fix nonce submission to avoid unnecessary calling coprocessor > multiple times > --- > > Key: HBASE-17149 > URL: https://issues.apache.org/jira/browse/HBASE-17149 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.1.7, 1.2.4 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi > Fix For: 2.0.0 > > Attachments: 17149.branch-1.incomplete.txt, > HBASE-17149.master.001.patch, HBASE-17149.master.002.patch, > HBASE-17149.master.002.patch, HBASE-17149.master.002.patch, > HBASE-17149.master.003.patch, nonce.patch > > > instead of having all the logic in submitProcedure(), split in > registerNonce() + submitProcedure(). > In this case we can avoid calling the coprocessor twice and having a clean > submit logic knowing that there will only be one submission. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (HBASE-17081) Flush the entire CompactingMemStore content to disk
[ https://issues.apache.org/jira/browse/HBASE-17081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu reopened HBASE-17081: Reopen due to reproducible test failure in TestHRegionWithInMemoryFlush > Flush the entire CompactingMemStore content to disk > --- > > Key: HBASE-17081 > URL: https://issues.apache.org/jira/browse/HBASE-17081 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Fix For: 2.0.0 > > Attachments: HBASE-15787_8.patch, HBASE-17081-V01.patch, > HBASE-17081-V02.patch, HBASE-17081-V03.patch, HBASE-17081-V04.patch, > HBASE-17081-V05.patch, HBASE-17081-V06.patch, HBASE-17081-V06.patch, > HBASE-17081-V07.patch, HBASE-17081-V10.patch, > HBaseMeetupDecember2016-V02.pptx, Pipelinememstore_fortrunk_3.patch > > > Part of CompactingMemStore's memory is held by an active segment, and another > part is divided between immutable segments in the compacting pipeline. Upon > flush-to-disk request we want to flush all of it to disk, in contrast to > flushing only tail of the compacting pipeline. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-17384) Consider aborting region server when MVCC#waitForRead() gets stuck
Ted Yu created HBASE-17384:
--
Summary: Consider aborting region server when MVCC#waitForRead()
gets stuck
Key: HBASE-17384
URL: https://issues.apache.org/jira/browse/HBASE-17384
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
>From
>https://builds.apache.org/job/PreCommit-HBASE-Build/5072/testReport/org.apache.hadoop.hbase.regionserver/TestHRegionWithInMemoryFlush/org_apache_hadoop_hbase_regionserver_TestHRegionWithInMemoryFlush/
> :
{code}
org.junit.runners.model.TestTimedOutException: test timed out after 10 minutes
at java.lang.Object.wait(Native Method)
at
org.apache.hadoop.hbase.regionserver.MultiVersionConcurrencyControl.waitForRead(MultiVersionConcurrencyControl.java:218)
at
org.apache.hadoop.hbase.regionserver.MultiVersionConcurrencyControl.completeAndWait(MultiVersionConcurrencyControl.java:149)
at
org.apache.hadoop.hbase.regionserver.HRegion.getNextSequenceId(HRegion.java:2732)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalPrepareFlushCache(HRegion.java:2447)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2343)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2314)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2304)
at
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1601)
at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1506)
at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1456)
at
org.apache.hadoop.hbase.HBaseTestingUtility.closeRegionAndWAL(HBaseTestingUtility.java:374)
at
org.apache.hadoop.hbase.regionserver.TestHRegion.testFlushCacheWhileScanning(TestHRegion.java:3839)
{code}
As can be seen from test output:
{code}
2016-12-28 13:43:28,379 INFO [Time-limited test] regionserver.HStore(1431):
Completed major compaction of 1 (all) file(s) in family1 of
testWritesWhileScanning,,1482932605883.2e46061b97a54d7f8434c4a705b3c4a2. into
255e7eb61cfc4945ac5887957d39b1fe(size=98.0 K), total size for store is 98.0 K
...[truncated 4062267 bytes]...
TUCK: MultiVersionConcurrencyControl{readPoint=1090, writePoint=1093}
2016-12-28 13:48:29,396 WARN [Time-limited test]
regionserver.MultiVersionConcurrencyControl(214): STUCK:
MultiVersionConcurrencyControl{readPoint=1090, writePoint=1093}
2016-12-28 13:48:30,406 WARN [Time-limited test]
regionserver.MultiVersionConcurrencyControl(214): STUCK:
MultiVersionConcurrencyControl{readPoint=1090, writePoint=1093}
2016-12-28 13:48:31,416 WARN [Time-limited test]
regionserver.MultiVersionConcurrencyControl(214): STUCK:
MultiVersionConcurrencyControl{readPoint=1090, writePoint=1093}
{code}
At least 5 minutes passed with the above log showing waitForRead() stuck.
Since the flush is blocked, we should consider aborting region server when
waitForRead() gets stuck for extended period of time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Successful: HBase Generate Website
Build status: Successful
If successful, the website and docs have been generated. To update the live
site, follow the instructions below. If failed, skip to the bottom of this
email.
Use the following commands to download the patch and apply it to a clean branch
based on origin/asf-site. If you prefer to keep the hbase-site repo around
permanently, you can skip the clone step.
git clone https://git-wip-us.apache.org/repos/asf/hbase-site.git
cd hbase-site
wget -O-
https://builds.apache.org/job/hbase_generate_website/448/artifact/website.patch.zip
| funzip > da97569eae662ad90fd3afd98ef148c94eee4ac1.patch
git fetch
git checkout -b asf-site-da97569eae662ad90fd3afd98ef148c94eee4ac1
origin/asf-site
git am --whitespace=fix da97569eae662ad90fd3afd98ef148c94eee4ac1.patch
At this point, you can preview the changes by opening index.html or any of the
other HTML pages in your local
asf-site-da97569eae662ad90fd3afd98ef148c94eee4ac1 branch.
There are lots of spurious changes, such as timestamps and CSS styles in
tables, so a generic git diff is not very useful. To see a list of files that
have been added, deleted, renamed, changed type, or are otherwise interesting,
use the following command:
git diff --name-status --diff-filter=ADCRTXUB origin/asf-site
To see only files that had 100 or more lines changed:
git diff --stat origin/asf-site | grep -E '[1-9][0-9]{2,}'
When you are satisfied, publish your changes to origin/asf-site using these
commands:
git commit --allow-empty -m "Empty commit" # to work around a current ASF
INFRA bug
git push origin asf-site-da97569eae662ad90fd3afd98ef148c94eee4ac1:asf-site
git checkout asf-site
git branch -D asf-site-da97569eae662ad90fd3afd98ef148c94eee4ac1
Changes take a couple of minutes to be propagated. You can verify whether they
have been propagated by looking at the Last Published date at the bottom of
http://hbase.apache.org/. It should match the date in the index.html on the
asf-site branch in Git.
As a courtesy- reply-all to this email to let other committers know you pushed
the site.
If failed, see https://builds.apache.org/job/hbase_generate_website/448/console
[jira] [Created] (HBASE-17383) Improve log msg when offheap memstore breaches higher water mark
ramkrishna.s.vasudevan created HBASE-17383:
--
Summary: Improve log msg when offheap memstore breaches higher
water mark
Key: HBASE-17383
URL: https://issues.apache.org/jira/browse/HBASE-17383
Project: HBase
Issue Type: Improvement
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Trivial
Fix For: 2.0.0
Currently we get this log
{code}
2016-12-28 21:11:14,349 INFO
[RpcServer.deafult.FPBQ.Fifo.handler=39,queue=9,port=16041]
regionserver.MemStoreFlusher: Blocking updates on
stobdtserver5,16041,1482938527980: the global offheap memstore size 12.6 G +
global memstore heap overhead 4.0 G is >= than blocking 12.6 G size
{code}
Here the global offheap memstore size is greater than the blocking size. The
memstore heap overhead need not be included in this log unless the higher water
mark breach is only due to the heap overhead.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-17382) Give RegionLocateType a better name
Duo Zhang created HBASE-17382: - Summary: Give RegionLocateType a better name Key: HBASE-17382 URL: https://issues.apache.org/jira/browse/HBASE-17382 Project: HBase Issue Type: Improvement Reporter: Duo Zhang Pointed out by [~tedyu] that 'Locate' is a verb and usually we need a noun here. 'Locating' or 'Location'? Suggestion are welcomed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
