[jira] [Updated] (HBASE-8666) META region isn't fully recovered during master initialization when META region recovery had chained failures
[ https://issues.apache.org/jira/browse/HBASE-8666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Zhong updated HBASE-8666: - Attachment: hbase-8666-v3.patch Cover one more situation such as znodes under META recovering region isn't empty when Master starts up. > META region isn't fully recovered during master initialization when META > region recovery had chained failures > - > > Key: HBASE-8666 > URL: https://issues.apache.org/jira/browse/HBASE-8666 > Project: HBase > Issue Type: Bug > Components: MTTR >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Fix For: 0.98.0, 0.95.2 > > Attachments: hbase-8666.patch, hbase-8666-v2.patch, > hbase-8666-v3.patch > > > In distributedLogReplay mode when Meta recovery had experienced chained > failures(recovery failed multiple times in a row), META region can't be fully > recovered during master starts up. > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8676) hbase.hstore.compaction.max.size parameter has no effect in minor compaction
Calvin Qu created HBASE-8676: Summary: hbase.hstore.compaction.max.size parameter has no effect in minor compaction Key: HBASE-8676 URL: https://issues.apache.org/jira/browse/HBASE-8676 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.94.7 Reporter: Calvin Qu In hbase-site.xml,I set(10G) hbase.hstore.compaction.max.size 10737418240 but in regionserver's log,a minor compaction deals with 10 Hfiles,include a 220G HFile larger than hbase.hstore.compaction.max.size=10G. 2013-06-03 10:27:51,147 DEBUG org.apache.hadoop.hbase.regionserver.Store: 8b6a4d4aae3099730b353183cf754ec4 - t: Initiating minorcompaction 2013-06-03 10:27:51,149 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on t in region transdb,1651763699103243149223370692144515807,1369372490168.8b6a4d4aae3099730b353183cf754ec4. 2013-06-03 10:27:51,150 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Large Compaction requested: regionName=transdb,1651763699103243149223370692144515807,1369372490168.8b6a4d4aae3099730b353183cf754ec4., storeName=t, fileCount=10, fileSize=234.4g (106.0m, 2.0g, 2.1g, 10.7m, 1.7g, 2.1g, 1.9g, 2.0g, 932.8m, 221.5g), priority=-110, time=2671513455305347; Because: Opening Region; compaction_queue=(0:0), split_queue=0 2013-06-03 10:27:51,151 INFO org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 10 file(s) in t of transdb,1651763699103243149223370692144515807,1369372490168.8b6a4d4aae3099730b353183cf754ec4. into tmpdir=hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/.tmp, seqid=0, totalSize=234.4g 2013-06-03 10:27:51,152 DEBUG org.apache.hadoop.hbase.regionserver.Compactor: Compacting hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/0a5fb3caf6be490ba739b77fcaa4b274.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/0a5fb3caf6be490ba739b77fcaa4b274-top, keycount=7948080, bloomtype=NONE, size=106.0m, encoding=NONE 2013-06-03 10:27:51,153 DEBUG org.apache.hadoop.hbase.regionserver.Compactor: Compacting hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/6bd849c93e3a45d4a4c8a5ab691f9b22.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/6bd849c93e3a45d4a4c8a5ab691f9b22-top, keycount=153911753, bloomtype=NONE, size=2.0g, encoding=NONE 2013-06-03 10:27:51,153 DEBUG org.apache.hadoop.hbase.regionserver.Compactor: Compacting hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/191ef6d1076341e1b7fb5c775d0f9fdb.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/191ef6d1076341e1b7fb5c775d0f9fdb-top, keycount=159372725, bloomtype=NONE, size=2.1g, encoding=NONE 2013-06-03 10:27:51,154 DEBUG org.apache.hadoop.hbase.regionserver.Compactor: Compacting hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/d907a7dca7d8466f843f126429f05665.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/d907a7dca7d8466f843f126429f05665-top, keycount=808800, bloomtype=NONE, size=10.7m, encoding=NONE 2013-06-03 10:27:51,154 DEBUG org.apache.hadoop.hbase.regionserver.Compactor: Compacting hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/35399e73e3764d0b88251c7285cf5406.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/35399e73e3764d0b88251c7285cf5406-top, keycount=133146091, bloomtype=NONE, size=1.7g, encoding=NONE 2013-06-03 10:27:51,154 DEBUG org.apache.hadoop.hbase.regionserver.Compactor: Compacting hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/9d5583ffde334bc79a692d400af7bae0.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/9d5583ffde334bc79a692d400af7bae0-top, keycount=159372708, bloomtype=NONE, size=2.1g, encoding=NONE 2013-06-03 10:27:51,155 DEBUG org.apache.hadoop.hbase.regionserver.Compactor: Compacting hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/506254eac1bb455283f578e71af23153.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/506254eac1bb455283f578e71af23153-top, keycount=152578349, bloomtype=NONE, size=1.9g, encoding=NONE 2013-06-03 10:27:51,155 DEBUG org.apache.hadoop.hbase.regionserver.Compactor: Compacting hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/72db7a39a2694bf58782e95992b012ae.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/72db7a39a2694bf58782e95992b012ae-top, keycount=159369079, bloomtype=NONE, size=2.0g, encoding=NONE 2013-06-03 10:27:51,155 DEBUG o
[jira] [Commented] (HBASE-8640) ServerName in master may not initialize with the configured ipc address of hbase.master.ipc.address
[
https://issues.apache.org/jira/browse/HBASE-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13672843#comment-13672843
]
Hudson commented on HBASE-8640:
---
Integrated in HBase-0.94 #1003 (See
[https://builds.apache.org/job/HBase-0.94/1003/])
HBASE-8640 ServerName in master may not initialize with the configured ipc
address of hbase.master.ipc.address (Revision 1488837)
Result = SUCCESS
stack :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
> ServerName in master may not initialize with the configured ipc address of
> hbase.master.ipc.address
> ---
>
> Key: HBASE-8640
> URL: https://issues.apache.org/jira/browse/HBASE-8640
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: rajeshbabu
>Assignee: rajeshbabu
> Fix For: 0.98.0, 0.95.2, 0.94.9
>
> Attachments: HBASE-8640_94.patch, HBASE-8640.patch
>
>
> We are starting rpc server with default interface hostname or configured ipc
> address
> {code}
> this.rpcServer = HBaseRPC.getServer(this,
> new Class[]{HMasterInterface.class, HMasterRegionInterface.class},
> initialIsa.getHostName(), // This is bindAddress if set else it's
> hostname
> initialIsa.getPort(),
> numHandlers,
> 0, // we dont use high priority handlers in master
> conf.getBoolean("hbase.rpc.verbose", false), conf,
> 0); // this is a DNC w/o high priority handlers
> {code}
> But we are initialzing servername with default hostname always master znode
> also have this hostname.
> {code}
> String hostname = Strings.domainNamePointerToHostName(DNS.getDefaultHost(
> conf.get("hbase.master.dns.interface", "default"),
> conf.get("hbase.master.dns.nameserver", "default")));
> ...
> this.serverName = new ServerName(hostname,
> this.isa.getPort(), System.currentTimeMillis());
> {code}
> If both default interface hostname and configured ipc address are not same
> clients will get MasterNotRunningException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8640) ServerName in master may not initialize with the configured ipc address of hbase.master.ipc.address
[
https://issues.apache.org/jira/browse/HBASE-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13672841#comment-13672841
]
Hudson commented on HBASE-8640:
---
Integrated in hbase-0.95 #224 (See
[https://builds.apache.org/job/hbase-0.95/224/])
HBASE-8640 ServerName in master may not initialize with the configured ipc
address of hbase.master.ipc.address (Revision 1488836)
Result = SUCCESS
stack :
Files :
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
> ServerName in master may not initialize with the configured ipc address of
> hbase.master.ipc.address
> ---
>
> Key: HBASE-8640
> URL: https://issues.apache.org/jira/browse/HBASE-8640
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: rajeshbabu
>Assignee: rajeshbabu
> Fix For: 0.98.0, 0.95.2, 0.94.9
>
> Attachments: HBASE-8640_94.patch, HBASE-8640.patch
>
>
> We are starting rpc server with default interface hostname or configured ipc
> address
> {code}
> this.rpcServer = HBaseRPC.getServer(this,
> new Class[]{HMasterInterface.class, HMasterRegionInterface.class},
> initialIsa.getHostName(), // This is bindAddress if set else it's
> hostname
> initialIsa.getPort(),
> numHandlers,
> 0, // we dont use high priority handlers in master
> conf.getBoolean("hbase.rpc.verbose", false), conf,
> 0); // this is a DNC w/o high priority handlers
> {code}
> But we are initialzing servername with default hostname always master znode
> also have this hostname.
> {code}
> String hostname = Strings.domainNamePointerToHostName(DNS.getDefaultHost(
> conf.get("hbase.master.dns.interface", "default"),
> conf.get("hbase.master.dns.nameserver", "default")));
> ...
> this.serverName = new ServerName(hostname,
> this.isa.getPort(), System.currentTimeMillis());
> {code}
> If both default interface hostname and configured ipc address are not same
> clients will get MasterNotRunningException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8640) ServerName in master may not initialize with the configured ipc address of hbase.master.ipc.address
[
https://issues.apache.org/jira/browse/HBASE-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13672842#comment-13672842
]
Hudson commented on HBASE-8640:
---
Integrated in HBase-TRUNK #4155 (See
[https://builds.apache.org/job/HBase-TRUNK/4155/])
HBASE-8640 ServerName in master may not initialize with the configured ipc
address of hbase.master.ipc.address (Revision 1488834)
Result = SUCCESS
stack :
Files :
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
> ServerName in master may not initialize with the configured ipc address of
> hbase.master.ipc.address
> ---
>
> Key: HBASE-8640
> URL: https://issues.apache.org/jira/browse/HBASE-8640
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: rajeshbabu
>Assignee: rajeshbabu
> Fix For: 0.98.0, 0.95.2, 0.94.9
>
> Attachments: HBASE-8640_94.patch, HBASE-8640.patch
>
>
> We are starting rpc server with default interface hostname or configured ipc
> address
> {code}
> this.rpcServer = HBaseRPC.getServer(this,
> new Class[]{HMasterInterface.class, HMasterRegionInterface.class},
> initialIsa.getHostName(), // This is bindAddress if set else it's
> hostname
> initialIsa.getPort(),
> numHandlers,
> 0, // we dont use high priority handlers in master
> conf.getBoolean("hbase.rpc.verbose", false), conf,
> 0); // this is a DNC w/o high priority handlers
> {code}
> But we are initialzing servername with default hostname always master znode
> also have this hostname.
> {code}
> String hostname = Strings.domainNamePointerToHostName(DNS.getDefaultHost(
> conf.get("hbase.master.dns.interface", "default"),
> conf.get("hbase.master.dns.nameserver", "default")));
> ...
> this.serverName = new ServerName(hostname,
> this.isa.getPort(), System.currentTimeMillis());
> {code}
> If both default interface hostname and configured ipc address are not same
> clients will get MasterNotRunningException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8675) Two active Hmasters for AUTH_FAILED in secure hbase cluster
[ https://issues.apache.org/jira/browse/HBASE-8675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Shaohui updated HBASE-8675: --- Summary: Two active Hmasters for AUTH_FAILED in secure hbase cluster (was: Two active Hmaster for AUTH_FAILED in secure hbase cluster) > Two active Hmasters for AUTH_FAILED in secure hbase cluster > --- > > Key: HBASE-8675 > URL: https://issues.apache.org/jira/browse/HBASE-8675 > Project: HBase > Issue Type: Bug > Components: master >Reporter: Liu Shaohui >Priority: Critical > > In our product cluster, because of the net problem to kerberos server, the > ZooKeeperWatcher in active hmaster fails to Auth , gets a connection Event of > AUTH_FAILED and loose the master lock. But the zookeeper watcher ignores the > event, so the old active hmaster keeps to be active. After the net problem is > fixed, the backup hmaster gets the master lock and becomes active. There are > two two active hmasters in the cluster. > 2013-05-30 09:44:21,004 ERROR > org.apache.zookeeper.client.ZooKeeperSaslClient: An error: > (java.security.PrivilegedActionException: javax.security.sasl.SaslException: > GSS initiate failed [Caused by GSSException: No valid credentials provided > (Mechanism level: krb1.xiaomi.net)]) occurred when evaluating Zookeeper > Quorum Member's received SASL token. Zookeeper Client will go to AUTH_FAILED > state. > 2013-05-30 09:54:07,755 WARN org.apache.hadoop.hbase.zookeeper.ZKUtil: > hconnection-0x3e10d98be405bc Unable to set watcher on znode /hbase/master > org.apache.zookeeper.KeeperException$AuthFailedException: KeeperErrorCode = > AuthFailed for /hbase/master > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:123) > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:51) > at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1036) > at > org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:166) > at > org.apache.hadoop.hbase.zookeeper.ZKUtil.watchAndCheckExists(ZKUtil.java:231) > at > org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:76) > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.ensureZookeeperTrackers(HConnectionManager.java:595) > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:850) > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:825) > at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:286) > at org.apache.hadoop.hbase.client.HTable.(HTable.java:201) > at > org.apache.hadoop.hbase.catalog.MetaReader.getHTable(MetaReader.java:200) > at > org.apache.hadoop.hbase.catalog.MetaReader.getMetaHTable(MetaReader.java:226) > at > org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:705) > at > org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:183) > at > org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:168) > at > org.apache.hadoop.hbase.master.CatalogJanitor.getSplitParents(CatalogJanitor.java:123) > at > org.apache.hadoop.hbase.master.CatalogJanitor.scan(CatalogJanitor.java:134) > at > org.apache.hadoop.hbase.master.CatalogJanitor.chore(CatalogJanitor.java:92) > at org.apache.hadoop.hbase.Chore.run(Chore.java:67) > at java.lang.Thread.run(Thread.java:662) > I want to just abort the hmaster server if AuthFailed or SaslAuthenticated. > Any better idea about this issue? > For ZookeeperWatcher is used in many classes, will the aborting will bring > more problems? Any more problems we need consider? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8640) ServerName in master may not initialize with the configured ipc address of hbase.master.ipc.address
[
https://issues.apache.org/jira/browse/HBASE-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13672832#comment-13672832
]
Hudson commented on HBASE-8640:
---
Integrated in HBase-0.94-security #157 (See
[https://builds.apache.org/job/HBase-0.94-security/157/])
HBASE-8640 ServerName in master may not initialize with the configured ipc
address of hbase.master.ipc.address (Revision 1488837)
Result = SUCCESS
stack :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
> ServerName in master may not initialize with the configured ipc address of
> hbase.master.ipc.address
> ---
>
> Key: HBASE-8640
> URL: https://issues.apache.org/jira/browse/HBASE-8640
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: rajeshbabu
>Assignee: rajeshbabu
> Fix For: 0.98.0, 0.95.2, 0.94.9
>
> Attachments: HBASE-8640_94.patch, HBASE-8640.patch
>
>
> We are starting rpc server with default interface hostname or configured ipc
> address
> {code}
> this.rpcServer = HBaseRPC.getServer(this,
> new Class[]{HMasterInterface.class, HMasterRegionInterface.class},
> initialIsa.getHostName(), // This is bindAddress if set else it's
> hostname
> initialIsa.getPort(),
> numHandlers,
> 0, // we dont use high priority handlers in master
> conf.getBoolean("hbase.rpc.verbose", false), conf,
> 0); // this is a DNC w/o high priority handlers
> {code}
> But we are initialzing servername with default hostname always master znode
> also have this hostname.
> {code}
> String hostname = Strings.domainNamePointerToHostName(DNS.getDefaultHost(
> conf.get("hbase.master.dns.interface", "default"),
> conf.get("hbase.master.dns.nameserver", "default")));
> ...
> this.serverName = new ServerName(hostname,
> this.isa.getPort(), System.currentTimeMillis());
> {code}
> If both default interface hostname and configured ipc address are not same
> clients will get MasterNotRunningException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8642) [Snapshot] List and delete snapshot by table
[
https://issues.apache.org/jira/browse/HBASE-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13672817#comment-13672817
]
Hadoop QA commented on HBASE-8642:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585796/8642-trunk-0.95-v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5907//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5907//console
This message is automatically generated.
> [Snapshot] List and delete snapshot by table
>
>
> Key: HBASE-8642
> URL: https://issues.apache.org/jira/browse/HBASE-8642
> Project: HBase
> Issue Type: Improvement
> Components: snapshots
>Affects Versions: 0.98.0, 0.95.0, 0.95.1, 0.95.2
>Reporter: Julian Zhou
>Assignee: Julian Zhou
>Priority: Minor
> Fix For: 0.98.0, 0.95.0, 0.95.1, 0.95.2
>
> Attachments: 8642-trunk-0.95-v0.patch, 8642-trunk-0.95-v1.patch,
> 8642-trunk-0.95-v2.patch
>
>
> Support list and delete snapshot by table name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8675) Two active Hmaster for AUTH_FAILED in secure hbase cluster
Liu Shaohui created HBASE-8675: -- Summary: Two active Hmaster for AUTH_FAILED in secure hbase cluster Key: HBASE-8675 URL: https://issues.apache.org/jira/browse/HBASE-8675 Project: HBase Issue Type: Bug Components: master Reporter: Liu Shaohui Priority: Critical In our product cluster, because of the net problem to kerberos server, the ZooKeeperWatcher in active hmaster fails to Auth , gets a connection Event of AUTH_FAILED and loose the master lock. But the zookeeper watcher ignores the event, so the old active hmaster keeps to be active. After the net problem is fixed, the backup hmaster gets the master lock and becomes active. There are two two active hmasters in the cluster. 2013-05-30 09:44:21,004 ERROR org.apache.zookeeper.client.ZooKeeperSaslClient: An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: krb1.xiaomi.net)]) occurred when evaluating Zookeeper Quorum Member's received SASL token. Zookeeper Client will go to AUTH_FAILED state. 2013-05-30 09:54:07,755 WARN org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x3e10d98be405bc Unable to set watcher on znode /hbase/master org.apache.zookeeper.KeeperException$AuthFailedException: KeeperErrorCode = AuthFailed for /hbase/master at org.apache.zookeeper.KeeperException.create(KeeperException.java:123) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1036) at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:166) at org.apache.hadoop.hbase.zookeeper.ZKUtil.watchAndCheckExists(ZKUtil.java:231) at org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:76) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.ensureZookeeperTrackers(HConnectionManager.java:595) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:850) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:825) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:286) at org.apache.hadoop.hbase.client.HTable.(HTable.java:201) at org.apache.hadoop.hbase.catalog.MetaReader.getHTable(MetaReader.java:200) at org.apache.hadoop.hbase.catalog.MetaReader.getMetaHTable(MetaReader.java:226) at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:705) at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:183) at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:168) at org.apache.hadoop.hbase.master.CatalogJanitor.getSplitParents(CatalogJanitor.java:123) at org.apache.hadoop.hbase.master.CatalogJanitor.scan(CatalogJanitor.java:134) at org.apache.hadoop.hbase.master.CatalogJanitor.chore(CatalogJanitor.java:92) at org.apache.hadoop.hbase.Chore.run(Chore.java:67) at java.lang.Thread.run(Thread.java:662) I want to just abort the hmaster server if AuthFailed or SaslAuthenticated. Any better idea about this issue? For ZookeeperWatcher is used in many classes, will the aborting will bring more problems? Any more problems we need consider? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-8673) Make code involving class.getMethod() more amenable to mocking
[
https://issues.apache.org/jira/browse/HBASE-8673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-8673.
--
Resolution: Invalid
Resolving trivial, issue w/o description of why it needs to be fixed nor of how
it is to be fixed.
I also believe this a non-issue. See
hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestFSHDFSUtils.java
for example of how to mock a DFS with a method that may or may not be present.
> Make code involving class.getMethod() more amenable to mocking
> --
>
> Key: HBASE-8673
> URL: https://issues.apache.org/jira/browse/HBASE-8673
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Priority: Minor
>
> In some place, such as the following (FSUtils#isInSafeMode), we retrieve
> method using getMethod() with hardcoded class name:
> {code}
> private static boolean isInSafeMode(DistributedFileSystem dfs) throws
> IOException {
> boolean inSafeMode = false;
> try {
> Method m = DistributedFileSystem.class.getMethod("setSafeMode", new
> Class []{
> {code}
> The above code is not friendly to mocking.
> The test class may define its own DistributedFileSystem descendent class and
> pass an instance to FSUtils#isInSafeMode(). In that case, assertion similar
> to the following would fail:
> {code}
> Mockito.verify(dfs, Mockito.times(1)).setSafeMode();
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8607) Run HBase server in an OSGi container
[ https://issues.apache.org/jira/browse/HBASE-8607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13672799#comment-13672799 ] stack commented on HBASE-8607: -- [~andrew.purt...@gmail.com] I like the direction you suggest; would proof out two concepts in the one go (osgi and moving cp hosting out of the regionserver). > Run HBase server in an OSGi container > - > > Key: HBASE-8607 > URL: https://issues.apache.org/jira/browse/HBASE-8607 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: James Taylor > > Run the HBase server in an OSGi container to support updating custom filters > and coprocessor updates without requiring a region server reboot. Typically, > applications that use coprocessors and custom filters also have shared > classes underneath, so putting the burden on the user to include some kind of > version name in the class is not adequate. Including the version name in the > package might work in some cases (at least until dependent jars start to > change as well), but is cumbersome and overburdens the app developer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8656) Rpc call may not be notified in SecureClient
[
https://issues.apache.org/jira/browse/HBASE-8656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13672800#comment-13672800
]
cuijianwei commented on HBASE-8656:
---
[~saint@gmail.com], in trunk/0.95, SecureClient may be replaced by
HBaseSaslRpcClient. The code changes a lot, I'am not sure whether
HBaseSaslRpcClient has the same problem.
> Rpc call may not be notified in SecureClient
>
>
> Key: HBASE-8656
> URL: https://issues.apache.org/jira/browse/HBASE-8656
> Project: HBase
> Issue Type: Bug
> Components: Client, IPC/RPC, security
>Affects Versions: 0.94.7
>Reporter: cuijianwei
>Priority: Critical
> Fix For: 0.94.9
>
> Attachments: HBASE-8656-0.94-v1.txt
>
>
> In SecureClient.java, rpc responses will be processed by receiveResponse()
> which looks like:
> {code}
> try {
> int id = in.readInt();// try to read an id
> if (LOG.isDebugEnabled())
> LOG.debug(getName() + " got value #" + id);
> Call call = calls.remove(id);
> int state = in.readInt(); // read call status
> if (LOG.isDebugEnabled()) {
> LOG.debug("call #"+id+" state is " + state);
> }
> if (state == Status.SUCCESS.state) {
> Writable value = ReflectionUtils.newInstance(valueClass, conf);
> value.readFields(in); // read value
> if (LOG.isDebugEnabled()) {
> LOG.debug("call #"+id+", response is:\n"+value.toString());
> }
> // it's possible that this call may have been cleaned up due to a
> RPC
> // timeout, so check if it still exists before setting the value.
> if (call != null) {
> call.setValue(value);
> }
> } else if (state == Status.ERROR.state) {
> if (call != null) {
> call.setException(new
> RemoteException(WritableUtils.readString(in), WritableUtils
> .readString(in)));
> }
> } else if (state == Status.FATAL.state) {
> // Close the connection
> markClosed(new RemoteException(WritableUtils.readString(in),
> WritableUtils.readString(in)));
> }
> } catch (IOException e) {
> if (e instanceof SocketTimeoutException && remoteId.rpcTimeout > 0) {
> // Clean up open calls but don't treat this as a fatal condition,
> // since we expect certain responses to not make it by the specified
> // {@link ConnectionId#rpcTimeout}.
> closeException = e;
> } else {
> // Since the server did not respond within the default ping interval
> // time, treat this as a fatal condition and close this connection
> markClosed(e);
> }
> } finally {
> if (remoteId.rpcTimeout > 0) {
> cleanupCalls(remoteId.rpcTimeout);
> }
> }
> }
> {code}
> In above code, in the try block, the call will be firstly removed from call
> map by:
> {code}
> Call call = calls.remove(id);
> {code}
> There may be two cases leading the call couldn't be notified and the invoking
> thread will wait forever.
> Firstly, if the returned status is Status.FATAL.state by:
> {code}
> int state = in.readInt(); // read call status
> {code}
> The code will come into:
> {code}
> } else if (state == Status.FATAL.state) {
> // Close the connection
> markClosed(new RemoteException(WritableUtils.readString(in),
> WritableUtils.readString(in)));
> }
> {code}
> Here, the SecureConnection is marked as closed and all rpc calls in call map
> of this connection will be notified to receive an exception. However, the
> current rpc call has been removed from the call map, it won't be notified.
> Secondly, after the call has been removed by:
> {code}
> Call call = calls.remove(id);
> {code}
> If we encounter any exception before the 'try' block finished, the code will
> come into 'catch' and 'finally' block, neither 'catch' block nor 'finally'
> block will notify the rpc call because it has been removed from call map.
> Compared with receiveResponse() in HBaseClient.java, it may be better to get
> the rpc call from call map and remove it at the end of the 'try' block.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8640) ServerName in master may not initialize with the configured ipc address of hbase.master.ipc.address
[
https://issues.apache.org/jira/browse/HBASE-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-8640:
-
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to trunk, 0.95 and to 0.94 (Pardon me if I presumed too much
[~lhofhansl])
> ServerName in master may not initialize with the configured ipc address of
> hbase.master.ipc.address
> ---
>
> Key: HBASE-8640
> URL: https://issues.apache.org/jira/browse/HBASE-8640
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: rajeshbabu
>Assignee: rajeshbabu
> Fix For: 0.98.0, 0.95.2, 0.94.9
>
> Attachments: HBASE-8640_94.patch, HBASE-8640.patch
>
>
> We are starting rpc server with default interface hostname or configured ipc
> address
> {code}
> this.rpcServer = HBaseRPC.getServer(this,
> new Class[]{HMasterInterface.class, HMasterRegionInterface.class},
> initialIsa.getHostName(), // This is bindAddress if set else it's
> hostname
> initialIsa.getPort(),
> numHandlers,
> 0, // we dont use high priority handlers in master
> conf.getBoolean("hbase.rpc.verbose", false), conf,
> 0); // this is a DNC w/o high priority handlers
> {code}
> But we are initialzing servername with default hostname always master znode
> also have this hostname.
> {code}
> String hostname = Strings.domainNamePointerToHostName(DNS.getDefaultHost(
> conf.get("hbase.master.dns.interface", "default"),
> conf.get("hbase.master.dns.nameserver", "default")));
> ...
> this.serverName = new ServerName(hostname,
> this.isa.getPort(), System.currentTimeMillis());
> {code}
> If both default interface hostname and configured ipc address are not same
> clients will get MasterNotRunningException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8656) Rpc call may not be notified in SecureClient
[
https://issues.apache.org/jira/browse/HBASE-8656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13672794#comment-13672794
]
stack commented on HBASE-8656:
--
We do not have this issue in trunk/0.95?
> Rpc call may not be notified in SecureClient
>
>
> Key: HBASE-8656
> URL: https://issues.apache.org/jira/browse/HBASE-8656
> Project: HBase
> Issue Type: Bug
> Components: Client, IPC/RPC, security
>Affects Versions: 0.94.7
>Reporter: cuijianwei
>Priority: Critical
> Fix For: 0.94.9
>
> Attachments: HBASE-8656-0.94-v1.txt
>
>
> In SecureClient.java, rpc responses will be processed by receiveResponse()
> which looks like:
> {code}
> try {
> int id = in.readInt();// try to read an id
> if (LOG.isDebugEnabled())
> LOG.debug(getName() + " got value #" + id);
> Call call = calls.remove(id);
> int state = in.readInt(); // read call status
> if (LOG.isDebugEnabled()) {
> LOG.debug("call #"+id+" state is " + state);
> }
> if (state == Status.SUCCESS.state) {
> Writable value = ReflectionUtils.newInstance(valueClass, conf);
> value.readFields(in); // read value
> if (LOG.isDebugEnabled()) {
> LOG.debug("call #"+id+", response is:\n"+value.toString());
> }
> // it's possible that this call may have been cleaned up due to a
> RPC
> // timeout, so check if it still exists before setting the value.
> if (call != null) {
> call.setValue(value);
> }
> } else if (state == Status.ERROR.state) {
> if (call != null) {
> call.setException(new
> RemoteException(WritableUtils.readString(in), WritableUtils
> .readString(in)));
> }
> } else if (state == Status.FATAL.state) {
> // Close the connection
> markClosed(new RemoteException(WritableUtils.readString(in),
> WritableUtils.readString(in)));
> }
> } catch (IOException e) {
> if (e instanceof SocketTimeoutException && remoteId.rpcTimeout > 0) {
> // Clean up open calls but don't treat this as a fatal condition,
> // since we expect certain responses to not make it by the specified
> // {@link ConnectionId#rpcTimeout}.
> closeException = e;
> } else {
> // Since the server did not respond within the default ping interval
> // time, treat this as a fatal condition and close this connection
> markClosed(e);
> }
> } finally {
> if (remoteId.rpcTimeout > 0) {
> cleanupCalls(remoteId.rpcTimeout);
> }
> }
> }
> {code}
> In above code, in the try block, the call will be firstly removed from call
> map by:
> {code}
> Call call = calls.remove(id);
> {code}
> There may be two cases leading the call couldn't be notified and the invoking
> thread will wait forever.
> Firstly, if the returned status is Status.FATAL.state by:
> {code}
> int state = in.readInt(); // read call status
> {code}
> The code will come into:
> {code}
> } else if (state == Status.FATAL.state) {
> // Close the connection
> markClosed(new RemoteException(WritableUtils.readString(in),
> WritableUtils.readString(in)));
> }
> {code}
> Here, the SecureConnection is marked as closed and all rpc calls in call map
> of this connection will be notified to receive an exception. However, the
> current rpc call has been removed from the call map, it won't be notified.
> Secondly, after the call has been removed by:
> {code}
> Call call = calls.remove(id);
> {code}
> If we encounter any exception before the 'try' block finished, the code will
> come into 'catch' and 'finally' block, neither 'catch' block nor 'finally'
> block will notify the rpc call because it has been removed from call map.
> Compared with receiveResponse() in HBaseClient.java, it may be better to get
> the rpc call from call map and remove it at the end of the 'try' block.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8674) JUnit and Surefire TRUNK-HBASE-2 plugins need a new home
[
https://issues.apache.org/jira/browse/HBASE-8674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13672791#comment-13672791
]
stack commented on HBASE-8674:
--
The stack repository was for a zookeeper and hadoop 1.0.0 RCs. It can be
removed.
[~nkeywal] Can we let go of the surefire or junit custom dependency? IIRC, we
can move to official release of one of these artifacts only?
That leaves us wanting a home for whatever is left. I could host it on
personal space (Our Dave Wang is asking if it can be hosted by Cloudera).
> JUnit and Surefire TRUNK-HBASE-2 plugins need a new home
>
>
> Key: HBASE-8674
> URL: https://issues.apache.org/jira/browse/HBASE-8674
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 0.98.0, 0.94.8, 0.95.1
>Reporter: Andrew Purtell
>Priority: Blocker
>
> people.apache.org cannot currently host personal or transient Maven repos.
> {noformat}
> $ curl --connect-timeout 60 -v
> http://people.apache.org/~garyh/mvn/org/apache/maven/plugins/maven-remote-resources-plugin/1.4/maven-remote-resources-plugin-1.4.pom
> * About to connect() to people.apache.org port 80 (#0)
> * Trying 140.211.11.9...
> * Connection timed out after 60064 milliseconds
> * Closing connection 0
> curl: (28) Connection timed out after 60064 milliseconds
> {noformat}
> All builds are at the moment broken if the HBase custom junit or surefire
> jars are not already in cache. Even if this is a temporary condition, we
> should find a new home for these artifacts, upgrade to versions that include
> our submitted changes (if any), or fall back to release versions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4811) Support reverse Scan
[ https://issues.apache.org/jira/browse/HBASE-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chunhui shen updated HBASE-4811: Attachment: hbase-4811-trunkv4.patch Addressing Ted's review comments > Support reverse Scan > > > Key: HBASE-4811 > URL: https://issues.apache.org/jira/browse/HBASE-4811 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.20.6, 0.94.7 >Reporter: John Carrino >Assignee: Liang Xie > Attachments: HBase-4811-0.94.3modified.txt, HBase-4811-0.94-v2.txt, > hbase-4811-trunkv1.patch, hbase-4811-trunkv4.patch > > > All the documentation I find about HBase says that if you want forward and > reverse scans you should just build 2 tables and one be ascending and one > descending. Is there a fundamental reason that HBase only supports forward > Scan? It seems like a lot of extra space overhead and coding overhead (to > keep them in sync) to support 2 tables. > I am assuming this has been discussed before, but I can't find the > discussions anywhere about it or why it would be infeasible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8642) [Snapshot] List and delete snapshot by table
[ https://issues.apache.org/jira/browse/HBASE-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Zhou updated HBASE-8642: --- Attachment: 8642-trunk-0.95-v2.patch Updated with new ruby module name. Could we go ahead with this? > [Snapshot] List and delete snapshot by table > > > Key: HBASE-8642 > URL: https://issues.apache.org/jira/browse/HBASE-8642 > Project: HBase > Issue Type: Improvement > Components: snapshots >Affects Versions: 0.98.0, 0.95.0, 0.95.1, 0.95.2 >Reporter: Julian Zhou >Assignee: Julian Zhou >Priority: Minor > Fix For: 0.98.0, 0.95.0, 0.95.1, 0.95.2 > > Attachments: 8642-trunk-0.95-v0.patch, 8642-trunk-0.95-v1.patch, > 8642-trunk-0.95-v2.patch > > > Support list and delete snapshot by table name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8642) [Snapshot] List and delete snapshot by table
[
https://issues.apache.org/jira/browse/HBASE-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13672778#comment-13672778
]
Matteo Bertozzi commented on HBASE-8642:
{quote}is that to seperate data and meta to enable possible file sharing among
tables. This could be a new catalog system for hbase.{quote}
Yeah that's the idea... it not just file sharing between table/snapshots is
also the ability to switch to managing blocks directly and avoid to rewrite
everything during compactions.
{quote}One byte (256 items could be attached) for each node anchor under .data
tree.{quote}
Not sure what you mean with this... The file placement using the first bytes of
the hash/name is just to avoid the problem of too many files in a directory
that old filesystems have, and a different way to store stuff removing the
knowledge of table/region/cf that today is hardcoded. This doesn't really have
effect on all the capabilities that you'll gain from the richer metadata system.
(This stuff is starting being off topic here, since is not related 100% with
this jira, if you've other question/suggestions/... just write it on HBASE-7806)
> [Snapshot] List and delete snapshot by table
>
>
> Key: HBASE-8642
> URL: https://issues.apache.org/jira/browse/HBASE-8642
> Project: HBase
> Issue Type: Improvement
> Components: snapshots
>Affects Versions: 0.98.0, 0.95.0, 0.95.1, 0.95.2
>Reporter: Julian Zhou
>Assignee: Julian Zhou
>Priority: Minor
> Fix For: 0.98.0, 0.95.0, 0.95.1, 0.95.2
>
> Attachments: 8642-trunk-0.95-v0.patch, 8642-trunk-0.95-v1.patch
>
>
> Support list and delete snapshot by table name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8642) [Snapshot] List and delete snapshot by table
[ https://issues.apache.org/jira/browse/HBASE-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13672774#comment-13672774 ] Julian Zhou commented on HBASE-8642: Hi, [~mbertozzi], I see that last slide of HBASE-7806 for future, is that to seperate .data and .meta to enable possible file sharing among tables. One byte (256 items could be attached) for each node anchor under .data tree. This could be a new catalog system for hbase. > [Snapshot] List and delete snapshot by table > > > Key: HBASE-8642 > URL: https://issues.apache.org/jira/browse/HBASE-8642 > Project: HBase > Issue Type: Improvement > Components: snapshots >Affects Versions: 0.98.0, 0.95.0, 0.95.1, 0.95.2 >Reporter: Julian Zhou >Assignee: Julian Zhou >Priority: Minor > Fix For: 0.98.0, 0.95.0, 0.95.1, 0.95.2 > > Attachments: 8642-trunk-0.95-v0.patch, 8642-trunk-0.95-v1.patch > > > Support list and delete snapshot by table name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8001) Avoid unnecessary lazy seek
[ https://issues.apache.org/jira/browse/HBASE-8001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13672771#comment-13672771 ] Raymond Liu commented on HBASE-8001: [~lhofhansl], [~ted_yu]: sorry for late on this issue. busy for other staff. I do this test again in with single RS. I still use M/R job, while the table only have one region. and is 2M rows ,1CF, 18col, without any compression or encoding. size about 3G on disk. And I don't use blockcache,every time the data is read from disk by a real seek. but as we discussed before, use of blockcache will only led to more gain with this patch. with this patch, a 18col full table scan cost 99-101s, while without this patch it will cost 108-109s. still noticeable difference. I test it for several times on each case. the result is pretty stable. Do you mind to take a end2end test? I am not sure is there any other thing might still have impact upon your test case. might be that the data size is too small? > Avoid unnecessary lazy seek > --- > > Key: HBASE-8001 > URL: https://issues.apache.org/jira/browse/HBASE-8001 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 0.94.5 >Reporter: Raymond Liu >Assignee: Raymond Liu > Fix For: 0.98.0 > > Attachments: HBASE-8001_onescanner.patch, > HBASE-8001_onescanner_v2.patch > > > Lazy seek helps to reduce the real seek needed for multi hfile, when the kv > from newer hfile is enough to satisfy the query. > While in many case, it just push the real seek later, and do not reduce the > number of real seek. e.g. there are only one hfile, or storefilescanner is > closed and only one left, or the scan need to go through all the versions, or > there are only one version of row and a sequence scan is performed. In these > case, lazy seek just bring extra overhead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8590) [0.94] BlockingMetaScannerVisitor should check for parent meta entry while waiting for split daughter
[ https://issues.apache.org/jira/browse/HBASE-8590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13672740#comment-13672740 ] Lars Hofhansl commented on HBASE-8590: -- Patch looks reasonable to me. Wanna commit [~enis]? > [0.94] BlockingMetaScannerVisitor should check for parent meta entry while > waiting for split daughter > - > > Key: HBASE-8590 > URL: https://issues.apache.org/jira/browse/HBASE-8590 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.94.9 > > Attachments: hbase-8590_v1.patch > > > This was discovered after HBASE-8505 went in, which introduces a test > sporadically triggering this bug. > From comments at HBASE-8505: > From the logs at > https://builds.apache.org/job/HBase-0.94-security/ws/trunk/target/surefire-reports/org.apache.hadoop.hbase.client.TestMetaScanner-output.txt, > I think I understand what is going on: > BlockingMetaScannerVisitor blocks and wait for the split daughter to appear > when it sees a parent region (HBASE-5986). CatalogJanitor on the other hand > will order the regions in a (kind-of) topological sort (based on parent child > relation) so that it will guarantee parents are not GC'd before daughters. > What is happening in this issue is not related to the patch in this jira, but > the test triggers this extremely rare case by running concurrent > catalogjanitor, splits and metascanners. We have parent, splita and splitb > regions, and catalogjanitor decides to delete parent first and splitb in one > run. While there is a concurrent metascanner which will go over the parent, > and sees that it is split, but before being able to read the split daughter, > catalog janitor will delete both the parent and the child, which will lead to > metascanner blocking until timeout and failing the test. > On solution might be to also check whether the parent is still there in > BlockingMetaScannerVisitor while we are blocking for the daughter. > Good thing is that with HBASE-7721, we don't need any of this in trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7055) port HBASE-6371 tier-based compaction from 0.89-fb to trunk (with changes)
[ https://issues.apache.org/jira/browse/HBASE-7055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13672738#comment-13672738 ] Lars Hofhansl commented on HBASE-7055: -- I think we should start some collaboration on this. At Salesforce we're working on a feature to separate old and new data for scenarios where we have to store very many versions for the odd historical query, but most queries are for the latest data. > port HBASE-6371 tier-based compaction from 0.89-fb to trunk (with changes) > -- > > Key: HBASE-7055 > URL: https://issues.apache.org/jira/browse/HBASE-7055 > Project: HBase > Issue Type: Task > Components: Compaction >Affects Versions: 0.95.2 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 0.95.1 > > Attachments: HBASE-6371-squashed.patch, HBASE-6371-v2-squashed.patch, > HBASE-6371-v3-refactor-only-squashed.patch, > HBASE-6371-v4-refactor-only-squashed.patch, > HBASE-6371-v5-refactor-only-squashed.patch, HBASE-7055-v0.patch, > HBASE-7055-v1.patch, HBASE-7055-v2.patch, HBASE-7055-v3.patch, > HBASE-7055-v4.patch, HBASE-7055-v5.patch, HBASE-7055-v6.patch, > HBASE-7055-v7.patch, HBASE-7055-v7.patch > > > See HBASE-6371 for details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8626) RowMutations fail when Delete and Put on same columnFamily/column/row
[ https://issues.apache.org/jira/browse/HBASE-8626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13672737#comment-13672737 ] Lars Hofhansl commented on HBASE-8626: -- You can always have the races when multiple clients issue requests at the same time. Two clients could issue a request within the same ms (or in the case of the patch within 1 ms of each other) and the result would still be not what you want - basically there *is* no correct order if things happen "at the same time". The race Ted outline is not worth than the one you get with the patch (well the chance might be slightly reduce since writes to the same rows are serialized, so the chance is lessened that the region server does them in the same ms, but it can still happen). > RowMutations fail when Delete and Put on same columnFamily/column/row > - > > Key: HBASE-8626 > URL: https://issues.apache.org/jira/browse/HBASE-8626 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.7, 0.95.0 > Environment: Ubuntu 12.04, HBase 0.94.7 >Reporter: Vinod >Assignee: Ted Yu > Attachments: 8626-v1.txt, TestRowMutations.java, > tests_for_row_mutations1.patch > > > When RowMutations have a Delete followed by Put to same column family or > columns or rows, only the Delete is happening while the Put is ignored so > atomicity of RowMutations is broken for such cases. > Attached is a unit test where the following tests are failing: > - testDeleteCFThenPutInSameCF: Delete a column family and then Put to same > column family. > - testDeleteColumnThenPutSameColumn: Delete a column and then Put to same > column. > - testDeleteRowThenPutSameRow: Delete a row and then Put to same row -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7055) port HBASE-6371 tier-based compaction from 0.89-fb to trunk (with changes)
[ https://issues.apache.org/jira/browse/HBASE-7055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13672713#comment-13672713 ] Sergey Shelukhin commented on HBASE-7055: - My take on this is that there appears to be at least few scenarios that sound viable. E.g. old data is not as important and read rarely, so we don't care if it stays uncompacted and assume blooms will help us ignore these files for the keys that are actually being read; or new data is in cache so again there's no reason to compact the latest files. Or both. However, testing if these actually work require setting up a complicated test scenario with specific read patterns, over large amounts of data, or a very good simulation tool that would estimate read impact (cache usage, how many files touched, what is going on in parallel); the will for either of these is presently lacking. On the other hand, patch is extremely low impact and will be off by default. So yeah maybe we can keep it for 0.96.X. > port HBASE-6371 tier-based compaction from 0.89-fb to trunk (with changes) > -- > > Key: HBASE-7055 > URL: https://issues.apache.org/jira/browse/HBASE-7055 > Project: HBase > Issue Type: Task > Components: Compaction >Affects Versions: 0.95.2 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 0.95.1 > > Attachments: HBASE-6371-squashed.patch, HBASE-6371-v2-squashed.patch, > HBASE-6371-v3-refactor-only-squashed.patch, > HBASE-6371-v4-refactor-only-squashed.patch, > HBASE-6371-v5-refactor-only-squashed.patch, HBASE-7055-v0.patch, > HBASE-7055-v1.patch, HBASE-7055-v2.patch, HBASE-7055-v3.patch, > HBASE-7055-v4.patch, HBASE-7055-v5.patch, HBASE-7055-v6.patch, > HBASE-7055-v7.patch, HBASE-7055-v7.patch > > > See HBASE-6371 for details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8626) RowMutations fail when Delete and Put on same columnFamily/column/row
[ https://issues.apache.org/jira/browse/HBASE-8626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13672702#comment-13672702 ] Vinod commented on HBASE-8626: -- Lars, the approach you suggest does not work for my use-case because of the race-condition mentioned by Ted in his comment above at: https://issues.apache.org/jira/browse/HBASE-8626?focusedCommentId=13669464&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13669464 Can we extend current RowMutations to support something like what Andrew stated in his previous comment here: https://issues.apache.org/jira/browse/HBASE-8626?focusedCommentId=13669152&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13669152 That would solve my use-case perfectly. > RowMutations fail when Delete and Put on same columnFamily/column/row > - > > Key: HBASE-8626 > URL: https://issues.apache.org/jira/browse/HBASE-8626 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.7, 0.95.0 > Environment: Ubuntu 12.04, HBase 0.94.7 >Reporter: Vinod >Assignee: Ted Yu > Attachments: 8626-v1.txt, TestRowMutations.java, > tests_for_row_mutations1.patch > > > When RowMutations have a Delete followed by Put to same column family or > columns or rows, only the Delete is happening while the Put is ignored so > atomicity of RowMutations is broken for such cases. > Attached is a unit test where the following tests are failing: > - testDeleteCFThenPutInSameCF: Delete a column family and then Put to same > column family. > - testDeleteColumnThenPutSameColumn: Delete a column and then Put to same > column. > - testDeleteRowThenPutSameRow: Delete a row and then Put to same row -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8656) Rpc call may not be notified in SecureClient
[
https://issues.apache.org/jira/browse/HBASE-8656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-8656:
-
Priority: Critical (was: Major)
> Rpc call may not be notified in SecureClient
>
>
> Key: HBASE-8656
> URL: https://issues.apache.org/jira/browse/HBASE-8656
> Project: HBase
> Issue Type: Bug
> Components: Client, IPC/RPC, security
>Affects Versions: 0.94.7
>Reporter: cuijianwei
>Priority: Critical
> Fix For: 0.94.9
>
> Attachments: HBASE-8656-0.94-v1.txt
>
>
> In SecureClient.java, rpc responses will be processed by receiveResponse()
> which looks like:
> {code}
> try {
> int id = in.readInt();// try to read an id
> if (LOG.isDebugEnabled())
> LOG.debug(getName() + " got value #" + id);
> Call call = calls.remove(id);
> int state = in.readInt(); // read call status
> if (LOG.isDebugEnabled()) {
> LOG.debug("call #"+id+" state is " + state);
> }
> if (state == Status.SUCCESS.state) {
> Writable value = ReflectionUtils.newInstance(valueClass, conf);
> value.readFields(in); // read value
> if (LOG.isDebugEnabled()) {
> LOG.debug("call #"+id+", response is:\n"+value.toString());
> }
> // it's possible that this call may have been cleaned up due to a
> RPC
> // timeout, so check if it still exists before setting the value.
> if (call != null) {
> call.setValue(value);
> }
> } else if (state == Status.ERROR.state) {
> if (call != null) {
> call.setException(new
> RemoteException(WritableUtils.readString(in), WritableUtils
> .readString(in)));
> }
> } else if (state == Status.FATAL.state) {
> // Close the connection
> markClosed(new RemoteException(WritableUtils.readString(in),
> WritableUtils.readString(in)));
> }
> } catch (IOException e) {
> if (e instanceof SocketTimeoutException && remoteId.rpcTimeout > 0) {
> // Clean up open calls but don't treat this as a fatal condition,
> // since we expect certain responses to not make it by the specified
> // {@link ConnectionId#rpcTimeout}.
> closeException = e;
> } else {
> // Since the server did not respond within the default ping interval
> // time, treat this as a fatal condition and close this connection
> markClosed(e);
> }
> } finally {
> if (remoteId.rpcTimeout > 0) {
> cleanupCalls(remoteId.rpcTimeout);
> }
> }
> }
> {code}
> In above code, in the try block, the call will be firstly removed from call
> map by:
> {code}
> Call call = calls.remove(id);
> {code}
> There may be two cases leading the call couldn't be notified and the invoking
> thread will wait forever.
> Firstly, if the returned status is Status.FATAL.state by:
> {code}
> int state = in.readInt(); // read call status
> {code}
> The code will come into:
> {code}
> } else if (state == Status.FATAL.state) {
> // Close the connection
> markClosed(new RemoteException(WritableUtils.readString(in),
> WritableUtils.readString(in)));
> }
> {code}
> Here, the SecureConnection is marked as closed and all rpc calls in call map
> of this connection will be notified to receive an exception. However, the
> current rpc call has been removed from the call map, it won't be notified.
> Secondly, after the call has been removed by:
> {code}
> Call call = calls.remove(id);
> {code}
> If we encounter any exception before the 'try' block finished, the code will
> come into 'catch' and 'finally' block, neither 'catch' block nor 'finally'
> block will notify the rpc call because it has been removed from call map.
> Compared with receiveResponse() in HBaseClient.java, it may be better to get
> the rpc call from call map and remove it at the end of the 'try' block.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8656) Rpc call may not be notified in SecureClient
[
https://issues.apache.org/jira/browse/HBASE-8656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-8656:
-
Fix Version/s: 0.94.9
> Rpc call may not be notified in SecureClient
>
>
> Key: HBASE-8656
> URL: https://issues.apache.org/jira/browse/HBASE-8656
> Project: HBase
> Issue Type: Bug
> Components: Client, IPC/RPC, security
>Affects Versions: 0.94.7
>Reporter: cuijianwei
> Fix For: 0.94.9
>
> Attachments: HBASE-8656-0.94-v1.txt
>
>
> In SecureClient.java, rpc responses will be processed by receiveResponse()
> which looks like:
> {code}
> try {
> int id = in.readInt();// try to read an id
> if (LOG.isDebugEnabled())
> LOG.debug(getName() + " got value #" + id);
> Call call = calls.remove(id);
> int state = in.readInt(); // read call status
> if (LOG.isDebugEnabled()) {
> LOG.debug("call #"+id+" state is " + state);
> }
> if (state == Status.SUCCESS.state) {
> Writable value = ReflectionUtils.newInstance(valueClass, conf);
> value.readFields(in); // read value
> if (LOG.isDebugEnabled()) {
> LOG.debug("call #"+id+", response is:\n"+value.toString());
> }
> // it's possible that this call may have been cleaned up due to a
> RPC
> // timeout, so check if it still exists before setting the value.
> if (call != null) {
> call.setValue(value);
> }
> } else if (state == Status.ERROR.state) {
> if (call != null) {
> call.setException(new
> RemoteException(WritableUtils.readString(in), WritableUtils
> .readString(in)));
> }
> } else if (state == Status.FATAL.state) {
> // Close the connection
> markClosed(new RemoteException(WritableUtils.readString(in),
> WritableUtils.readString(in)));
> }
> } catch (IOException e) {
> if (e instanceof SocketTimeoutException && remoteId.rpcTimeout > 0) {
> // Clean up open calls but don't treat this as a fatal condition,
> // since we expect certain responses to not make it by the specified
> // {@link ConnectionId#rpcTimeout}.
> closeException = e;
> } else {
> // Since the server did not respond within the default ping interval
> // time, treat this as a fatal condition and close this connection
> markClosed(e);
> }
> } finally {
> if (remoteId.rpcTimeout > 0) {
> cleanupCalls(remoteId.rpcTimeout);
> }
> }
> }
> {code}
> In above code, in the try block, the call will be firstly removed from call
> map by:
> {code}
> Call call = calls.remove(id);
> {code}
> There may be two cases leading the call couldn't be notified and the invoking
> thread will wait forever.
> Firstly, if the returned status is Status.FATAL.state by:
> {code}
> int state = in.readInt(); // read call status
> {code}
> The code will come into:
> {code}
> } else if (state == Status.FATAL.state) {
> // Close the connection
> markClosed(new RemoteException(WritableUtils.readString(in),
> WritableUtils.readString(in)));
> }
> {code}
> Here, the SecureConnection is marked as closed and all rpc calls in call map
> of this connection will be notified to receive an exception. However, the
> current rpc call has been removed from the call map, it won't be notified.
> Secondly, after the call has been removed by:
> {code}
> Call call = calls.remove(id);
> {code}
> If we encounter any exception before the 'try' block finished, the code will
> come into 'catch' and 'finally' block, neither 'catch' block nor 'finally'
> block will notify the rpc call because it has been removed from call map.
> Compared with receiveResponse() in HBaseClient.java, it may be better to get
> the rpc call from call map and remove it at the end of the 'try' block.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8640) ServerName in master may not initialize with the configured ipc address of hbase.master.ipc.address
[
https://issues.apache.org/jira/browse/HBASE-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13672632#comment-13672632
]
Hadoop QA commented on HBASE-8640:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585773/HBASE-8640_94.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5906//console
This message is automatically generated.
> ServerName in master may not initialize with the configured ipc address of
> hbase.master.ipc.address
> ---
>
> Key: HBASE-8640
> URL: https://issues.apache.org/jira/browse/HBASE-8640
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: rajeshbabu
>Assignee: rajeshbabu
> Fix For: 0.98.0, 0.95.2, 0.94.9
>
> Attachments: HBASE-8640_94.patch, HBASE-8640.patch
>
>
> We are starting rpc server with default interface hostname or configured ipc
> address
> {code}
> this.rpcServer = HBaseRPC.getServer(this,
> new Class[]{HMasterInterface.class, HMasterRegionInterface.class},
> initialIsa.getHostName(), // This is bindAddress if set else it's
> hostname
> initialIsa.getPort(),
> numHandlers,
> 0, // we dont use high priority handlers in master
> conf.getBoolean("hbase.rpc.verbose", false), conf,
> 0); // this is a DNC w/o high priority handlers
> {code}
> But we are initialzing servername with default hostname always master znode
> also have this hostname.
> {code}
> String hostname = Strings.domainNamePointerToHostName(DNS.getDefaultHost(
> conf.get("hbase.master.dns.interface", "default"),
> conf.get("hbase.master.dns.nameserver", "default")));
> ...
> this.serverName = new ServerName(hostname,
> this.isa.getPort(), System.currentTimeMillis());
> {code}
> If both default interface hostname and configured ipc address are not same
> clients will get MasterNotRunningException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8674) JUnit and Surefire TRUNK-HBASE-2 plugins need a new home
[
https://issues.apache.org/jira/browse/HBASE-8674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13672628#comment-13672628
]
Andrew Purtell commented on HBASE-8674:
---
In 0.94 builds, we also have http://people.apache.org/~stack/m2/repository
> JUnit and Surefire TRUNK-HBASE-2 plugins need a new home
>
>
> Key: HBASE-8674
> URL: https://issues.apache.org/jira/browse/HBASE-8674
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 0.98.0, 0.94.8, 0.95.1
>Reporter: Andrew Purtell
>Priority: Blocker
>
> people.apache.org cannot currently host personal or transient Maven repos.
> {noformat}
> $ curl --connect-timeout 60 -v
> http://people.apache.org/~garyh/mvn/org/apache/maven/plugins/maven-remote-resources-plugin/1.4/maven-remote-resources-plugin-1.4.pom
> * About to connect() to people.apache.org port 80 (#0)
> * Trying 140.211.11.9...
> * Connection timed out after 60064 milliseconds
> * Closing connection 0
> curl: (28) Connection timed out after 60064 milliseconds
> {noformat}
> All builds are at the moment broken if the HBase custom junit or surefire
> jars are not already in cache. Even if this is a temporary condition, we
> should find a new home for these artifacts, upgrade to versions that include
> our submitted changes (if any), or fall back to release versions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8640) ServerName in master may not initialize with the configured ipc address of hbase.master.ipc.address
[
https://issues.apache.org/jira/browse/HBASE-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
rajeshbabu updated HBASE-8640:
--
Attachment: HBASE-8640_94.patch
Patch for 94
> ServerName in master may not initialize with the configured ipc address of
> hbase.master.ipc.address
> ---
>
> Key: HBASE-8640
> URL: https://issues.apache.org/jira/browse/HBASE-8640
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: rajeshbabu
>Assignee: rajeshbabu
> Fix For: 0.98.0, 0.95.2, 0.94.9
>
> Attachments: HBASE-8640_94.patch, HBASE-8640.patch
>
>
> We are starting rpc server with default interface hostname or configured ipc
> address
> {code}
> this.rpcServer = HBaseRPC.getServer(this,
> new Class[]{HMasterInterface.class, HMasterRegionInterface.class},
> initialIsa.getHostName(), // This is bindAddress if set else it's
> hostname
> initialIsa.getPort(),
> numHandlers,
> 0, // we dont use high priority handlers in master
> conf.getBoolean("hbase.rpc.verbose", false), conf,
> 0); // this is a DNC w/o high priority handlers
> {code}
> But we are initialzing servername with default hostname always master znode
> also have this hostname.
> {code}
> String hostname = Strings.domainNamePointerToHostName(DNS.getDefaultHost(
> conf.get("hbase.master.dns.interface", "default"),
> conf.get("hbase.master.dns.nameserver", "default")));
> ...
> this.serverName = new ServerName(hostname,
> this.isa.getPort(), System.currentTimeMillis());
> {code}
> If both default interface hostname and configured ipc address are not same
> clients will get MasterNotRunningException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8674) JUnit and Surefire TRUNK-HBASE-2 plugins need a new home
[
https://issues.apache.org/jira/browse/HBASE-8674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-8674:
--
Description:
people.apache.org cannot currently host personal or transient Maven repos.
{noformat}
$ curl --connect-timeout 60 -v
http://people.apache.org/~garyh/mvn/org/apache/maven/plugins/maven-remote-resources-plugin/1.4/maven-remote-resources-plugin-1.4.pom
* About to connect() to people.apache.org port 80 (#0)
* Trying 140.211.11.9...
* Connection timed out after 60064 milliseconds
* Closing connection 0
curl: (28) Connection timed out after 60064 milliseconds
{noformat}
All builds are at the moment broken if the HBase custom junit or surefire jars
are not already in cache. Even if this is a temporary condition, we should find
a new home for these artifacts, upgrade to versions that include our submitted
changes (if any), or fall back to release versions.
was:
people.apache.org can no longer host personal or transient Maven repos.
{noformat}
$ curl --connect-timeout 60 -v
http://people.apache.org/~garyh/mvn/org/apache/maven/plugins/maven-remote-resources-plugin/1.4/maven-remote-resources-plugin-1.4.pom
* About to connect() to people.apache.org port 80 (#0)
* Trying 140.211.11.9...
* Connection timed out after 60064 milliseconds
* Closing connection 0
curl: (28) Connection timed out after 60064 milliseconds
{noformat}
All builds are broken if the HBase custom junit or surefire jars are not
already in cache.
> JUnit and Surefire TRUNK-HBASE-2 plugins need a new home
>
>
> Key: HBASE-8674
> URL: https://issues.apache.org/jira/browse/HBASE-8674
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 0.98.0, 0.94.8, 0.95.1
>Reporter: Andrew Purtell
>Priority: Blocker
>
> people.apache.org cannot currently host personal or transient Maven repos.
> {noformat}
> $ curl --connect-timeout 60 -v
> http://people.apache.org/~garyh/mvn/org/apache/maven/plugins/maven-remote-resources-plugin/1.4/maven-remote-resources-plugin-1.4.pom
> * About to connect() to people.apache.org port 80 (#0)
> * Trying 140.211.11.9...
> * Connection timed out after 60064 milliseconds
> * Closing connection 0
> curl: (28) Connection timed out after 60064 milliseconds
> {noformat}
> All builds are at the moment broken if the HBase custom junit or surefire
> jars are not already in cache. Even if this is a temporary condition, we
> should find a new home for these artifacts, upgrade to versions that include
> our submitted changes (if any), or fall back to release versions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8674) JUnit and Surefire TRUNK-HBASE-2 plugins need a new home
[
https://issues.apache.org/jira/browse/HBASE-8674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-8674:
--
Priority: Blocker (was: Critical)
> JUnit and Surefire TRUNK-HBASE-2 plugins need a new home
>
>
> Key: HBASE-8674
> URL: https://issues.apache.org/jira/browse/HBASE-8674
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 0.98.0, 0.94.8, 0.95.1
>Reporter: Andrew Purtell
>Priority: Blocker
>
> people.apache.org can no longer host personal or transient Maven repos.
> {noformat}
> $ curl --connect-timeout 60 -v
> http://people.apache.org/~garyh/mvn/org/apache/maven/plugins/maven-remote-resources-plugin/1.4/maven-remote-resources-plugin-1.4.pom
> * About to connect() to people.apache.org port 80 (#0)
> * Trying 140.211.11.9...
> * Connection timed out after 60064 milliseconds
> * Closing connection 0
> curl: (28) Connection timed out after 60064 milliseconds
> {noformat}
> All builds are broken if the HBase custom junit or surefire jars are not
> already in cache.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8674) JUnit and Surefire TRUNK-HBASE-2 plugins need a new home
Andrew Purtell created HBASE-8674:
-
Summary: JUnit and Surefire TRUNK-HBASE-2 plugins need a new home
Key: HBASE-8674
URL: https://issues.apache.org/jira/browse/HBASE-8674
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 0.98.0, 0.94.8, 0.95.1
Reporter: Andrew Purtell
Priority: Critical
people.apache.org can no longer host personal or transient Maven repos.
{noformat}
$ curl --connect-timeout 60 -v
http://people.apache.org/~garyh/mvn/org/apache/maven/plugins/maven-remote-resources-plugin/1.4/maven-remote-resources-plugin-1.4.pom
* About to connect() to people.apache.org port 80 (#0)
* Trying 140.211.11.9...
* Connection timed out after 60064 milliseconds
* Closing connection 0
curl: (28) Connection timed out after 60064 milliseconds
{noformat}
All builds are broken if the HBase custom junit or surefire jars are not
already in cache.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8645) Change ServerName so it uses hostname only, not FQDN hostnames
[
https://issues.apache.org/jira/browse/HBASE-8645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13672491#comment-13672491
]
stack commented on HBASE-8645:
--
The SeverName.toShortString drops the startcode and intentionally makes a
String that does not look like a ServerName#toString, rather it is a host+port
only host is missing domain. Here is sample of what logging looks like now:
{code}
2013-05-31 02:25:03,091 DEBUG [M:0;asf000:41822] zookeeper.ZKUtil(433):
master:41822-0x13ef867090c Set watcher on znode that does not yet exist,
/hbase/running
2013-05-31 02:25:03,115 WARN [M:0;asf000:41822] hbase.ZNodeClearer(57):
Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on
crash by start scripts (Longer MTTR!)
2013-05-31 02:25:03,115 INFO [M:0;asf000:41822]
master.ActiveMasterManager(170): Registered Active
Master=asf000.sp2.ygridcore.net,41822,1369967102774
2013-05-31 02:25:03,123 INFO [M:0;asf000:41822] master.SplitLogManager(201):
timeout=12, unassigned timeout=18
2013-05-31 02:25:03,124 INFO [M:0;asf000:41822] master.SplitLogManager(210):
distributedLogReplay = false
2013-05-31 02:25:03,127 INFO [M:0;asf000:41822] master.SplitLogManager(1082):
Found 0 orphan tasks and 0 rescan nodes
2013-05-31 02:25:03,182 DEBUG [M:0;asf000:41822] util.FSUtils(757): Created
cluster ID file at hdfs://localhost:52392/user/jenkins/hbase/hbase.id with ID:
df6e7248-191d-479e-86da-6bd2a844e00b
2013-05-31 02:25:03,189 INFO [pool-1-thread-1] regionserver.ShutdownHook(87):
Installed shutdown hook thread: Shutdownhook:RS:0;asf000:47701
2013-05-31 02:25:03,190 INFO [pool-1-thread-1] regionserver.ShutdownHook(87):
Installed shutdown hook thread: Shutdownhook:RS:1;asf000:48644
2013-05-31 02:25:03,192 INFO [pool-1-thread-1] regionserver.ShutdownHook(87):
Installed shutdown hook thread: Shutdownhook:RS:2;asf000:47305
2013-05-31 02:25:03,198 INFO [RS:1;asf000:48644]
zookeeper.RecoverableZooKeeper(119): Process identifier=regionserver:48644
connecting to ZooKeeper ensemble=localhost:54561
2013-05-31 02:25:03,201 INFO [RS:2;asf000:47305]
zookeeper.RecoverableZooKeeper(119): Process identifier=regionserver:47305
connecting to ZooKeeper ensemble=localhost:54561
2013-05-31 02:25:03,204 INFO [RS:0;asf000:47701]
zookeeper.RecoverableZooKeeper(119): Process identifier=regionserver:47701
connecting to ZooKeeper ensemble=localhost:54561
2013-05-31 02:25:03,207 DEBUG [RS:1;asf000:48644-EventThread]
zookeeper.ZooKeeperWatcher(307): regionserver:48644 Received ZooKeeper Event,
type=None, state=SyncConnected, path=null
2013-05-31 02:25:03,207 DEBUG [RS:2;asf000:47305-EventThread]
zookeeper.ZooKeeperWatcher(307): regionserver:47305 Received ZooKeeper Event,
type=None, state=SyncConnected, path=null
2013-05-31 02:25:03,209 DEBUG [RS:1;asf000:48644-EventThread]
zookeeper.ZooKeeperWatcher(384): regionserver:48644-0x13ef867090c0001 connected
2013-05-31 02:25:03,210 DEBUG [RS:2;asf000:47305-EventThread]
zookeeper.ZooKeeperWatcher(384): regionserver:47305-0x13ef867090c0002 connected
2013-05-31 02:25:03,211 DEBUG [RS:0;asf000:47701-EventThread]
zookeeper.ZooKeeperWatcher(307): regionserver:47701 Received ZooKeeper Event,
type=None, state=SyncConnected, path=null
2013-05-31 02:25:03,212 DEBUG [RS:0;asf000:47701-EventThread]
zookeeper.ZooKeeperWatcher(384): regionserver:47701-0x13ef867090c0003 connected
2013-05-31 02:25:03,212 DEBUG [RS:1;asf000:48644] zookeeper.ZKUtil(431):
regionserver:48644-0x13ef867090c0001 Set watcher on existing znode=/hbase/master
2013-05-31 02:25:03,213 DEBUG [RS:2;asf000:47305] zookeeper.ZKUtil(431):
regionserver:47305-0x13ef867090c0002 Set watcher on existing znode=/hbase/master
2013-05-31 02:25:03,214 DEBUG [RS:0;asf000:47701] zookeeper.ZKUtil(431):
regionserver:47701-0x13ef867090c0003 Set watcher on existing znode=/hbase/master
2013-05-31 02:25:03,221 DEBUG [RS:0;asf000:47701] zookeeper.ZKUtil(433):
regionserver:47701-0x13ef867090c0003 Set watcher on znode that does not yet
exist, /hbase/running
2013-05-31 02:25:03,221 DEBUG [RS:2;asf000:47305] zookeeper.ZKUtil(433):
regionserver:47305-0x13ef867090c0002 Set watcher on znode that does not yet
exist, /hbase/running
2013-05-31 02:25:03,222 DEBUG [RS:1;asf000:48644] zookeeper.ZKUtil(433):
regionserver:48644-0x13ef867090c0001 Set watcher on znode that does not yet
exist, /hbase/running
{code}
> Change ServerName so it uses hostname only, not FQDN hostnames
> ---
>
> Key: HBASE-8645
> URL: https://issues.apache.org/jira/browse/HBASE-8645
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Assignee: stack
> Attachments: 8645.txt, 8645.txt, 8645v2.txt, 8645v3.txt, 8645v4.txt,
> 8645v5.txt
>
>
> No need of dragging around domain part of a hostname when we mak
[jira] [Commented] (HBASE-8645) Change ServerName so it uses hostname only, not FQDN hostnames
[ https://issues.apache.org/jira/browse/HBASE-8645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13672490#comment-13672490 ] stack commented on HBASE-8645: -- Any chance of a review of this approach? Thanks lads. > Change ServerName so it uses hostname only, not FQDN hostnames > --- > > Key: HBASE-8645 > URL: https://issues.apache.org/jira/browse/HBASE-8645 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack > Attachments: 8645.txt, 8645.txt, 8645v2.txt, 8645v3.txt, 8645v4.txt, > 8645v5.txt > > > No need of dragging around domain part of a hostname when we make > ServerNames. Rather than do a.example.org,6,12345 as we currently do, > just output: 1,6,12345. This will make names displayed in UI and or shell > smaller. Will also tighten up our logs a little, especially where ServerName > is part of a thrad name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7462) TestDrainingServer is an integration test. It should be a unit test instead
[ https://issues.apache.org/jira/browse/HBASE-7462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13672484#comment-13672484 ] Gustavo Anatoly commented on HBASE-7462: Hi, Nicolas. After long time, I have some code. Could you please review the test? http://goo.gl/1WVj3 Thankful. > TestDrainingServer is an integration test. It should be a unit test instead > --- > > Key: HBASE-7462 > URL: https://issues.apache.org/jira/browse/HBASE-7462 > Project: HBase > Issue Type: Wish > Components: test >Affects Versions: 0.95.2 >Reporter: Nicolas Liochon >Priority: Trivial > Labels: noob > > TestDrainingServer tests the function that allows to say that a regionserver > should not get new regions. > As it is written today, it's an integration test: it starts & stops a cluster. > The test would be more efficient if it would just check that the > AssignmentManager does not use the drained region server; whatever the > circumstances (bulk assign or not for example). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8652) Number of compacting KVs is not reset at the end of compaction
[
https://issues.apache.org/jira/browse/HBASE-8652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13672331#comment-13672331
]
samar commented on HBASE-8652:
--
Can we instead just change Heading on the master status page to someting like
"Total compaction KVs".
or
instead of resting to show that its done.. We could gracefully display
(totalCompactingKVs - currentCompactedKVs) which would finally become 0 after
its 100% complete
> Number of compacting KVs is not reset at the end of compaction
> --
>
> Key: HBASE-8652
> URL: https://issues.apache.org/jira/browse/HBASE-8652
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>
> Looking at master:60010/master-status#compactStas , I noticed that 'Num.
> Compacting KVs' column stays unchanged at non-zero value(s).
> In DefaultCompactor#compact(), we have this at the beginning:
> {code}
> this.progress = new CompactionProgress(fd.maxKeyCount);
> {code}
> But progress.totalCompactingKVs is not reset at the end of compact().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
