[jira] [Assigned] (HBASE-3680) Publish more metrics about mslab
[ https://issues.apache.org/jira/browse/HBASE-3680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon reassigned HBASE-3680: -- Assignee: (was: Todd Lipcon) > Publish more metrics about mslab > > > Key: HBASE-3680 > URL: https://issues.apache.org/jira/browse/HBASE-3680 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.90.1 >Reporter: Jean-Daniel Cryans > Fix For: 0.92.3 > > Attachments: hbase-3680.txt, hbase-3680.txt > > > We have been using mslab on all our clusters for a while now and it seems it > tends to OOME or send us into GC loops of death a lot more than it used to. > For example, one RS with mslab enabled and 7GB of heap died out of OOME this > afternoon; it had .55GB in the block cache and 2.03GB in the memstores which > doesn't account for much... but it could be that because of mslab a lot of > space was lost in those incomplete 2MB blocks and without metrics we can't > really tell. Compactions were running at the time of the OOME and I see block > cache activity. The average load on that cluster is 531. > We should at least publish the total size of all those blocks and maybe even > take actions based on that (like force flushing). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15843) Replace RegionState.getRegionInTransition() Map with a Set
[
https://issues.apache.org/jira/browse/HBASE-15843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286073#comment-15286073
]
Hadoop QA commented on HBASE-15843:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 5m 55s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
4s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 21s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 4s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
41s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 31s
{color} | {color:red} hbase-rsgroup in master has 7 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 13s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 30s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 30s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 2s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 8m
37s {color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1
2.5.2 2.6.0. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 12s
{color} | {color:red} hbase-server generated 1 new + 0 unchanged - 0 fixed = 1
total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 13s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 55s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 99m 9s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 24s
{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
37s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 142m 0s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
| | String is incompatible with expected argument type
org.apache.hadoop.hbase.master.RegionState in
org.apache.hadoop.hbase.util.HBaseFsckRepair.waitUntilAssigned(Admin,
HRegionInfo) At
[jira] [Commented] (HBASE-14850) C++ client implementation
[ https://issues.apache.org/jira/browse/HBASE-14850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286069#comment-15286069 ] Sudeep Sunthankar commented on HBASE-14850: --- Apologies for posting the comment twice as I had encountered an exception on Jira while posting the first time. -- Regards, Sudeep Sunthankar > C++ client implementation > - > > Key: HBASE-14850 > URL: https://issues.apache.org/jira/browse/HBASE-14850 > Project: HBase > Issue Type: Task >Reporter: Elliott Clark > > It's happening. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14850) C++ client implementation
[
https://issues.apache.org/jira/browse/HBASE-14850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286065#comment-15286065
]
Sudeep Sunthankar commented on HBASE-14850:
---
Hi,
This is Sudeep. I am part of Vamsi's team. As per discussion with Vamsi, we are
working on compiling hbase-native-client using make. The compilation was
working fine and I was able to build using both make and buck. It is giving me
the below error since I synced the most recent changes today.
In file included from if/Admin.pb.h:30:0,
from if/Admin.pb.cc:5:
if/WAL.pb.h:1103:37: error: 'google::protobuf::uint64
hbase::pb::StoreDescriptor::store_file_size() const' cannot be overloaded
inline ::google::protobuf::uint64 store_file_size() const;
^
if/WAL.pb.h:1084:14: error: with 'int
hbase::pb::StoreDescriptor::store_file_size() const'
inline int store_file_size() const;
^
if/WAL.pb.h:3527:35: error: prototype for 'google::protobuf::uint64
hbase::pb::StoreDescriptor::store_file_size() const' does not match any in
class 'hbase::pb::StoreDescriptor'
inline ::google::protobuf::uint64 StoreDescriptor::store_file_size() const {
^
if/WAL.pb.h:3460:12: error: candidate is: int
hbase::pb::StoreDescriptor::store_file_size() const
inline int StoreDescriptor::store_file_size() const {
--
Best Regards,
Sudeep Sunthankar
> C++ client implementation
> -
>
> Key: HBASE-14850
> URL: https://issues.apache.org/jira/browse/HBASE-14850
> Project: HBase
> Issue Type: Task
>Reporter: Elliott Clark
>
> It's happening.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15835) HBaseTestingUtility#startMiniCluster throws "HMasterAddress already in use" RuntimeException when a local instance of HBase is running
[
https://issues.apache.org/jira/browse/HBASE-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Vimont updated HBASE-15835:
--
Status: Patch Available (was: Open)
Note that I went with "OPTION 2" as outlined in the description to this JIRA
item (i.e. force random port assignment if user has not explicitly defined
alternate port).
> HBaseTestingUtility#startMiniCluster throws "HMasterAddress already in use"
> RuntimeException when a local instance of HBase is running
> --
>
> Key: HBASE-15835
> URL: https://issues.apache.org/jira/browse/HBASE-15835
> Project: HBase
> Issue Type: Bug
> Components: API
>Affects Versions: 2.0.0
>Reporter: Daniel Vimont
>Assignee: Daniel Vimont
> Labels: easyfix
> Fix For: 2.0.0
>
> Attachments: HBASE-15835-v1.patch
>
>
> When a MiniCluster is being started with the
> {{HBaseTestUtility#startMiniCluster}} method (most typically in the context
> of JUnit testing), if a local HBase instance is already running (or for that
> matter, another thread with another MiniCluster is already running), the
> startup will fail with a RuntimeException saying "HMasterAddress already in
> use", referring explicitly to contention for the same default master info
> port (16010).
> This problem most recently came up in conjunction with HBASE-14876 and its
> sub-JIRAs (development of new HBase-oriented Maven archetypes), but this is
> apparently a known issue to veteran developers, who tend to set up the
> @BeforeClass sections of their test modules with code similar to the
> following:
> {code}
> UTIL = HBaseTestingUtility.createLocalHTU();
> // disable UI's on test cluster.
> UTIL.getConfiguration().setInt("hbase.master.info.port", -1);
> UTIL.getConfiguration().setInt("hbase.regionserver.info.port", -1);
> UTIL.startMiniCluster();
> {code}
> A comprehensive solution modeled on this should be put directly into
> HBaseTestUtility's main constructor, using one of the following options:
> OPTION 1 (always force random port assignment):
> {code}
> this.getConfiguration().setInt(HConstants.MASTER_INFO_PORT, -1);
> this.getConfiguration().setInt(HConstants.REGIONSERVER_PORT, -1);
> {code}
> OPTION 2 (always force random port assignment if user has not explicitly
> defined alternate port):
> {code}
> Configuration conf = this.getConfiguration();
> if (conf.getInt(HConstants.MASTER_INFO_PORT,
> HConstants.DEFAULT_MASTER_INFOPORT)
> == HConstants.DEFAULT_MASTER_INFOPORT) {
> conf.setInt(HConstants.MASTER_INFO_PORT, -1);
> }
> if (conf.getInt(HConstants.REGIONSERVER_PORT,
> HConstants.DEFAULT_REGIONSERVER_PORT)
> == HConstants.DEFAULT_REGIONSERVER_PORT) {
> conf.setInt(HConstants.REGIONSERVER_PORT, -1);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14850) C++ client implementation
[
https://issues.apache.org/jira/browse/HBASE-14850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286058#comment-15286058
]
Sudeep Sunthankar commented on HBASE-14850:
---
Hi,
This is Sudeep. I am part of Vamsi's team. As per discussion with Vamsi, we are
working on compiling hbase-native-client using make. The compilation was
working fine and I was able to build using both make and buck. It is giving me
the below error since I synced the most recent changes today.
In file included from if/Admin.pb.h:30:0,
from if/Admin.pb.cc:5:
if/WAL.pb.h:1103:37: error: 'google::protobuf::uint64
hbase::pb::StoreDescriptor::store_file_size() const' cannot be overloaded
inline ::google::protobuf::uint64 store_file_size() const;
^
if/WAL.pb.h:1084:14: error: with 'int
hbase::pb::StoreDescriptor::store_file_size() const'
inline int store_file_size() const;
^
if/WAL.pb.h:3527:35: error: prototype for 'google::protobuf::uint64
hbase::pb::StoreDescriptor::store_file_size() const' does not match any in
class 'hbase::pb::StoreDescriptor'
inline ::google::protobuf::uint64 StoreDescriptor::store_file_size() const {
^
if/WAL.pb.h:3460:12: error: candidate is: int
hbase::pb::StoreDescriptor::store_file_size() const
inline int StoreDescriptor::store_file_size() const {
--
Best Regards,
Sudeep Sunthankar
> C++ client implementation
> -
>
> Key: HBASE-14850
> URL: https://issues.apache.org/jira/browse/HBASE-14850
> Project: HBase
> Issue Type: Task
>Reporter: Elliott Clark
>
> It's happening.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15835) HBaseTestingUtility#startMiniCluster throws "HMasterAddress already in use" RuntimeException when a local instance of HBase is running
[
https://issues.apache.org/jira/browse/HBASE-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Vimont updated HBASE-15835:
--
Attachment: HBASE-15835-v1.patch
> HBaseTestingUtility#startMiniCluster throws "HMasterAddress already in use"
> RuntimeException when a local instance of HBase is running
> --
>
> Key: HBASE-15835
> URL: https://issues.apache.org/jira/browse/HBASE-15835
> Project: HBase
> Issue Type: Bug
> Components: API
>Affects Versions: 2.0.0
>Reporter: Daniel Vimont
>Assignee: Daniel Vimont
> Labels: easyfix
> Fix For: 2.0.0
>
> Attachments: HBASE-15835-v1.patch
>
>
> When a MiniCluster is being started with the
> {{HBaseTestUtility#startMiniCluster}} method (most typically in the context
> of JUnit testing), if a local HBase instance is already running (or for that
> matter, another thread with another MiniCluster is already running), the
> startup will fail with a RuntimeException saying "HMasterAddress already in
> use", referring explicitly to contention for the same default master info
> port (16010).
> This problem most recently came up in conjunction with HBASE-14876 and its
> sub-JIRAs (development of new HBase-oriented Maven archetypes), but this is
> apparently a known issue to veteran developers, who tend to set up the
> @BeforeClass sections of their test modules with code similar to the
> following:
> {code}
> UTIL = HBaseTestingUtility.createLocalHTU();
> // disable UI's on test cluster.
> UTIL.getConfiguration().setInt("hbase.master.info.port", -1);
> UTIL.getConfiguration().setInt("hbase.regionserver.info.port", -1);
> UTIL.startMiniCluster();
> {code}
> A comprehensive solution modeled on this should be put directly into
> HBaseTestUtility's main constructor, using one of the following options:
> OPTION 1 (always force random port assignment):
> {code}
> this.getConfiguration().setInt(HConstants.MASTER_INFO_PORT, -1);
> this.getConfiguration().setInt(HConstants.REGIONSERVER_PORT, -1);
> {code}
> OPTION 2 (always force random port assignment if user has not explicitly
> defined alternate port):
> {code}
> Configuration conf = this.getConfiguration();
> if (conf.getInt(HConstants.MASTER_INFO_PORT,
> HConstants.DEFAULT_MASTER_INFOPORT)
> == HConstants.DEFAULT_MASTER_INFOPORT) {
> conf.setInt(HConstants.MASTER_INFO_PORT, -1);
> }
> if (conf.getInt(HConstants.REGIONSERVER_PORT,
> HConstants.DEFAULT_REGIONSERVER_PORT)
> == HConstants.DEFAULT_REGIONSERVER_PORT) {
> conf.setInt(HConstants.REGIONSERVER_PORT, -1);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-3727) MultiHFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286024#comment-15286024 ] Anoop Sam John commented on HBASE-3727: --- It is unassigned. Pls go ahead and assign to ur name and work on patch. Good on you [~nmarwah] > MultiHFileOutputFormat > -- > > Key: HBASE-3727 > URL: https://issues.apache.org/jira/browse/HBASE-3727 > Project: HBase > Issue Type: New Feature >Reporter: Andrew Purtell >Priority: Minor > Attachments: MultiHFileOutputFormat.java, MultiHFileOutputFormat.java > > > Like MultiTableOutputFormat, but outputting HFiles. Key is tablename as an > IBW. Creates sub-writers (code cut and pasted from HFileOutputFormat) on > demand that produce HFiles in per-table subdirectories of the configured > output path. Does not currently support partitioning for existing tables / > incremental update. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15843) Replace RegionState.getRegionInTransition() Map with a Set
[
https://issues.apache.org/jira/browse/HBASE-15843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285989#comment-15285989
]
stack commented on HBASE-15843:
---
+1
> Replace RegionState.getRegionInTransition() Map with a Set
> --
>
> Key: HBASE-15843
> URL: https://issues.apache.org/jira/browse/HBASE-15843
> Project: HBase
> Issue Type: Improvement
> Components: master, Region Assignment
>Affects Versions: 2.0.0, 1.3.0, 1.2.1
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Trivial
> Fix For: 2.0.0
>
> Attachments: HBASE-15843-v0.patch
>
>
> RegionState.getRegionInTransition() is always used as a Set.
> replace the Map with a Set, avoid some allocation and extra code.
> also ClusterStatus.RegionInTransition has duplicated information.
> The spec field contains the regionName (not encoded).
> but we have the same info as part of the region_state with the HRegionInfo
> serialized.
> unfortunately I don't think we can get rid of 'spec' that being a required
> field.
> {noformat}
> message RegionInTransition {
> required RegionSpecifier spec = 1;
> required RegionState region_state = 2;
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15837) More gracefully handle a negative memstoreSize
[ https://issues.apache.org/jira/browse/HBASE-15837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-15837: -- Attachment: hbase-memstore-size-accounting.patch v0 patch against branch-1.1 illustrating the fix. Not tested yet. > More gracefully handle a negative memstoreSize > -- > > Key: HBASE-15837 > URL: https://issues.apache.org/jira/browse/HBASE-15837 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: 2.0.0 > > Attachments: HBASE-15837.001.patch, > hbase-memstore-size-accounting.patch > > > Over in PHOENIX-2883, I've been trying to figure out how to track down the > root cause of an issue we were seeing where a negative memstoreSize was > ultimately causing an RS to abort. The tl;dr version is > * Something causes memstoreSize to be negative (not sure what is doing this > yet) > * All subsequent flushes short-circuit and don't run because they think there > is no data to flush > * The region is eventually closed (commonly, for a move). > * A final flush is attempted on each store before closing (which also > short-circuit for the same reason), leaving unflushed data in each store. > * The sanity check that each store's size is zero fails and the RS aborts. > I have a little patch which I think should improve our failure case around > this, preventing the RS abort safely (forcing a flush when memstoreSize is > negative) and logging a calltrace when an update to memstoreSize make it > negative (to find culprits in the future). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15837) More gracefully handle a negative memstoreSize
[
https://issues.apache.org/jira/browse/HBASE-15837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285967#comment-15285967
]
Enis Soztutar commented on HBASE-15837:
---
[~elserj] I was also looking at this to understand what may have happened. The
memstore size discrepancy happens when the index update fails for Phoenix. I
also have a patch that should fix the root cause.
It is something like this:
- HRegion.memstoreSize gets updated only after a batch is finished.
HStore.getMemstoreSize() gets updated everytime we add a new cell. If the
transaction rolls back from the memstore, we also decrement the size in
HStore.
- HRegion.batchMutate() has the code:
{code}
long addedSize = doMiniBatchMutation(batchOp);
long newSize = this.addAndGetGlobalMemstoreSize(addedSize);
{code}
Which means that the HRegion.memstoreSize will get update ONLY when
doMiniBatchMutation DOES NOT throw an exception.
- In most cases when doMiniBatchMutation() throws an exception, we sometimes
undo the memstore operations by rolling back. This ensures that updates are
removed and hstore size is decremented back.
- However, in the case that postBatchMutate() throws exception, the WAL is
already sync()'ed, so we cannot rollback the transaction. We actually have the
edits in the memstore and WAL, but fail to update the region memstore size.
HStore size is correctly updated, thus leaving the discrepancy.
- Phoenix secondary indexing does the scans in the preBatchMutate() call and
does the secondary index writes in the postBatchMutate() call. Thus, if the
secondary index writes fail, we end up with the accounting error, and
subsequent aborting of regionservers in region close.
> More gracefully handle a negative memstoreSize
> --
>
> Key: HBASE-15837
> URL: https://issues.apache.org/jira/browse/HBASE-15837
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: 2.0.0
>
> Attachments: HBASE-15837.001.patch
>
>
> Over in PHOENIX-2883, I've been trying to figure out how to track down the
> root cause of an issue we were seeing where a negative memstoreSize was
> ultimately causing an RS to abort. The tl;dr version is
> * Something causes memstoreSize to be negative (not sure what is doing this
> yet)
> * All subsequent flushes short-circuit and don't run because they think there
> is no data to flush
> * The region is eventually closed (commonly, for a move).
> * A final flush is attempted on each store before closing (which also
> short-circuit for the same reason), leaving unflushed data in each store.
> * The sanity check that each store's size is zero fails and the RS aborts.
> I have a little patch which I think should improve our failure case around
> this, preventing the RS abort safely (forcing a flush when memstoreSize is
> negative) and logging a calltrace when an update to memstoreSize make it
> negative (to find culprits in the future).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15842) SnapshotInfo should display ownership information
[
https://issues.apache.org/jira/browse/HBASE-15842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15842:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
Thanks for the review, Matteo.
> SnapshotInfo should display ownership information
> -
>
> Key: HBASE-15842
> URL: https://issues.apache.org/jira/browse/HBASE-15842
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 15842.v1.txt, 15842.v1.txt
>
>
> Currently SnapshotInfo doesn't show snapshot owner:
> {code}
> hbase org.apache.hadoop.hbase.snapshot.SnapshotInfo -snapshot
> snapshot_table_qm0uxvk19x -stats -schema
> ...
> Snapshot Info
>
>Name: snapshot_table_qm0uxvk19x
>Type: FLUSH
> Table: table_qm0uxvk19x
> Format: 2
> Created: 2016-05-16T20:54:08
> ...
> {code}
> This JIRA is to add ownership information to the display.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15843) Replace RegionState.getRegionInTransition() Map with a Set
[
https://issues.apache.org/jira/browse/HBASE-15843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matteo Bertozzi updated HBASE-15843:
Status: Patch Available (was: Open)
> Replace RegionState.getRegionInTransition() Map with a Set
> --
>
> Key: HBASE-15843
> URL: https://issues.apache.org/jira/browse/HBASE-15843
> Project: HBase
> Issue Type: Improvement
> Components: master, Region Assignment
>Affects Versions: 1.2.1, 2.0.0, 1.3.0
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Trivial
> Fix For: 2.0.0
>
> Attachments: HBASE-15843-v0.patch
>
>
> RegionState.getRegionInTransition() is always used as a Set.
> replace the Map with a Set, avoid some allocation and extra code.
> also ClusterStatus.RegionInTransition has duplicated information.
> The spec field contains the regionName (not encoded).
> but we have the same info as part of the region_state with the HRegionInfo
> serialized.
> unfortunately I don't think we can get rid of 'spec' that being a required
> field.
> {noformat}
> message RegionInTransition {
> required RegionSpecifier spec = 1;
> required RegionState region_state = 2;
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15843) Replace RegionState.getRegionInTransition() Map with a Set
[
https://issues.apache.org/jira/browse/HBASE-15843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matteo Bertozzi updated HBASE-15843:
Attachment: HBASE-15843-v0.patch
> Replace RegionState.getRegionInTransition() Map with a Set
> --
>
> Key: HBASE-15843
> URL: https://issues.apache.org/jira/browse/HBASE-15843
> Project: HBase
> Issue Type: Improvement
> Components: master, Region Assignment
>Affects Versions: 2.0.0, 1.3.0, 1.2.1
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Trivial
> Fix For: 2.0.0
>
> Attachments: HBASE-15843-v0.patch
>
>
> RegionState.getRegionInTransition() is always used as a Set.
> replace the Map with a Set, avoid some allocation and extra code.
> also ClusterStatus.RegionInTransition has duplicated information.
> The spec field contains the regionName (not encoded).
> but we have the same info as part of the region_state with the HRegionInfo
> serialized.
> unfortunately I don't think we can get rid of 'spec' that being a required
> field.
> {noformat}
> message RegionInTransition {
> required RegionSpecifier spec = 1;
> required RegionState region_state = 2;
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15843) Replace RegionState.getRegionInTransition() Map with a Set
Matteo Bertozzi created HBASE-15843:
---
Summary: Replace RegionState.getRegionInTransition() Map with a Set
Key: HBASE-15843
URL: https://issues.apache.org/jira/browse/HBASE-15843
Project: HBase
Issue Type: Improvement
Components: master, Region Assignment
Affects Versions: 1.2.1, 2.0.0, 1.3.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Trivial
Fix For: 2.0.0
RegionState.getRegionInTransition() is always used as a Set.
replace the Map with a Set, avoid some allocation and extra code.
also ClusterStatus.RegionInTransition has duplicated information.
The spec field contains the regionName (not encoded).
but we have the same info as part of the region_state with the HRegionInfo
serialized.
unfortunately I don't think we can get rid of 'spec' that being a required
field.
{noformat}
message RegionInTransition {
required RegionSpecifier spec = 1;
required RegionState region_state = 2;
}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15834) Correct Bloom filter documentation in section 96.4 of Reference Guide
[
https://issues.apache.org/jira/browse/HBASE-15834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
li xiang updated HBASE-15834:
-
Attachment: HBASE-15834.patch.v1
Revised to remove the change from 0.96 to 0.95.1, as 0.95.x is not official
releases. Thanks for Jerry He's suggestion!
> Correct Bloom filter documentation in section 96.4 of Reference Guide
> -
>
> Key: HBASE-15834
> URL: https://issues.apache.org/jira/browse/HBASE-15834
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: li xiang
>Priority: Minor
> Attachments: HBASE-15834.patch.v0, HBASE-15834.patch.v1
>
>
> In section 96.4, the second paragraph from the bottom
> {code}
> Since HBase 0.96, row-based Bloom filters are enabled by default. (HBASE-)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15835) HBaseTestingUtility#startMiniCluster throws "HMasterAddress already in use" RuntimeException when a local instance of HBase is running
[
https://issues.apache.org/jira/browse/HBASE-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285902#comment-15285902
]
Daniel Vimont commented on HBASE-15835:
---
Yep, thanks!!
> HBaseTestingUtility#startMiniCluster throws "HMasterAddress already in use"
> RuntimeException when a local instance of HBase is running
> --
>
> Key: HBASE-15835
> URL: https://issues.apache.org/jira/browse/HBASE-15835
> Project: HBase
> Issue Type: Bug
> Components: API
>Affects Versions: 2.0.0
>Reporter: Daniel Vimont
>Assignee: Daniel Vimont
> Labels: easyfix
> Fix For: 2.0.0
>
>
> When a MiniCluster is being started with the
> {{HBaseTestUtility#startMiniCluster}} method (most typically in the context
> of JUnit testing), if a local HBase instance is already running (or for that
> matter, another thread with another MiniCluster is already running), the
> startup will fail with a RuntimeException saying "HMasterAddress already in
> use", referring explicitly to contention for the same default master info
> port (16010).
> This problem most recently came up in conjunction with HBASE-14876 and its
> sub-JIRAs (development of new HBase-oriented Maven archetypes), but this is
> apparently a known issue to veteran developers, who tend to set up the
> @BeforeClass sections of their test modules with code similar to the
> following:
> {code}
> UTIL = HBaseTestingUtility.createLocalHTU();
> // disable UI's on test cluster.
> UTIL.getConfiguration().setInt("hbase.master.info.port", -1);
> UTIL.getConfiguration().setInt("hbase.regionserver.info.port", -1);
> UTIL.startMiniCluster();
> {code}
> A comprehensive solution modeled on this should be put directly into
> HBaseTestUtility's main constructor, using one of the following options:
> OPTION 1 (always force random port assignment):
> {code}
> this.getConfiguration().setInt(HConstants.MASTER_INFO_PORT, -1);
> this.getConfiguration().setInt(HConstants.REGIONSERVER_PORT, -1);
> {code}
> OPTION 2 (always force random port assignment if user has not explicitly
> defined alternate port):
> {code}
> Configuration conf = this.getConfiguration();
> if (conf.getInt(HConstants.MASTER_INFO_PORT,
> HConstants.DEFAULT_MASTER_INFOPORT)
> == HConstants.DEFAULT_MASTER_INFOPORT) {
> conf.setInt(HConstants.MASTER_INFO_PORT, -1);
> }
> if (conf.getInt(HConstants.REGIONSERVER_PORT,
> HConstants.DEFAULT_REGIONSERVER_PORT)
> == HConstants.DEFAULT_REGIONSERVER_PORT) {
> conf.setInt(HConstants.REGIONSERVER_PORT, -1);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-15835) HBaseTestingUtility#startMiniCluster throws "HMasterAddress already in use" RuntimeException when a local instance of HBase is running
[
https://issues.apache.org/jira/browse/HBASE-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Vimont reassigned HBASE-15835:
-
Assignee: Daniel Vimont
> HBaseTestingUtility#startMiniCluster throws "HMasterAddress already in use"
> RuntimeException when a local instance of HBase is running
> --
>
> Key: HBASE-15835
> URL: https://issues.apache.org/jira/browse/HBASE-15835
> Project: HBase
> Issue Type: Bug
> Components: API
>Affects Versions: 2.0.0
>Reporter: Daniel Vimont
>Assignee: Daniel Vimont
> Labels: easyfix
> Fix For: 2.0.0
>
>
> When a MiniCluster is being started with the
> {{HBaseTestUtility#startMiniCluster}} method (most typically in the context
> of JUnit testing), if a local HBase instance is already running (or for that
> matter, another thread with another MiniCluster is already running), the
> startup will fail with a RuntimeException saying "HMasterAddress already in
> use", referring explicitly to contention for the same default master info
> port (16010).
> This problem most recently came up in conjunction with HBASE-14876 and its
> sub-JIRAs (development of new HBase-oriented Maven archetypes), but this is
> apparently a known issue to veteran developers, who tend to set up the
> @BeforeClass sections of their test modules with code similar to the
> following:
> {code}
> UTIL = HBaseTestingUtility.createLocalHTU();
> // disable UI's on test cluster.
> UTIL.getConfiguration().setInt("hbase.master.info.port", -1);
> UTIL.getConfiguration().setInt("hbase.regionserver.info.port", -1);
> UTIL.startMiniCluster();
> {code}
> A comprehensive solution modeled on this should be put directly into
> HBaseTestUtility's main constructor, using one of the following options:
> OPTION 1 (always force random port assignment):
> {code}
> this.getConfiguration().setInt(HConstants.MASTER_INFO_PORT, -1);
> this.getConfiguration().setInt(HConstants.REGIONSERVER_PORT, -1);
> {code}
> OPTION 2 (always force random port assignment if user has not explicitly
> defined alternate port):
> {code}
> Configuration conf = this.getConfiguration();
> if (conf.getInt(HConstants.MASTER_INFO_PORT,
> HConstants.DEFAULT_MASTER_INFOPORT)
> == HConstants.DEFAULT_MASTER_INFOPORT) {
> conf.setInt(HConstants.MASTER_INFO_PORT, -1);
> }
> if (conf.getInt(HConstants.REGIONSERVER_PORT,
> HConstants.DEFAULT_REGIONSERVER_PORT)
> == HConstants.DEFAULT_REGIONSERVER_PORT) {
> conf.setInt(HConstants.REGIONSERVER_PORT, -1);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11625) Reading datablock throws "Invalid HFile block magic" and can not switch to hdfs checksum
[
https://issues.apache.org/jira/browse/HBASE-11625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285896#comment-15285896
]
Hadoop QA commented on HBASE-11625:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
14s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 54s
{color} | {color:green} branch-1.2 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s
{color} | {color:green} branch-1.2 passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
21s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
18s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 2s
{color} | {color:green} branch-1.2 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 46s
{color} | {color:green} branch-1.2 passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
2s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 4m
53s {color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1
2.5.2 2.6.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
33s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 19m 26s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 51m 46s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.replication.regionserver.TestReplicationThrottler |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804313/HBASE-11625-branch-1.2-v2.patch
|
| JIRA Issue | HBASE-11625 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux proserpina.apache.org 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/test_framework/yetus-0.2.1/lib/precommit/personality/hbase.sh
|
| git revision | branch-1.2 / 7c0fc0d |
| Default Java | 1.7.0_79 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/usr/local/jenkins/java/jdk1.7.0_79:1.7.0_79 |
|
[jira] [Commented] (HBASE-15765) RPC handlers should steal from other queues in the same or higher priority
[
https://issues.apache.org/jira/browse/HBASE-15765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285893#comment-15285893
]
Heng Chen commented on HBASE-15765:
---
{quote}
we can steal 1 RPC every ~3seconds if I read the patch right.
{quote}
It is not steal task every 3 seconds. In the patch, we firstly get task from
our original queue, if timeout (3s), then we will try to steal task from other
queue.
{quote}
the handlers still do not steal from the queues in the same tier, but a
different "shard"
{quote}
Sorry, i am not sure what is "queues in the same tier"?
If i understand correctly, the queues in one handler pools are in the same
priority level, right? Or you mean general pool and replication pool are in
the same tier?
> RPC handlers should steal from other queues in the same or higher priority
> --
>
> Key: HBASE-15765
> URL: https://issues.apache.org/jira/browse/HBASE-15765
> Project: HBase
> Issue Type: Improvement
>Reporter: Enis Soztutar
> Attachments: HBASE-15765.patch, HBASE-15765_v1.patch
>
>
> With the simple rpc scheduler model by default, a request first falls in one
> type of queue (priority, general, replication) and it will get "sharded" in a
> queue within that priority level (for example there are 2 queues for the high
> priority level). The handlers can only take from their specific queues, and
> nothing else.
> Even if a request is waiting in a high priority queue, the handlers within
> the same priority level or a lower priority level will not see these requests
> (which is the root cause for HBASE-15613). We should be able to steal from
> other queues, without losing the prioritization and sharding benefits.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15842) SnapshotInfo should display ownership information
[
https://issues.apache.org/jira/browse/HBASE-15842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285878#comment-15285878
]
Hadoop QA commented on HBASE-15842:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 1s
{color} | {color:blue} The patch file was not named according to hbase's naming
conventions. Please see
https://yetus.apache.org/documentation/0.2.1/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
52s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 8m
33s {color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1
2.5.2 2.6.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 113m 47s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 137m 52s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804291/15842.v1.txt |
| JIRA Issue | HBASE-15842 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux penates.apache.org 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT
Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HBASE-15667) Add more clarity to Reference Guide related to importing Eclipse Formatter
[
https://issues.apache.org/jira/browse/HBASE-15667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285868#comment-15285868
]
Hadoop QA commented on HBASE-15667:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 30s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 50s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
8s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 9m
58s {color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1
2.5.2 2.6.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 3s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 45s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 29m 29s {color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 61m 43s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.coprocessor.TestCoprocessorInterface |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804304/HBASE-15667v2%20%281%29%20%281%29.patch
|
| JIRA Issue | HBASE-15667 |
| Optional Tests | asflicense javac javadoc unit |
| uname | Linux asf910.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/test_framework/yetus-0.2.1/lib/precommit/personality/hbase.sh
|
| git revision | master / c3223a5 |
| Default Java | 1.7.0_79 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/usr/local/jenkins/java/jdk1.7.0_79:1.7.0_79 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/1913/artifact/patchprocess/patch-unit-root.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/1913/artifact/patchprocess/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/1913/testReport/ |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/1913/console |
| Powered by | Apache Yetus 0.2.1 http://yetus.apache.org |
This message was automatically generated.
> Add more clarity to Reference Guide related to importing Eclipse Formatter
> --
>
> Key: HBASE-15667
> URL: https://issues.apache.org/jira/browse/HBASE-15667
> Project: HBase
> Issue Type: Improvement
> Components: documentation
>Reporter: Sai Teja Ranuva
>Assignee: Sai Teja Ranuva
>Priority: Trivial
> Labels: trivial
> Attachments: HBASE-15667.patch, HBASE-15667.patch,
> HBASE-15667_1.patch, HBASE-15667_1.patch, HBASE-15667v2 (1) (1).patch,
> HBASE-15667v2 (1).patch, HBASE-15667v2.patch, donothing.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> In Hbase Reference Guide:
> 141.1.1. Code Formatting
> in procedure bullet point 2: It is not clear what the menu item is.
> It should be changed to the following:
> "In Preferences, click
[jira] [Commented] (HBASE-11625) Reading datablock throws "Invalid HFile block magic" and can not switch to hdfs checksum
[ https://issues.apache.org/jira/browse/HBASE-11625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285822#comment-15285822 ] Appy commented on HBASE-11625: -- Yeah, these were related. Uploaded a new patch, lets see its results. > Reading datablock throws "Invalid HFile block magic" and can not switch to > hdfs checksum > - > > Key: HBASE-11625 > URL: https://issues.apache.org/jira/browse/HBASE-11625 > Project: HBase > Issue Type: Bug > Components: HFile >Affects Versions: 0.94.21, 0.98.4, 0.98.5, 1.0.1.1, 1.0.3 >Reporter: qian wang >Assignee: Appy > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: 2711de1fdf73419d9f8afc6a8b86ce64.gz, > HBASE-11625-branch-1-v1.patch, HBASE-11625-branch-1.2-v1.patch, > HBASE-11625-branch-1.2-v2.patch, HBASE-11625-master-v2.patch, > HBASE-11625-master-v3.patch, HBASE-11625-master.patch, HBASE-11625.patch, > correct-hfile, corrupted-header-hfile > > > when using hbase checksum,call readBlockDataInternal() in hfileblock.java, it > could happen file corruption but it only can switch to hdfs checksum > inputstream till validateBlockChecksum(). If the datablock's header corrupted > when b = new HFileBlock(),it throws the exception "Invalid HFile block magic" > and the rpc call fail -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11625) Reading datablock throws "Invalid HFile block magic" and can not switch to hdfs checksum
[ https://issues.apache.org/jira/browse/HBASE-11625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-11625: - Attachment: HBASE-11625-branch-1.2-v2.patch > Reading datablock throws "Invalid HFile block magic" and can not switch to > hdfs checksum > - > > Key: HBASE-11625 > URL: https://issues.apache.org/jira/browse/HBASE-11625 > Project: HBase > Issue Type: Bug > Components: HFile >Affects Versions: 0.94.21, 0.98.4, 0.98.5, 1.0.1.1, 1.0.3 >Reporter: qian wang >Assignee: Appy > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: 2711de1fdf73419d9f8afc6a8b86ce64.gz, > HBASE-11625-branch-1-v1.patch, HBASE-11625-branch-1.2-v1.patch, > HBASE-11625-branch-1.2-v2.patch, HBASE-11625-master-v2.patch, > HBASE-11625-master-v3.patch, HBASE-11625-master.patch, HBASE-11625.patch, > correct-hfile, corrupted-header-hfile > > > when using hbase checksum,call readBlockDataInternal() in hfileblock.java, it > could happen file corruption but it only can switch to hdfs checksum > inputstream till validateBlockChecksum(). If the datablock's header corrupted > when b = new HFileBlock(),it throws the exception "Invalid HFile block magic" > and the rpc call fail -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15812) HttpServer fails to come up against hadoop trunk
[
https://issues.apache.org/jira/browse/HBASE-15812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285762#comment-15285762
]
Ted Yu commented on HBASE-15812:
I manually edited master UI URL appending /metrics.
> HttpServer fails to come up against hadoop trunk
>
>
> Key: HBASE-15812
> URL: https://issues.apache.org/jira/browse/HBASE-15812
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Sangjin Lee
>Assignee: Ted Yu
> Fix For: 2.0.0
>
> Attachments: 15812.v1.txt, 15812.v1.txt
>
>
> If you run HBase HttpServer against the hadoop trunk, it fails.
> {noformat}
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/hadoop/metrics/MetricsServlet
> at
> org.apache.hadoop.hbase.http.HttpServer.addDefaultServlets(HttpServer.java:677)
> at
> org.apache.hadoop.hbase.http.HttpServer.initializeWebServer(HttpServer.java:546)
> at org.apache.hadoop.hbase.http.HttpServer.(HttpServer.java:500)
> at org.apache.hadoop.hbase.http.HttpServer.(HttpServer.java:104)
> at
> org.apache.hadoop.hbase.http.HttpServer$Builder.build(HttpServer.java:345)
> at org.apache.hadoop.hbase.http.InfoServer.(InfoServer.java:77)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.putUpWebUI(HRegionServer.java:1697)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:550)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:333)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:139)
> ... 26 more
> {noformat}
> The hadoop trunk removed {{MetricsServlet}} (HADOOP-12504).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15667) Add more clarity to Reference Guide related to importing Eclipse Formatter
[ https://issues.apache.org/jira/browse/HBASE-15667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15667: -- Attachment: HBASE-15667v2 (1) (1).patch > Add more clarity to Reference Guide related to importing Eclipse Formatter > -- > > Key: HBASE-15667 > URL: https://issues.apache.org/jira/browse/HBASE-15667 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Sai Teja Ranuva >Assignee: Sai Teja Ranuva >Priority: Trivial > Labels: trivial > Attachments: HBASE-15667.patch, HBASE-15667.patch, > HBASE-15667_1.patch, HBASE-15667_1.patch, HBASE-15667v2 (1) (1).patch, > HBASE-15667v2 (1).patch, HBASE-15667v2.patch, donothing.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > In Hbase Reference Guide: > 141.1.1. Code Formatting > in procedure bullet point 2: It is not clear what the menu item is. > It should be changed to the following: > "In Preferences, click Java->Code Style->Formatter" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15812) HttpServer fails to come up against hadoop trunk
[
https://issues.apache.org/jira/browse/HBASE-15812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285751#comment-15285751
]
stack commented on HBASE-15812:
---
So, '/metrics' is not exposed in master? How did you get your exception above?
By manually typing /metrics into the URL or by clicking on a link exposed in
the UI?
> HttpServer fails to come up against hadoop trunk
>
>
> Key: HBASE-15812
> URL: https://issues.apache.org/jira/browse/HBASE-15812
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Sangjin Lee
>Assignee: Ted Yu
> Fix For: 2.0.0
>
> Attachments: 15812.v1.txt, 15812.v1.txt
>
>
> If you run HBase HttpServer against the hadoop trunk, it fails.
> {noformat}
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/hadoop/metrics/MetricsServlet
> at
> org.apache.hadoop.hbase.http.HttpServer.addDefaultServlets(HttpServer.java:677)
> at
> org.apache.hadoop.hbase.http.HttpServer.initializeWebServer(HttpServer.java:546)
> at org.apache.hadoop.hbase.http.HttpServer.(HttpServer.java:500)
> at org.apache.hadoop.hbase.http.HttpServer.(HttpServer.java:104)
> at
> org.apache.hadoop.hbase.http.HttpServer$Builder.build(HttpServer.java:345)
> at org.apache.hadoop.hbase.http.InfoServer.(InfoServer.java:77)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.putUpWebUI(HRegionServer.java:1697)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:550)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:333)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:139)
> ... 26 more
> {noformat}
> The hadoop trunk removed {{MetricsServlet}} (HADOOP-12504).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15841) Performance Evaluation tool total rows may not be set correctly
[
https://issues.apache.org/jira/browse/HBASE-15841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285724#comment-15285724
]
Hadoop QA commented on HBASE-15841:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 53s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 8s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 44s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 44s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 9m
28s {color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1
2.5.2 2.6.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
20s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 103m 13s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 130m 39s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804271/HBASE-15841-master.patch
|
| JIRA Issue | HBASE-15841 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/test_framework/yetus-0.2.1/lib/precommit/personality/hbase.sh
|
| git revision | master / c3223a5 |
| Default Java | 1.7.0_79 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/usr/local/jenkins/java/jdk1.7.0_79:1.7.0_79 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/1910/testReport/ |
| modules | C: hbase-server U:
[jira] [Commented] (HBASE-15615) Wrong sleep time when RegionServerCallable need retry
[
https://issues.apache.org/jira/browse/HBASE-15615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285718#comment-15285718
]
Hudson commented on HBASE-15615:
SUCCESS: Integrated in HBase-1.1-JDK7 #1716 (See
[https://builds.apache.org/job/HBase-1.1-JDK7/1716/])
HBASE-15615 Wrong sleep time when RegionServerCallable need retry (antonov: rev
ce6f111a3aa811a6383d84606fe534008f204433)
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionServerCallable.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionAdminServiceCallable.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCaller.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestConnectionUtils.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionUtils.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
> Wrong sleep time when RegionServerCallable need retry
> -
>
> Key: HBASE-15615
> URL: https://issues.apache.org/jira/browse/HBASE-15615
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 1.0.0, 2.0.0, 1.1.0, 1.2.0, 1.3.0, 0.98.19
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Fix For: 1.3.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15615-branch-0.98.patch,
> HBASE-15615-branch-1-v3.patch, HBASE-15615-branch-1.0-v2.patch,
> HBASE-15615-branch-1.1-v2.patch, HBASE-15615-branch-1.1-v2.patch,
> HBASE-15615-branch-1.patch, HBASE-15615-v1.patch, HBASE-15615-v1.patch,
> HBASE-15615-v2.patch, HBASE-15615-v2.patch, HBASE-15615-v3.patch,
> HBASE-15615-v4.patch, HBASE-15615-v5.patch, HBASE-15615.patch
>
>
> In RpcRetryingCallerImpl, it get pause time by expectedSleep =
> callable.sleep(pause, tries + 1); And in RegionServerCallable, it get pasue
> time by sleep = ConnectionUtils.getPauseTime(pause, tries + 1). So tries will
> be bumped up twice. And the pasue time is 3 * hbase.client.pause when tries
> is 0.
> RETRY_BACKOFF = {1, 2, 3, 5, 10, 20, 40, 100, 100, 100, 100, 200, 200}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15824) LocalHBaseCluster gets bind exception in master info port
[
https://issues.apache.org/jira/browse/HBASE-15824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285719#comment-15285719
]
Hudson commented on HBASE-15824:
SUCCESS: Integrated in HBase-1.1-JDK7 #1716 (See
[https://builds.apache.org/job/HBase-1.1-JDK7/1716/])
HBASE-15824 LocalHBaseCluster gets bind exception in master info port (enis:
rev 46f4e142ebe86de3b51c44a9685915d9dda734d8)
* hbase-server/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java
> LocalHBaseCluster gets bind exception in master info port
> -
>
> Key: HBASE-15824
> URL: https://issues.apache.org/jira/browse/HBASE-15824
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: hbase-15824_v1.patch
>
>
> This is seen from Phoenix, since we already configure the ports in
> hbase-site.xml under hbase-server/src/test/resources.
> But this is also important for other {{hbase-testing-util}} consumers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15617) Canary in regionserver mode might not enumerate all regionservers
[ https://issues.apache.org/jira/browse/HBASE-15617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285716#comment-15285716 ] Andrew Purtell commented on HBASE-15617: Looks all good. Committing today. I'll port it back to all active branches. > Canary in regionserver mode might not enumerate all regionservers > - > > Key: HBASE-15617 > URL: https://issues.apache.org/jira/browse/HBASE-15617 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Andrew Purtell >Assignee: Sanjeev Lakshmanan >Priority: Minor > Labels: beginner > Attachments: HBASE-15617-v1.patch, HBASE-15617.patch > > > When running in regionserver mode the Canary is expected to probe for service > health one time per regionserver during a probe interval. > Each time the canary chore fires, we create a RegionServerMonitor, which uses > filterRegionServerByName (via getAllRegionServerByName) to enumerate over all > table descriptors, find the locations for each table, then assemble the list > of regionservers to probe from this result. The list may not contain all live > regionservers, if there is a regionserver up but for some reason not serving > any regions. To ensure we have the complete list of live regionservers I > think it would be better to use Admin#getClusterStatus and enumerate the live > server list returned in the result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15824) LocalHBaseCluster gets bind exception in master info port
[
https://issues.apache.org/jira/browse/HBASE-15824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285711#comment-15285711
]
Hudson commented on HBASE-15824:
FAILURE: Integrated in HBase-Trunk_matrix #922 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/922/])
HBASE-15824 LocalHBaseCluster gets bind exception in master info port (enis:
rev c3223a59ffc0f4c8b0f5c7bc452677ef05cdbc11)
* hbase-server/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java
> LocalHBaseCluster gets bind exception in master info port
> -
>
> Key: HBASE-15824
> URL: https://issues.apache.org/jira/browse/HBASE-15824
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: hbase-15824_v1.patch
>
>
> This is seen from Phoenix, since we already configure the ports in
> hbase-site.xml under hbase-server/src/test/resources.
> But this is also important for other {{hbase-testing-util}} consumers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15615) Wrong sleep time when RegionServerCallable need retry
[
https://issues.apache.org/jira/browse/HBASE-15615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285673#comment-15285673
]
Hudson commented on HBASE-15615:
FAILURE: Integrated in HBase-1.1-JDK8 #1802 (See
[https://builds.apache.org/job/HBase-1.1-JDK8/1802/])
HBASE-15615 Wrong sleep time when RegionServerCallable need retry (antonov: rev
ce6f111a3aa811a6383d84606fe534008f204433)
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCaller.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionUtils.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionAdminServiceCallable.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestConnectionUtils.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionServerCallable.java
> Wrong sleep time when RegionServerCallable need retry
> -
>
> Key: HBASE-15615
> URL: https://issues.apache.org/jira/browse/HBASE-15615
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 1.0.0, 2.0.0, 1.1.0, 1.2.0, 1.3.0, 0.98.19
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Fix For: 1.3.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15615-branch-0.98.patch,
> HBASE-15615-branch-1-v3.patch, HBASE-15615-branch-1.0-v2.patch,
> HBASE-15615-branch-1.1-v2.patch, HBASE-15615-branch-1.1-v2.patch,
> HBASE-15615-branch-1.patch, HBASE-15615-v1.patch, HBASE-15615-v1.patch,
> HBASE-15615-v2.patch, HBASE-15615-v2.patch, HBASE-15615-v3.patch,
> HBASE-15615-v4.patch, HBASE-15615-v5.patch, HBASE-15615.patch
>
>
> In RpcRetryingCallerImpl, it get pause time by expectedSleep =
> callable.sleep(pause, tries + 1); And in RegionServerCallable, it get pasue
> time by sleep = ConnectionUtils.getPauseTime(pause, tries + 1). So tries will
> be bumped up twice. And the pasue time is 3 * hbase.client.pause when tries
> is 0.
> RETRY_BACKOFF = {1, 2, 3, 5, 10, 20, 40, 100, 100, 100, 100, 200, 200}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15824) LocalHBaseCluster gets bind exception in master info port
[
https://issues.apache.org/jira/browse/HBASE-15824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285674#comment-15285674
]
Hudson commented on HBASE-15824:
FAILURE: Integrated in HBase-1.1-JDK8 #1802 (See
[https://builds.apache.org/job/HBase-1.1-JDK8/1802/])
HBASE-15824 LocalHBaseCluster gets bind exception in master info port (enis:
rev 46f4e142ebe86de3b51c44a9685915d9dda734d8)
* hbase-server/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java
> LocalHBaseCluster gets bind exception in master info port
> -
>
> Key: HBASE-15824
> URL: https://issues.apache.org/jira/browse/HBASE-15824
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: hbase-15824_v1.patch
>
>
> This is seen from Phoenix, since we already configure the ports in
> hbase-site.xml under hbase-server/src/test/resources.
> But this is also important for other {{hbase-testing-util}} consumers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15842) SnapshotInfo should display ownership information
[
https://issues.apache.org/jira/browse/HBASE-15842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15842:
---
Hadoop Flags: Reviewed
Fix Version/s: 1.4.0
2.0.0
> SnapshotInfo should display ownership information
> -
>
> Key: HBASE-15842
> URL: https://issues.apache.org/jira/browse/HBASE-15842
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 15842.v1.txt, 15842.v1.txt
>
>
> Currently SnapshotInfo doesn't show snapshot owner:
> {code}
> hbase org.apache.hadoop.hbase.snapshot.SnapshotInfo -snapshot
> snapshot_table_qm0uxvk19x -stats -schema
> ...
> Snapshot Info
>
>Name: snapshot_table_qm0uxvk19x
>Type: FLUSH
> Table: table_qm0uxvk19x
> Format: 2
> Created: 2016-05-16T20:54:08
> ...
> {code}
> This JIRA is to add ownership information to the display.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15839) Track our flaky tests and use them to improve our build environment
[ https://issues.apache.org/jira/browse/HBASE-15839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-15839: - Attachment: Screen Shot 2016-05-16 at 4.02.46 PM.png !Screen Shot 2016-05-16 at 4.02.46 PM.png! > Track our flaky tests and use them to improve our build environment > --- > > Key: HBASE-15839 > URL: https://issues.apache.org/jira/browse/HBASE-15839 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy > Attachments: Screen Shot 2016-05-16 at 4.02.46 PM.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15842) SnapshotInfo should display ownership information
[
https://issues.apache.org/jira/browse/HBASE-15842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15842:
---
Attachment: 15842.v1.txt
Patch with added spaces.
> SnapshotInfo should display ownership information
> -
>
> Key: HBASE-15842
> URL: https://issues.apache.org/jira/browse/HBASE-15842
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15842.v1.txt, 15842.v1.txt
>
>
> Currently SnapshotInfo doesn't show snapshot owner:
> {code}
> hbase org.apache.hadoop.hbase.snapshot.SnapshotInfo -snapshot
> snapshot_table_qm0uxvk19x -stats -schema
> ...
> Snapshot Info
>
>Name: snapshot_table_qm0uxvk19x
>Type: FLUSH
> Table: table_qm0uxvk19x
> Format: 2
> Created: 2016-05-16T20:54:08
> ...
> {code}
> This JIRA is to add ownership information to the display.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15842) SnapshotInfo should display ownership information
[
https://issues.apache.org/jira/browse/HBASE-15842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285646#comment-15285646
]
Matteo Bertozzi commented on HBASE-15842:
-
+1, on commit add two spaces so everything is nicely aligned in the output :)
> SnapshotInfo should display ownership information
> -
>
> Key: HBASE-15842
> URL: https://issues.apache.org/jira/browse/HBASE-15842
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15842.v1.txt
>
>
> Currently SnapshotInfo doesn't show snapshot owner:
> {code}
> hbase org.apache.hadoop.hbase.snapshot.SnapshotInfo -snapshot
> snapshot_table_qm0uxvk19x -stats -schema
> ...
> Snapshot Info
>
>Name: snapshot_table_qm0uxvk19x
>Type: FLUSH
> Table: table_qm0uxvk19x
> Format: 2
> Created: 2016-05-16T20:54:08
> ...
> {code}
> This JIRA is to add ownership information to the display.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15824) LocalHBaseCluster gets bind exception in master info port
[
https://issues.apache.org/jira/browse/HBASE-15824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285633#comment-15285633
]
Hudson commented on HBASE-15824:
SUCCESS: Integrated in HBase-1.2 #626 (See
[https://builds.apache.org/job/HBase-1.2/626/])
HBASE-15824 LocalHBaseCluster gets bind exception in master info port (enis:
rev fa3b39d22f6384f0e93fc60d7fc6a9c0382355b6)
* hbase-server/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java
> LocalHBaseCluster gets bind exception in master info port
> -
>
> Key: HBASE-15824
> URL: https://issues.apache.org/jira/browse/HBASE-15824
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: hbase-15824_v1.patch
>
>
> This is seen from Phoenix, since we already configure the ports in
> hbase-site.xml under hbase-server/src/test/resources.
> But this is also important for other {{hbase-testing-util}} consumers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15842) SnapshotInfo should display ownership information
[
https://issues.apache.org/jira/browse/HBASE-15842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15842:
---
Attachment: 15842.v1.txt
> SnapshotInfo should display ownership information
> -
>
> Key: HBASE-15842
> URL: https://issues.apache.org/jira/browse/HBASE-15842
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15842.v1.txt
>
>
> Currently SnapshotInfo doesn't show snapshot owner:
> {code}
> hbase org.apache.hadoop.hbase.snapshot.SnapshotInfo -snapshot
> snapshot_table_qm0uxvk19x -stats -schema
> ...
> Snapshot Info
>
>Name: snapshot_table_qm0uxvk19x
>Type: FLUSH
> Table: table_qm0uxvk19x
> Format: 2
> Created: 2016-05-16T20:54:08
> ...
> {code}
> This JIRA is to add ownership information to the display.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15842) SnapshotInfo should display ownership information
[
https://issues.apache.org/jira/browse/HBASE-15842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15842:
---
Status: Patch Available (was: Open)
> SnapshotInfo should display ownership information
> -
>
> Key: HBASE-15842
> URL: https://issues.apache.org/jira/browse/HBASE-15842
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15842.v1.txt
>
>
> Currently SnapshotInfo doesn't show snapshot owner:
> {code}
> hbase org.apache.hadoop.hbase.snapshot.SnapshotInfo -snapshot
> snapshot_table_qm0uxvk19x -stats -schema
> ...
> Snapshot Info
>
>Name: snapshot_table_qm0uxvk19x
>Type: FLUSH
> Table: table_qm0uxvk19x
> Format: 2
> Created: 2016-05-16T20:54:08
> ...
> {code}
> This JIRA is to add ownership information to the display.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15615) Wrong sleep time when RegionServerCallable need retry
[
https://issues.apache.org/jira/browse/HBASE-15615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285632#comment-15285632
]
Hudson commented on HBASE-15615:
SUCCESS: Integrated in HBase-1.2 #626 (See
[https://builds.apache.org/job/HBase-1.2/626/])
HBASE-15615 Wrong sleep time when RegionServerCallable need retry (antonov: rev
7c0fc0d6c7eaee4e134f103a9ad1bc5e62d32421)
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionServerCallable.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCaller.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestConnectionUtils.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionAdminServiceCallable.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionUtils.java
> Wrong sleep time when RegionServerCallable need retry
> -
>
> Key: HBASE-15615
> URL: https://issues.apache.org/jira/browse/HBASE-15615
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 1.0.0, 2.0.0, 1.1.0, 1.2.0, 1.3.0, 0.98.19
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Fix For: 1.3.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15615-branch-0.98.patch,
> HBASE-15615-branch-1-v3.patch, HBASE-15615-branch-1.0-v2.patch,
> HBASE-15615-branch-1.1-v2.patch, HBASE-15615-branch-1.1-v2.patch,
> HBASE-15615-branch-1.patch, HBASE-15615-v1.patch, HBASE-15615-v1.patch,
> HBASE-15615-v2.patch, HBASE-15615-v2.patch, HBASE-15615-v3.patch,
> HBASE-15615-v4.patch, HBASE-15615-v5.patch, HBASE-15615.patch
>
>
> In RpcRetryingCallerImpl, it get pause time by expectedSleep =
> callable.sleep(pause, tries + 1); And in RegionServerCallable, it get pasue
> time by sleep = ConnectionUtils.getPauseTime(pause, tries + 1). So tries will
> be bumped up twice. And the pasue time is 3 * hbase.client.pause when tries
> is 0.
> RETRY_BACKOFF = {1, 2, 3, 5, 10, 20, 40, 100, 100, 100, 100, 200, 200}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15826) Clean up ASF license issues
[
https://issues.apache.org/jira/browse/HBASE-15826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285636#comment-15285636
]
Hadoop QA commented on HBASE-15826:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
52s {color} | {color:green} HBASE-14850 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m 26s
{color} | {color:green} HBASE-14850 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m 16s
{color} | {color:green} HBASE-14850 passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m 14s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 4m 14s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 30s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 3m 30s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
3s {color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
3s {color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
12m 38s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.1 2.5.2 2.6.0. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 174m 7s
{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
59s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 217m 33s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804234/HBASE-15826.v2.HBASE-14850.patch
|
| JIRA Issue | HBASE-15826 |
| Optional Tests | asflicense shellcheck shelldocs cc unit compile |
| uname | Linux pomona.apache.org 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT
Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/test_framework/yetus-0.2.1/lib/precommit/personality/hbase.sh
|
| git revision | HBASE-14850 / e002553 |
| shellcheck | v0.3.3 (This is an old version that has serious bugs. Consider
upgrading.) |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/1909/testReport/ |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/1909/console |
| Powered by | Apache Yetus 0.2.1 http://yetus.apache.org |
This message was automatically generated.
> Clean up ASF license issues
> ---
>
> Key: HBASE-15826
> URL: https://issues.apache.org/jira/browse/HBASE-15826
> Project: HBase
> Issue Type: Sub-task
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Attachments: HBASE-15826-HBASE-14850.patch,
> HBASE-15826.v2.HBASE-14850.patch
>
>
> Gotta make sure that everything is well licensed
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15842) SnapshotInfo should display ownership information
Ted Yu created HBASE-15842:
--
Summary: SnapshotInfo should display ownership information
Key: HBASE-15842
URL: https://issues.apache.org/jira/browse/HBASE-15842
Project: HBase
Issue Type: Improvement
Reporter: Ted Yu
Assignee: Ted Yu
Currently SnapshotInfo doesn't show snapshot owner:
{code}
hbase org.apache.hadoop.hbase.snapshot.SnapshotInfo -snapshot
snapshot_table_qm0uxvk19x -stats -schema
...
Snapshot Info
Name: snapshot_table_qm0uxvk19x
Type: FLUSH
Table: table_qm0uxvk19x
Format: 2
Created: 2016-05-16T20:54:08
...
{code}
This JIRA is to add ownership information to the display.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-15651) Add report-flakies.py to use jenkins api to get failing tests
[ https://issues.apache.org/jira/browse/HBASE-15651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-15651. -- Resolution: Fixed > Add report-flakies.py to use jenkins api to get failing tests > - > > Key: HBASE-15651 > URL: https://issues.apache.org/jira/browse/HBASE-15651 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Appy > Fix For: master > > Attachments: HBASE-15651-master-v2.patch, > HBASE-15651-master-v3.patch, HBASE-15651-master-v4.patch, > HBASE-15651-master.patch, flakies.py > > > So i have written this simple script (attached) which looks at history of the > [post commit build|https://builds.apache.org/view/All/job/HBase-Trunk_matrix] > and outputs a list of flaky tests with some numbers. > Next steps: > 1. Setup a jenkins job (say *find-flaky-tests*) to run this script daily. We > can either directly pull these results into other jobs using curl on this > job's artifacts, or commit the list of flaky test to repo (idk if it's > possible to commit something from jenkins job). > We'll collect results from both *post-commit* job (to add new flakies) and > *flaky-tests* job (to delete tests which are no more flaky). > 2. Change *pre-commit* and *post-commit* jobs to ignore these tests using > --exclude maven flag. Someone familiar with yetus might be able to do it > easily. > 3. Setup a new job (say *flaky-tests*) to run only these flaky tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15651) Add report-flakies.py to use jenkins api to get failing tests
[ https://issues.apache.org/jira/browse/HBASE-15651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-15651: --- Issue Type: Sub-task (was: New Feature) Parent: HBASE-15839 > Add report-flakies.py to use jenkins api to get failing tests > - > > Key: HBASE-15651 > URL: https://issues.apache.org/jira/browse/HBASE-15651 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Appy > Fix For: master > > Attachments: HBASE-15651-master-v2.patch, > HBASE-15651-master-v3.patch, HBASE-15651-master-v4.patch, > HBASE-15651-master.patch, flakies.py > > > So i have written this simple script (attached) which looks at history of the > [post commit build|https://builds.apache.org/view/All/job/HBase-Trunk_matrix] > and outputs a list of flaky tests with some numbers. > Next steps: > 1. Setup a jenkins job (say *find-flaky-tests*) to run this script daily. We > can either directly pull these results into other jobs using curl on this > job's artifacts, or commit the list of flaky test to repo (idk if it's > possible to commit something from jenkins job). > We'll collect results from both *post-commit* job (to add new flakies) and > *flaky-tests* job (to delete tests which are no more flaky). > 2. Change *pre-commit* and *post-commit* jobs to ignore these tests using > --exclude maven flag. Someone familiar with yetus might be able to do it > easily. > 3. Setup a new job (say *flaky-tests*) to run only these flaky tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15651) Add report-flakies.py to use jenkins api to get failing tests
[ https://issues.apache.org/jira/browse/HBASE-15651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-15651: --- Summary: Add report-flakies.py to use jenkins api to get failing tests (was: Track our flaky tests and use them to improve our build environment) > Add report-flakies.py to use jenkins api to get failing tests > - > > Key: HBASE-15651 > URL: https://issues.apache.org/jira/browse/HBASE-15651 > Project: HBase > Issue Type: New Feature >Reporter: Appy >Assignee: Appy > Fix For: master > > Attachments: HBASE-15651-master-v2.patch, > HBASE-15651-master-v3.patch, HBASE-15651-master-v4.patch, > HBASE-15651-master.patch, flakies.py > > > So i have written this simple script (attached) which looks at history of the > [post commit build|https://builds.apache.org/view/All/job/HBase-Trunk_matrix] > and outputs a list of flaky tests with some numbers. > Next steps: > 1. Setup a jenkins job (say *find-flaky-tests*) to run this script daily. We > can either directly pull these results into other jobs using curl on this > job's artifacts, or commit the list of flaky test to repo (idk if it's > possible to commit something from jenkins job). > We'll collect results from both *post-commit* job (to add new flakies) and > *flaky-tests* job (to delete tests which are no more flaky). > 2. Change *pre-commit* and *post-commit* jobs to ignore these tests using > --exclude maven flag. Someone familiar with yetus might be able to do it > easily. > 3. Setup a new job (say *flaky-tests*) to run only these flaky tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15824) LocalHBaseCluster gets bind exception in master info port
[
https://issues.apache.org/jira/browse/HBASE-15824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285620#comment-15285620
]
Hudson commented on HBASE-15824:
SUCCESS: Integrated in HBase-1.4 #156 (See
[https://builds.apache.org/job/HBase-1.4/156/])
HBASE-15824 LocalHBaseCluster gets bind exception in master info port (enis:
rev c96227eed1df9398b816534a5c462b27c8375a69)
* hbase-server/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java
> LocalHBaseCluster gets bind exception in master info port
> -
>
> Key: HBASE-15824
> URL: https://issues.apache.org/jira/browse/HBASE-15824
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: hbase-15824_v1.patch
>
>
> This is seen from Phoenix, since we already configure the ports in
> hbase-site.xml under hbase-server/src/test/resources.
> But this is also important for other {{hbase-testing-util}} consumers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15824) LocalHBaseCluster gets bind exception in master info port
[
https://issues.apache.org/jira/browse/HBASE-15824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285583#comment-15285583
]
Hudson commented on HBASE-15824:
FAILURE: Integrated in HBase-1.3 #700 (See
[https://builds.apache.org/job/HBase-1.3/700/])
HBASE-15824 LocalHBaseCluster gets bind exception in master info port (enis:
rev cbcc8c34b9f3d9ce82dd96f5809b705c752b4fc3)
* hbase-server/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java
> LocalHBaseCluster gets bind exception in master info port
> -
>
> Key: HBASE-15824
> URL: https://issues.apache.org/jira/browse/HBASE-15824
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: hbase-15824_v1.patch
>
>
> This is seen from Phoenix, since we already configure the ports in
> hbase-site.xml under hbase-server/src/test/resources.
> But this is also important for other {{hbase-testing-util}} consumers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15236) Inconsistent cell reads over multiple bulk-loaded HFiles
[
https://issues.apache.org/jira/browse/HBASE-15236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-15236:
-
Attachment: HBASE-15236-branch-1.2-v2.patch
if we choose to backport, here's v2 patch for branch-1.2 which fixes the issue
mentioned in last comment.
> Inconsistent cell reads over multiple bulk-loaded HFiles
>
>
> Key: HBASE-15236
> URL: https://issues.apache.org/jira/browse/HBASE-15236
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-15236-branch-1-v1.patch,
> HBASE-15236-branch-1.2-v1.patch, HBASE-15236-branch-1.2-v2.patch,
> HBASE-15236-branch-1.3-v1.patch, HBASE-15236-master-v2.patch,
> HBASE-15236-master-v3.patch, HBASE-15236-master-v4.patch,
> HBASE-15236-master-v5.patch, HBASE-15236-master-v6.patch,
> HBASE-15236-master-v7.patch, HBASE-15236-master-v8.patch,
> HBASE-15236-master-v9.patch, HBASE-15236.patch, TestWithSingleHRegion.java,
> tables_data.zip
>
>
> If there are two bulkloaded hfiles in a region with same seqID, same
> timestamps and duplicate keys*, get and scan may return different values for
> a key. Not sure how this would happen, but one of our customer uploaded a
> dataset with 2 files in a single region and both having same bulk load
> timestamp. These files are small ~50M (I couldn't find any setting for max
> file size that could lead to 2 files). The range of keys in two hfiles are
> overlapping to some extent, but not fully (so the two files are because of
> region merge).
> In such a case, depending on file sizes (used in
> StoreFile.Comparators.SEQ_ID), we may get different values for the same cell
> (say "r", "cf:50") depending on what we call: get "r" "cf:50" or get "r"
> "cf:".
> To replicate this, i ran ImportTsv twice with 2 different csv files but same
> timestamp (-Dimporttsv.timestamp=1). Collected two resulting hfiles in a
> single directory and loaded with LoadIncrementalHFiles. Here are the commands.
> {noformat}
> //CSV files should be in hdfs:///user/hbase/tmp
> sudo -u hbase hbase org.apache.hadoop.hbase.mapreduce.ImportTsv
> -Dimporttsv.columns=HBASE_ROW_KEY,c:1,c:2,c:3,c:4,c:5,c:6
> -Dimporttsv.separator='|' -Dimporttsv.timestamp=1
> -Dimporttsv.bulk.output=hdfs:///user/hbase/1 t tmp
> {noformat}
> tables_data.zip contains three tables: t, t2, and t3.
> To load these tables and test with them, use the attached
> TestWithSingleHRegion.java. (Change ROOT_DIR and TABLE_NAME variables
> appropriately)
> Hfiles for table 't':
> 1) 07922ac90208445883b2ddc89c424c89 : length = 1094: KVs = (c:5, 55) (c:6, 66)
> 2) 143137fa4fa84625b4e8d1d84a494f08 : length = 1192: KVs = (c:1, 1) (c:2, 2)
> (c:3, 3) (c:4, 4) (c:5, 5) (c:6, 6)
> Get returns 5 where as Scan returns 55.
> On the other hand if I make the first hfile, the one with values 55 and 66
> bigger in size than second hfile, then:
> Get returns 55 where as Scan returns 55.
> This can be seen in table 't2'.
> Weird things:
> 1. Get consistently returns values from larger hfile.
> 2. Scan consistently returns 55.
> Reason:
> Explanation 1: We add StoreFileScanners to heap by iterating over list of
> store files which is kept sorted in DefaultStoreFileManger using
> StoreFile.Comparators.SEQ_ID. It sorts the file first by increasing seq id,
> then by decreasing filesize. So our larger files sizes are always in the
> start.
> Explanation 2: In KeyValueHeap.next(), if both current and this.heap.peek()
> scanners (type is StoreFileScanner in this case) point to KVs which compare
> as equal, we put back the current scanner into heap, and call pollRealKV() to
> get new one. While inserting, KVScannerComparator is used which compares the
> two (one already in heap and the one being inserted) as equal and inserts it
> after the existing one. \[1\]
> Say s1 is scanner over first hfile and s2 over second.
> 1. We scan with s2 to get values 1, 2, 3, 4
> 2. see that both scanners have c:5 as next key, put current scanner s2 back
> in heap, take s1 out
> 4. return 55, advance s1 to key c:6
> 5. compare c:6 to c:5, put back s2 in heap since it has larger key, take out
> s1
> 6. ignore it's value (there is code for that too), advance s2 to c:6
> Go back to step 2 with both scanners having c:6 as next key
> This is wrong because if we add a key between c:4 and c:5 to first hfile, say
> (c:44, foo) which makes s1 as the 'current' scanner when we hit step 2, then
> we'll get the values - 1, 2, 3, 4, foo, 5, 6.
> (try table t3)
> Fix:
> Assign priority ids to StoreFileScanners when inserting them into heap,
> latest hfiles get higher priority, and use them in KVComparator instead of
> just seq id.
> \[1\]
>
[jira] [Updated] (HBASE-15841) Performance Evaluation tool total rows may not be set correctly
[ https://issues.apache.org/jira/browse/HBASE-15841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry He updated HBASE-15841: - Assignee: Jerry He Fix Version/s: 1.2.2 1.4.0 1.3.0 2.0.0 Status: Patch Available (was: Open) > Performance Evaluation tool total rows may not be set correctly > --- > > Key: HBASE-15841 > URL: https://issues.apache.org/jira/browse/HBASE-15841 > Project: HBase > Issue Type: Bug >Reporter: Jerry He >Assignee: Jerry He >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2 > > Attachments: HBASE-15841-branch-1.patch, HBASE-15841-master.patch > > > Carried my comment on HBASE-15403 to here: > Recently when I ran PerformanceEvaluation, I did notice some problem with the > number of rows. > hbase org.apache.hadoop.hbase.PerformanceEvaluation --table=TestTable1 > randomWrite 1 > hbase org.apache.hadoop.hbase.PerformanceEvaluation --table=TestTable5 > randomWrite 5 > hbase org.apache.hadoop.hbase.PerformanceEvaluation --table=TestTable10 > randomWrite 10 > hbase org.apache.hadoop.hbase.PerformanceEvaluation --table=TestTable10 > randomWrite 20 > All produced similar number of rows, and on the file system, they look like > in similar size as well: > hadoop fs -du -h /apps/hbase/data/data/default > 786.5 M /apps/hbase/data/data/default/TestTable1 > 786.0 M /apps/hbase/data/data/default/TestTable10 > 782.0 M /apps/hbase/data/data/default/TestTable20 > 713.4 M /apps/hbase/data/data/default/TestTable5 > HBase is 1.2.0. Looks like a regression somewhere. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15841) Performance Evaluation tool total rows may not be set correctly
[ https://issues.apache.org/jira/browse/HBASE-15841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry He updated HBASE-15841: - Attachment: HBASE-15841-master.patch master branch is slightly different. > Performance Evaluation tool total rows may not be set correctly > --- > > Key: HBASE-15841 > URL: https://issues.apache.org/jira/browse/HBASE-15841 > Project: HBase > Issue Type: Bug >Reporter: Jerry He >Priority: Minor > Attachments: HBASE-15841-branch-1.patch, HBASE-15841-master.patch > > > Carried my comment on HBASE-15403 to here: > Recently when I ran PerformanceEvaluation, I did notice some problem with the > number of rows. > hbase org.apache.hadoop.hbase.PerformanceEvaluation --table=TestTable1 > randomWrite 1 > hbase org.apache.hadoop.hbase.PerformanceEvaluation --table=TestTable5 > randomWrite 5 > hbase org.apache.hadoop.hbase.PerformanceEvaluation --table=TestTable10 > randomWrite 10 > hbase org.apache.hadoop.hbase.PerformanceEvaluation --table=TestTable10 > randomWrite 20 > All produced similar number of rows, and on the file system, they look like > in similar size as well: > hadoop fs -du -h /apps/hbase/data/data/default > 786.5 M /apps/hbase/data/data/default/TestTable1 > 786.0 M /apps/hbase/data/data/default/TestTable10 > 782.0 M /apps/hbase/data/data/default/TestTable20 > 713.4 M /apps/hbase/data/data/default/TestTable5 > HBase is 1.2.0. Looks like a regression somewhere. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15835) HBaseTestingUtility#startMiniCluster throws "HMasterAddress already in use" RuntimeException when a local instance of HBase is running
[
https://issues.apache.org/jira/browse/HBASE-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285524#comment-15285524
]
Sean Busbey commented on HBASE-15835:
-
you should be able to assign to yourself now.
> HBaseTestingUtility#startMiniCluster throws "HMasterAddress already in use"
> RuntimeException when a local instance of HBase is running
> --
>
> Key: HBASE-15835
> URL: https://issues.apache.org/jira/browse/HBASE-15835
> Project: HBase
> Issue Type: Bug
> Components: API
>Affects Versions: 2.0.0
>Reporter: Daniel Vimont
> Labels: easyfix
> Fix For: 2.0.0
>
>
> When a MiniCluster is being started with the
> {{HBaseTestUtility#startMiniCluster}} method (most typically in the context
> of JUnit testing), if a local HBase instance is already running (or for that
> matter, another thread with another MiniCluster is already running), the
> startup will fail with a RuntimeException saying "HMasterAddress already in
> use", referring explicitly to contention for the same default master info
> port (16010).
> This problem most recently came up in conjunction with HBASE-14876 and its
> sub-JIRAs (development of new HBase-oriented Maven archetypes), but this is
> apparently a known issue to veteran developers, who tend to set up the
> @BeforeClass sections of their test modules with code similar to the
> following:
> {code}
> UTIL = HBaseTestingUtility.createLocalHTU();
> // disable UI's on test cluster.
> UTIL.getConfiguration().setInt("hbase.master.info.port", -1);
> UTIL.getConfiguration().setInt("hbase.regionserver.info.port", -1);
> UTIL.startMiniCluster();
> {code}
> A comprehensive solution modeled on this should be put directly into
> HBaseTestUtility's main constructor, using one of the following options:
> OPTION 1 (always force random port assignment):
> {code}
> this.getConfiguration().setInt(HConstants.MASTER_INFO_PORT, -1);
> this.getConfiguration().setInt(HConstants.REGIONSERVER_PORT, -1);
> {code}
> OPTION 2 (always force random port assignment if user has not explicitly
> defined alternate port):
> {code}
> Configuration conf = this.getConfiguration();
> if (conf.getInt(HConstants.MASTER_INFO_PORT,
> HConstants.DEFAULT_MASTER_INFOPORT)
> == HConstants.DEFAULT_MASTER_INFOPORT) {
> conf.setInt(HConstants.MASTER_INFO_PORT, -1);
> }
> if (conf.getInt(HConstants.REGIONSERVER_PORT,
> HConstants.DEFAULT_REGIONSERVER_PORT)
> == HConstants.DEFAULT_REGIONSERVER_PORT) {
> conf.setInt(HConstants.REGIONSERVER_PORT, -1);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14140) HBase Backup/Restore Phase 3: Enhance HBaseAdmin API to include backup/restore - related API
[ https://issues.apache.org/jira/browse/HBASE-14140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14140: -- Attachment: HBASE-14140-v4.patch The latest patch. > HBase Backup/Restore Phase 3: Enhance HBaseAdmin API to include > backup/restore - related API > > > Key: HBASE-14140 > URL: https://issues.apache.org/jira/browse/HBASE-14140 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-14140-v1.patch, HBASE-14140-v4.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14140) HBase Backup/Restore Phase 3: Enhance HBaseAdmin API to include backup/restore - related API
[
https://issues.apache.org/jira/browse/HBASE-14140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285502#comment-15285502
]
Vladimir Rodionov commented on HBASE-14140:
---
{quote}
backupTablesAsync() and backupTables() were dropped from Admin interface.
{quote}
Yes, but implementation is still in HBaseAdmin and it is not easy to move it to
HBaseBackupAdmin. If you want this refactoring to be done, please open JIRA in
Phase 3.
> HBase Backup/Restore Phase 3: Enhance HBaseAdmin API to include
> backup/restore - related API
>
>
> Key: HBASE-14140
> URL: https://issues.apache.org/jira/browse/HBASE-14140
> Project: HBase
> Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-14140-v1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14140) HBase Backup/Restore Phase 3: Enhance HBaseAdmin API to include backup/restore - related API
[ https://issues.apache.org/jira/browse/HBASE-14140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14140: -- Status: Open (was: Patch Available) > HBase Backup/Restore Phase 3: Enhance HBaseAdmin API to include > backup/restore - related API > > > Key: HBASE-14140 > URL: https://issues.apache.org/jira/browse/HBASE-14140 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-14140-v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15841) Performance Evaluation tool total rows may not be set correctly
[ https://issues.apache.org/jira/browse/HBASE-15841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry He updated HBASE-15841: - Attachment: HBASE-15841-branch-1.patch > Performance Evaluation tool total rows may not be set correctly > --- > > Key: HBASE-15841 > URL: https://issues.apache.org/jira/browse/HBASE-15841 > Project: HBase > Issue Type: Bug >Reporter: Jerry He >Priority: Minor > Attachments: HBASE-15841-branch-1.patch > > > Carried my comment on HBASE-15403 to here: > Recently when I ran PerformanceEvaluation, I did notice some problem with the > number of rows. > hbase org.apache.hadoop.hbase.PerformanceEvaluation --table=TestTable1 > randomWrite 1 > hbase org.apache.hadoop.hbase.PerformanceEvaluation --table=TestTable5 > randomWrite 5 > hbase org.apache.hadoop.hbase.PerformanceEvaluation --table=TestTable10 > randomWrite 10 > hbase org.apache.hadoop.hbase.PerformanceEvaluation --table=TestTable10 > randomWrite 20 > All produced similar number of rows, and on the file system, they look like > in similar size as well: > hadoop fs -du -h /apps/hbase/data/data/default > 786.5 M /apps/hbase/data/data/default/TestTable1 > 786.0 M /apps/hbase/data/data/default/TestTable10 > 782.0 M /apps/hbase/data/data/default/TestTable20 > 713.4 M /apps/hbase/data/data/default/TestTable5 > HBase is 1.2.0. Looks like a regression somewhere. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15836) The -set parameter in "hbase backup" should be in the help text
[
https://issues.apache.org/jira/browse/HBASE-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15836:
---
Labels: backup (was: )
> The -set parameter in "hbase backup" should be in the help text
> ---
>
> Key: HBASE-15836
> URL: https://issues.apache.org/jira/browse/HBASE-15836
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Labels: backup
> Attachments: 15836.v1.patch, 15836.v2.patch
>
>
> [~cartershanklin] noticed the following:
> {code}
> $ hbase backup create
> ERROR: wrong number of arguments
> Usage: hbase backup create [tables] [-s name]
> [-convert] [-silent] [-w workers][-b bandwith]
> type "full" to create a full backup image;
>"incremental" to create an incremental backup image
> backup_root_path The full root path to store the backup image,
> the prefix can be hdfs, webhdfs or gpfs
> Options:
> tables If no tables ("") are specified, all tables are backed up.
> Otherwise it is a
>comma separated list of tables.
> -s name Use the specified snapshot for full backup
> -convertFor an incremental backup, convert WAL files to HFiles
> -w number of parallel workers.
> -b bandwith per one worker (in MB sec)
> {code}
> A working backup command may look like the following (assuming the set named
> green exists):
> {code}
> sudo -u hbase hbase backup create full hdfs://ip:8020/tmp/backup -set green
> {code}
> Help text for backup command should be updated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15836) The -set parameter in "hbase backup" should be in the help text
[
https://issues.apache.org/jira/browse/HBASE-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15836:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks for the review, Vlad.
> The -set parameter in "hbase backup" should be in the help text
> ---
>
> Key: HBASE-15836
> URL: https://issues.apache.org/jira/browse/HBASE-15836
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Labels: backup
> Attachments: 15836.v1.patch, 15836.v2.patch
>
>
> [~cartershanklin] noticed the following:
> {code}
> $ hbase backup create
> ERROR: wrong number of arguments
> Usage: hbase backup create [tables] [-s name]
> [-convert] [-silent] [-w workers][-b bandwith]
> type "full" to create a full backup image;
>"incremental" to create an incremental backup image
> backup_root_path The full root path to store the backup image,
> the prefix can be hdfs, webhdfs or gpfs
> Options:
> tables If no tables ("") are specified, all tables are backed up.
> Otherwise it is a
>comma separated list of tables.
> -s name Use the specified snapshot for full backup
> -convertFor an incremental backup, convert WAL files to HFiles
> -w number of parallel workers.
> -b bandwith per one worker (in MB sec)
> {code}
> A working backup command may look like the following (assuming the set named
> green exists):
> {code}
> sudo -u hbase hbase backup create full hdfs://ip:8020/tmp/backup -set green
> {code}
> Help text for backup command should be updated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15765) RPC handlers should steal from other queues in the same or higher priority
[
https://issues.apache.org/jira/browse/HBASE-15765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285496#comment-15285496
]
Enis Soztutar commented on HBASE-15765:
---
[~chenheng] with this patch, the handlers still do not steal from the queues in
the same tier, but a different "shard" (the config controlled by
{{hbase.ipc.server.callqueue.handler.factor}} ). The reason we have these
shards is that, the single blocking queue becomes the bottleneck for 10+
readers a producers and 100+ handlers as consumers. I believe previous perf
tests by others proved that having sharding here helps a lot.
Plus, we can steal 1 RPC every ~3seconds if I read the patch right. This will
not have any affect on latency tail smoothing other than potentially working
around deadlock-type cases a la HBASE-15613.
I am not sure whether we can find a good data structure to give us a highly
concurrent multi-priority queue, where every handler type can only take from
its queue, or higher priority ones.
> RPC handlers should steal from other queues in the same or higher priority
> --
>
> Key: HBASE-15765
> URL: https://issues.apache.org/jira/browse/HBASE-15765
> Project: HBase
> Issue Type: Improvement
>Reporter: Enis Soztutar
> Attachments: HBASE-15765.patch, HBASE-15765_v1.patch
>
>
> With the simple rpc scheduler model by default, a request first falls in one
> type of queue (priority, general, replication) and it will get "sharded" in a
> queue within that priority level (for example there are 2 queues for the high
> priority level). The handlers can only take from their specific queues, and
> nothing else.
> Even if a request is waiting in a high priority queue, the handlers within
> the same priority level or a lower priority level will not see these requests
> (which is the root cause for HBASE-15613). We should be able to steal from
> other queues, without losing the prioritization and sharding benefits.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15836) The -set parameter in "hbase backup" should be in the help text
[
https://issues.apache.org/jira/browse/HBASE-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285489#comment-15285489
]
Vladimir Rodionov commented on HBASE-15836:
---
Good to go. Thanks, Ted.
> The -set parameter in "hbase backup" should be in the help text
> ---
>
> Key: HBASE-15836
> URL: https://issues.apache.org/jira/browse/HBASE-15836
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15836.v1.patch, 15836.v2.patch
>
>
> [~cartershanklin] noticed the following:
> {code}
> $ hbase backup create
> ERROR: wrong number of arguments
> Usage: hbase backup create [tables] [-s name]
> [-convert] [-silent] [-w workers][-b bandwith]
> type "full" to create a full backup image;
>"incremental" to create an incremental backup image
> backup_root_path The full root path to store the backup image,
> the prefix can be hdfs, webhdfs or gpfs
> Options:
> tables If no tables ("") are specified, all tables are backed up.
> Otherwise it is a
>comma separated list of tables.
> -s name Use the specified snapshot for full backup
> -convertFor an incremental backup, convert WAL files to HFiles
> -w number of parallel workers.
> -b bandwith per one worker (in MB sec)
> {code}
> A working backup command may look like the following (assuming the set named
> green exists):
> {code}
> sudo -u hbase hbase backup create full hdfs://ip:8020/tmp/backup -set green
> {code}
> Help text for backup command should be updated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15841) Performance Evaluation tool total rows may not be set correctly
Jerry He created HBASE-15841: Summary: Performance Evaluation tool total rows may not be set correctly Key: HBASE-15841 URL: https://issues.apache.org/jira/browse/HBASE-15841 Project: HBase Issue Type: Bug Reporter: Jerry He Priority: Minor Carried my comment on HBASE-15403 to here: Recently when I ran PerformanceEvaluation, I did notice some problem with the number of rows. hbase org.apache.hadoop.hbase.PerformanceEvaluation --table=TestTable1 randomWrite 1 hbase org.apache.hadoop.hbase.PerformanceEvaluation --table=TestTable5 randomWrite 5 hbase org.apache.hadoop.hbase.PerformanceEvaluation --table=TestTable10 randomWrite 10 hbase org.apache.hadoop.hbase.PerformanceEvaluation --table=TestTable10 randomWrite 20 All produced similar number of rows, and on the file system, they look like in similar size as well: hadoop fs -du -h /apps/hbase/data/data/default 786.5 M /apps/hbase/data/data/default/TestTable1 786.0 M /apps/hbase/data/data/default/TestTable10 782.0 M /apps/hbase/data/data/default/TestTable20 713.4 M /apps/hbase/data/data/default/TestTable5 HBase is 1.2.0. Looks like a regression somewhere. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15651) Track our flaky tests and use them to improve our build environment
[ https://issues.apache.org/jira/browse/HBASE-15651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-15651: - Attachment: (was: hbase-personality-master.patch) > Track our flaky tests and use them to improve our build environment > --- > > Key: HBASE-15651 > URL: https://issues.apache.org/jira/browse/HBASE-15651 > Project: HBase > Issue Type: New Feature >Reporter: Appy >Assignee: Appy > Fix For: master > > Attachments: HBASE-15651-master-v2.patch, > HBASE-15651-master-v3.patch, HBASE-15651-master-v4.patch, > HBASE-15651-master.patch, flakies.py > > > So i have written this simple script (attached) which looks at history of the > [post commit build|https://builds.apache.org/view/All/job/HBase-Trunk_matrix] > and outputs a list of flaky tests with some numbers. > Next steps: > 1. Setup a jenkins job (say *find-flaky-tests*) to run this script daily. We > can either directly pull these results into other jobs using curl on this > job's artifacts, or commit the list of flaky test to repo (idk if it's > possible to commit something from jenkins job). > We'll collect results from both *post-commit* job (to add new flakies) and > *flaky-tests* job (to delete tests which are no more flaky). > 2. Change *pre-commit* and *post-commit* jobs to ignore these tests using > --exclude maven flag. Someone familiar with yetus might be able to do it > easily. > 3. Setup a new job (say *flaky-tests*) to run only these flaky tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (HBASE-15651) Track our flaky tests and use them to improve our build environment
[ https://issues.apache.org/jira/browse/HBASE-15651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy reopened HBASE-15651: -- Reopening to edit title and re-organize the jiras related to flaky work together. > Track our flaky tests and use them to improve our build environment > --- > > Key: HBASE-15651 > URL: https://issues.apache.org/jira/browse/HBASE-15651 > Project: HBase > Issue Type: New Feature >Reporter: Appy >Assignee: Appy > Fix For: master > > Attachments: HBASE-15651-master-v2.patch, > HBASE-15651-master-v3.patch, HBASE-15651-master-v4.patch, > HBASE-15651-master.patch, flakies.py, hbase-personality-master.patch > > > So i have written this simple script (attached) which looks at history of the > [post commit build|https://builds.apache.org/view/All/job/HBase-Trunk_matrix] > and outputs a list of flaky tests with some numbers. > Next steps: > 1. Setup a jenkins job (say *find-flaky-tests*) to run this script daily. We > can either directly pull these results into other jobs using curl on this > job's artifacts, or commit the list of flaky test to repo (idk if it's > possible to commit something from jenkins job). > We'll collect results from both *post-commit* job (to add new flakies) and > *flaky-tests* job (to delete tests which are no more flaky). > 2. Change *pre-commit* and *post-commit* jobs to ignore these tests using > --exclude maven flag. Someone familiar with yetus might be able to do it > easily. > 3. Setup a new job (say *flaky-tests*) to run only these flaky tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15840) WAL.proto compilation broken for cpp
Elliott Clark created HBASE-15840:
-
Summary: WAL.proto compilation broken for cpp
Key: HBASE-15840
URL: https://issues.apache.org/jira/browse/HBASE-15840
Project: HBase
Issue Type: Bug
Reporter: Elliott Clark
Assignee: Elliott Clark
HBASE-15669 seems to have broken generating code for WAL.proto in cpp.
{code}
In file included from buck-out/gen/if/WAL.pb.cc/WAL.pb.cc:5:0:
buck-out/gen/if/WAL.pb.cc/WAL.pb.h:1103:37: error: 'google::protobuf::uint64
hbase::pb::StoreDescriptor::store_file_size() const' cannot be overloaded
inline ::google::protobuf::uint64 store_file_size() const;
^
buck-out/gen/if/WAL.pb.cc/WAL.pb.h:1084:14: error: with 'int
hbase::pb::StoreDescriptor::store_file_size() const'
inline int store_file_size() const;
^
In file included from buck-out/gen/if/WAL.pb.cc/WAL.pb.cc:5:0:
buck-out/gen/if/WAL.pb.cc/WAL.pb.h:3527:35: error: prototype for
'google::protobuf::uint64 hbase::pb::StoreDescriptor::store_file_size() const'
does not match any in class 'hbase::pb::StoreDescriptor'
inline ::google::protobuf::uint64 StoreDescriptor::store_file_size() const {
^
buck-out/gen/if/WAL.pb.cc/WAL.pb.h:3460:12: error: candidate is: int
hbase::pb::StoreDescriptor::store_file_size() const
inline int StoreDescriptor::store_file_size() const {
^
BUILD FAILED: //if:if#compile-WAL.pb.cc.oa0460dda,default failed with exit code
1:
c++-cpp-output compile
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15651) Track our flaky tests and use them to improve our build environment
[ https://issues.apache.org/jira/browse/HBASE-15651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-15651: - Issue Type: New Feature (was: Task) > Track our flaky tests and use them to improve our build environment > --- > > Key: HBASE-15651 > URL: https://issues.apache.org/jira/browse/HBASE-15651 > Project: HBase > Issue Type: New Feature >Reporter: Appy >Assignee: Appy > Fix For: master > > Attachments: HBASE-15651-master-v2.patch, > HBASE-15651-master-v3.patch, HBASE-15651-master-v4.patch, > HBASE-15651-master.patch, flakies.py, hbase-personality-master.patch > > > So i have written this simple script (attached) which looks at history of the > [post commit build|https://builds.apache.org/view/All/job/HBase-Trunk_matrix] > and outputs a list of flaky tests with some numbers. > Next steps: > 1. Setup a jenkins job (say *find-flaky-tests*) to run this script daily. We > can either directly pull these results into other jobs using curl on this > job's artifacts, or commit the list of flaky test to repo (idk if it's > possible to commit something from jenkins job). > We'll collect results from both *post-commit* job (to add new flakies) and > *flaky-tests* job (to delete tests which are no more flaky). > 2. Change *pre-commit* and *post-commit* jobs to ignore these tests using > --exclude maven flag. Someone familiar with yetus might be able to do it > easily. > 3. Setup a new job (say *flaky-tests*) to run only these flaky tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15689) Changes to hbase-personality.sh to include/exclude flaky tests
[ https://issues.apache.org/jira/browse/HBASE-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-15689: - Parent Issue: HBASE-15839 (was: HBASE-15651) > Changes to hbase-personality.sh to include/exclude flaky tests > -- > > Key: HBASE-15689 > URL: https://issues.apache.org/jira/browse/HBASE-15689 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Appy > Fix For: 2.0.0 > > Attachments: HBASE-15689-master-v2.patch, > HBASE-15689-master-v3.patch, HBASE-15689-master.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15839) Track our flaky tests and use them to improve our build environment
Appy created HBASE-15839: Summary: Track our flaky tests and use them to improve our build environment Key: HBASE-15839 URL: https://issues.apache.org/jira/browse/HBASE-15839 Project: HBase Issue Type: Improvement Reporter: Appy Assignee: Appy -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15236) Inconsistent cell reads over multiple bulk-loaded HFiles
[
https://issues.apache.org/jira/browse/HBASE-15236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285291#comment-15285291
]
Appy commented on HBASE-15236:
--
TL;DR: Don't push branch-1.2-v1. Some tests will start failing.
Reason for the tests failing in 1.2 but not in other branches.
So we sort the store files
[here|https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java#L140].
But sometimes the readers in these store files may be null (for instance,
private readers for compactions) and we'll get NPE when the comparator will try
to get store file size using null reader.
Doesn't happen in 1.3+ because of [this else
condition|https://github.com/apache/hbase/blob/064c3c09ec4ee1641691aee43156bbc3736c53b3/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java#L1707].
> Inconsistent cell reads over multiple bulk-loaded HFiles
>
>
> Key: HBASE-15236
> URL: https://issues.apache.org/jira/browse/HBASE-15236
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-15236-branch-1-v1.patch,
> HBASE-15236-branch-1.2-v1.patch, HBASE-15236-branch-1.3-v1.patch,
> HBASE-15236-master-v2.patch, HBASE-15236-master-v3.patch,
> HBASE-15236-master-v4.patch, HBASE-15236-master-v5.patch,
> HBASE-15236-master-v6.patch, HBASE-15236-master-v7.patch,
> HBASE-15236-master-v8.patch, HBASE-15236-master-v9.patch, HBASE-15236.patch,
> TestWithSingleHRegion.java, tables_data.zip
>
>
> If there are two bulkloaded hfiles in a region with same seqID, same
> timestamps and duplicate keys*, get and scan may return different values for
> a key. Not sure how this would happen, but one of our customer uploaded a
> dataset with 2 files in a single region and both having same bulk load
> timestamp. These files are small ~50M (I couldn't find any setting for max
> file size that could lead to 2 files). The range of keys in two hfiles are
> overlapping to some extent, but not fully (so the two files are because of
> region merge).
> In such a case, depending on file sizes (used in
> StoreFile.Comparators.SEQ_ID), we may get different values for the same cell
> (say "r", "cf:50") depending on what we call: get "r" "cf:50" or get "r"
> "cf:".
> To replicate this, i ran ImportTsv twice with 2 different csv files but same
> timestamp (-Dimporttsv.timestamp=1). Collected two resulting hfiles in a
> single directory and loaded with LoadIncrementalHFiles. Here are the commands.
> {noformat}
> //CSV files should be in hdfs:///user/hbase/tmp
> sudo -u hbase hbase org.apache.hadoop.hbase.mapreduce.ImportTsv
> -Dimporttsv.columns=HBASE_ROW_KEY,c:1,c:2,c:3,c:4,c:5,c:6
> -Dimporttsv.separator='|' -Dimporttsv.timestamp=1
> -Dimporttsv.bulk.output=hdfs:///user/hbase/1 t tmp
> {noformat}
> tables_data.zip contains three tables: t, t2, and t3.
> To load these tables and test with them, use the attached
> TestWithSingleHRegion.java. (Change ROOT_DIR and TABLE_NAME variables
> appropriately)
> Hfiles for table 't':
> 1) 07922ac90208445883b2ddc89c424c89 : length = 1094: KVs = (c:5, 55) (c:6, 66)
> 2) 143137fa4fa84625b4e8d1d84a494f08 : length = 1192: KVs = (c:1, 1) (c:2, 2)
> (c:3, 3) (c:4, 4) (c:5, 5) (c:6, 6)
> Get returns 5 where as Scan returns 55.
> On the other hand if I make the first hfile, the one with values 55 and 66
> bigger in size than second hfile, then:
> Get returns 55 where as Scan returns 55.
> This can be seen in table 't2'.
> Weird things:
> 1. Get consistently returns values from larger hfile.
> 2. Scan consistently returns 55.
> Reason:
> Explanation 1: We add StoreFileScanners to heap by iterating over list of
> store files which is kept sorted in DefaultStoreFileManger using
> StoreFile.Comparators.SEQ_ID. It sorts the file first by increasing seq id,
> then by decreasing filesize. So our larger files sizes are always in the
> start.
> Explanation 2: In KeyValueHeap.next(), if both current and this.heap.peek()
> scanners (type is StoreFileScanner in this case) point to KVs which compare
> as equal, we put back the current scanner into heap, and call pollRealKV() to
> get new one. While inserting, KVScannerComparator is used which compares the
> two (one already in heap and the one being inserted) as equal and inserts it
> after the existing one. \[1\]
> Say s1 is scanner over first hfile and s2 over second.
> 1. We scan with s2 to get values 1, 2, 3, 4
> 2. see that both scanners have c:5 as next key, put current scanner s2 back
> in heap, take s1 out
> 4. return 55, advance s1 to key c:6
> 5. compare c:6 to c:5, put back s2 in heap since it has larger key, take out
> s1
> 6. ignore it's value (there is code for that too), advance s2
[jira] [Commented] (HBASE-15403) Performance Evaluation tool isn't working as expected
[ https://issues.apache.org/jira/browse/HBASE-15403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285273#comment-15285273 ] Jerry He commented on HBASE-15403: -- The total row count is not being updated correctly. Will open another JIRA to address it. > Performance Evaluation tool isn't working as expected > - > > Key: HBASE-15403 > URL: https://issues.apache.org/jira/browse/HBASE-15403 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 1.2.0 >Reporter: Appy >Priority: Critical > > hbase pe --nomapred --rows=100 --table='t4' randomWrite 10 > # count on t4 gives 620 rows > hbase pe --nomapred --rows=200 --table='t5' randomWrite 10 > # count on t5 gives 1257 rows > hbase pe --nomapred --table='t6' --rows=200 randomWrite 1 > # count on t6 gives 126 rows > I was working with 1.2.0, but it's likely that it'll also be affecting master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-3727) MultiHFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285262#comment-15285262 ] NIDHI GAMBHIR commented on HBASE-3727: -- I have not seen any activity on this jira. is it fine if I pick it up? > MultiHFileOutputFormat > -- > > Key: HBASE-3727 > URL: https://issues.apache.org/jira/browse/HBASE-3727 > Project: HBase > Issue Type: New Feature >Reporter: Andrew Purtell >Priority: Minor > Attachments: MultiHFileOutputFormat.java, MultiHFileOutputFormat.java > > > Like MultiTableOutputFormat, but outputting HFiles. Key is tablename as an > IBW. Creates sub-writers (code cut and pasted from HFileOutputFormat) on > demand that produce HFiles in per-table subdirectories of the configured > output path. Does not currently support partitioning for existing tables / > incremental update. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15838) Fix broken external links across hbase.apache.org website and docs
Misty Stanley-Jones created HBASE-15838: --- Summary: Fix broken external links across hbase.apache.org website and docs Key: HBASE-15838 URL: https://issues.apache.org/jira/browse/HBASE-15838 Project: HBase Issue Type: Bug Reporter: Misty Stanley-Jones See https://builds.apache.org/job/HBase%20Website%20Link%20Ckecker/43/artifact/link_report/urlfailX.html. One hint is that all the links that reference www.cloudera.com/blog should be changed to blog.cloudera.com/blog. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15615) Wrong sleep time when RegionServerCallable need retry
[
https://issues.apache.org/jira/browse/HBASE-15615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285176#comment-15285176
]
Hudson commented on HBASE-15615:
SUCCESS: Integrated in HBase-1.2-IT #508 (See
[https://builds.apache.org/job/HBase-1.2-IT/508/])
HBASE-15615 Wrong sleep time when RegionServerCallable need retry (antonov: rev
7c0fc0d6c7eaee4e134f103a9ad1bc5e62d32421)
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCaller.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestConnectionUtils.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionUtils.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionAdminServiceCallable.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionServerCallable.java
> Wrong sleep time when RegionServerCallable need retry
> -
>
> Key: HBASE-15615
> URL: https://issues.apache.org/jira/browse/HBASE-15615
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 1.0.0, 2.0.0, 1.1.0, 1.2.0, 1.3.0, 0.98.19
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Fix For: 1.3.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15615-branch-0.98.patch,
> HBASE-15615-branch-1-v3.patch, HBASE-15615-branch-1.0-v2.patch,
> HBASE-15615-branch-1.1-v2.patch, HBASE-15615-branch-1.1-v2.patch,
> HBASE-15615-branch-1.patch, HBASE-15615-v1.patch, HBASE-15615-v1.patch,
> HBASE-15615-v2.patch, HBASE-15615-v2.patch, HBASE-15615-v3.patch,
> HBASE-15615-v4.patch, HBASE-15615-v5.patch, HBASE-15615.patch
>
>
> In RpcRetryingCallerImpl, it get pause time by expectedSleep =
> callable.sleep(pause, tries + 1); And in RegionServerCallable, it get pasue
> time by sleep = ConnectionUtils.getPauseTime(pause, tries + 1). So tries will
> be bumped up twice. And the pasue time is 3 * hbase.client.pause when tries
> is 0.
> RETRY_BACKOFF = {1, 2, 3, 5, 10, 20, 40, 100, 100, 100, 100, 200, 200}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15824) LocalHBaseCluster gets bind exception in master info port
[
https://issues.apache.org/jira/browse/HBASE-15824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285177#comment-15285177
]
Hudson commented on HBASE-15824:
SUCCESS: Integrated in HBase-1.2-IT #508 (See
[https://builds.apache.org/job/HBase-1.2-IT/508/])
HBASE-15824 LocalHBaseCluster gets bind exception in master info port (enis:
rev fa3b39d22f6384f0e93fc60d7fc6a9c0382355b6)
* hbase-server/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java
> LocalHBaseCluster gets bind exception in master info port
> -
>
> Key: HBASE-15824
> URL: https://issues.apache.org/jira/browse/HBASE-15824
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: hbase-15824_v1.patch
>
>
> This is seen from Phoenix, since we already configure the ports in
> hbase-site.xml under hbase-server/src/test/resources.
> But this is also important for other {{hbase-testing-util}} consumers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15725) make_patch.sh should add the branch name when -b is passed.
[ https://issues.apache.org/jira/browse/HBASE-15725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285137#comment-15285137 ] Elliott Clark commented on HBASE-15725: --- Trying it out in HBASE-15826 I'll commit if that run pick up all the information correctly. > make_patch.sh should add the branch name when -b is passed. > --- > > Key: HBASE-15725 > URL: https://issues.apache.org/jira/browse/HBASE-15725 > Project: HBase > Issue Type: Bug > Components: tooling >Affects Versions: 2.0.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0 > > Attachments: HBASE-15725-v4.patch, HBASE-15725-v5.patch, > HBASE-15725.v6-addendum.patch > > > When using something other than master as the base branch we should default > to adding the branch name to the patch file name. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15615) Wrong sleep time when RegionServerCallable need retry
[
https://issues.apache.org/jira/browse/HBASE-15615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-15615:
Fix Version/s: 1.1.6
> Wrong sleep time when RegionServerCallable need retry
> -
>
> Key: HBASE-15615
> URL: https://issues.apache.org/jira/browse/HBASE-15615
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 1.0.0, 2.0.0, 1.1.0, 1.2.0, 1.3.0, 0.98.19
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Fix For: 1.3.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15615-branch-0.98.patch,
> HBASE-15615-branch-1-v3.patch, HBASE-15615-branch-1.0-v2.patch,
> HBASE-15615-branch-1.1-v2.patch, HBASE-15615-branch-1.1-v2.patch,
> HBASE-15615-branch-1.patch, HBASE-15615-v1.patch, HBASE-15615-v1.patch,
> HBASE-15615-v2.patch, HBASE-15615-v2.patch, HBASE-15615-v3.patch,
> HBASE-15615-v4.patch, HBASE-15615-v5.patch, HBASE-15615.patch
>
>
> In RpcRetryingCallerImpl, it get pause time by expectedSleep =
> callable.sleep(pause, tries + 1); And in RegionServerCallable, it get pasue
> time by sleep = ConnectionUtils.getPauseTime(pause, tries + 1). So tries will
> be bumped up twice. And the pasue time is 3 * hbase.client.pause when tries
> is 0.
> RETRY_BACKOFF = {1, 2, 3, 5, 10, 20, 40, 100, 100, 100, 100, 200, 200}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15615) Wrong sleep time when RegionServerCallable need retry
[
https://issues.apache.org/jira/browse/HBASE-15615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285136#comment-15285136
]
Mikhail Antonov commented on HBASE-15615:
-
All right, pushed to branch-1.1.
> Wrong sleep time when RegionServerCallable need retry
> -
>
> Key: HBASE-15615
> URL: https://issues.apache.org/jira/browse/HBASE-15615
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 1.0.0, 2.0.0, 1.1.0, 1.2.0, 1.3.0, 0.98.19
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Fix For: 1.3.0, 1.2.2
>
> Attachments: HBASE-15615-branch-0.98.patch,
> HBASE-15615-branch-1-v3.patch, HBASE-15615-branch-1.0-v2.patch,
> HBASE-15615-branch-1.1-v2.patch, HBASE-15615-branch-1.1-v2.patch,
> HBASE-15615-branch-1.patch, HBASE-15615-v1.patch, HBASE-15615-v1.patch,
> HBASE-15615-v2.patch, HBASE-15615-v2.patch, HBASE-15615-v3.patch,
> HBASE-15615-v4.patch, HBASE-15615-v5.patch, HBASE-15615.patch
>
>
> In RpcRetryingCallerImpl, it get pause time by expectedSleep =
> callable.sleep(pause, tries + 1); And in RegionServerCallable, it get pasue
> time by sleep = ConnectionUtils.getPauseTime(pause, tries + 1). So tries will
> be bumped up twice. And the pasue time is 3 * hbase.client.pause when tries
> is 0.
> RETRY_BACKOFF = {1, 2, 3, 5, 10, 20, 40, 100, 100, 100, 100, 200, 200}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15825) Fix the null pointer in DynamicLogicExpressionSuite
[ https://issues.apache.org/jira/browse/HBASE-15825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285134#comment-15285134 ] Hudson commented on HBASE-15825: FAILURE: Integrated in HBase-Trunk_matrix #921 (See [https://builds.apache.org/job/HBase-Trunk_matrix/921/]) HBASE-15825 Fix the null pointer in DynamicLogicExpressionSuite (Zhan (tedyu: rev 084b036cb2abc19d13262ff8d5e2ef2f451e4bf3) * hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DynamicLogicExpressionSuite.scala > Fix the null pointer in DynamicLogicExpressionSuite > --- > > Key: HBASE-15825 > URL: https://issues.apache.org/jira/browse/HBASE-15825 > Project: HBase > Issue Type: Sub-task >Reporter: Zhan Zhang >Assignee: Zhan Zhang > Fix For: 2.0.0 > > Attachments: HBASE-15825-1.patch > > > It only happens in test cases. Not sure why it is not caught. Will submit > patch soon -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15826) Clean up ASF license issues
[ https://issues.apache.org/jira/browse/HBASE-15826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-15826: -- Attachment: HBASE-15826.v2.HBASE-14850.patch > Clean up ASF license issues > --- > > Key: HBASE-15826 > URL: https://issues.apache.org/jira/browse/HBASE-15826 > Project: HBase > Issue Type: Sub-task >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-15826-HBASE-14850.patch, > HBASE-15826.v2.HBASE-14850.patch > > > Gotta make sure that everything is well licensed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (HBASE-13195) IntegrationTestImportTsv doesn't support Chaos Monkey options
[ https://issues.apache.org/jira/browse/HBASE-13195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-13195 started by Dima Spivak. --- > IntegrationTestImportTsv doesn't support Chaos Monkey options > - > > Key: HBASE-13195 > URL: https://issues.apache.org/jira/browse/HBASE-13195 > Project: HBase > Issue Type: Bug > Components: integration tests >Reporter: Dima Spivak >Assignee: Dima Spivak > > IntegrationTestImportTsv comes from a pre-monkey era and needs some simple > changes to support primative behaviors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15615) Wrong sleep time when RegionServerCallable need retry
[
https://issues.apache.org/jira/browse/HBASE-15615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285119#comment-15285119
]
Nick Dimiduk commented on HBASE-15615:
--
Yes please in 1.1. Thanks [~mantonov]
> Wrong sleep time when RegionServerCallable need retry
> -
>
> Key: HBASE-15615
> URL: https://issues.apache.org/jira/browse/HBASE-15615
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 1.0.0, 2.0.0, 1.1.0, 1.2.0, 1.3.0, 0.98.19
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Fix For: 1.3.0, 1.2.2
>
> Attachments: HBASE-15615-branch-0.98.patch,
> HBASE-15615-branch-1-v3.patch, HBASE-15615-branch-1.0-v2.patch,
> HBASE-15615-branch-1.1-v2.patch, HBASE-15615-branch-1.1-v2.patch,
> HBASE-15615-branch-1.patch, HBASE-15615-v1.patch, HBASE-15615-v1.patch,
> HBASE-15615-v2.patch, HBASE-15615-v2.patch, HBASE-15615-v3.patch,
> HBASE-15615-v4.patch, HBASE-15615-v5.patch, HBASE-15615.patch
>
>
> In RpcRetryingCallerImpl, it get pause time by expectedSleep =
> callable.sleep(pause, tries + 1); And in RegionServerCallable, it get pasue
> time by sleep = ConnectionUtils.getPauseTime(pause, tries + 1). So tries will
> be bumped up twice. And the pasue time is 3 * hbase.client.pause when tries
> is 0.
> RETRY_BACKOFF = {1, 2, 3, 5, 10, 20, 40, 100, 100, 100, 100, 200, 200}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15725) make_patch.sh should add the branch name when -b is passed.
[
https://issues.apache.org/jira/browse/HBASE-15725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285101#comment-15285101
]
Hadoop QA commented on HBASE-15725:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
3s {color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
2s {color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 9s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.1 2.5.2 2.6.0. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 38s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804230/HBASE-15725.v6-addendum.patch
|
| JIRA Issue | HBASE-15725 |
| Optional Tests | asflicense shellcheck shelldocs |
| uname | Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/test_framework/yetus-0.2.1/lib/precommit/personality/hbase.sh
|
| git revision | master / c3223a5 |
| shellcheck | v0.3.3 (This is an old version that has serious bugs. Consider
upgrading.) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/1908/console |
| Powered by | Apache Yetus 0.2.1 http://yetus.apache.org |
This message was automatically generated.
> make_patch.sh should add the branch name when -b is passed.
> ---
>
> Key: HBASE-15725
> URL: https://issues.apache.org/jira/browse/HBASE-15725
> Project: HBase
> Issue Type: Bug
> Components: tooling
>Affects Versions: 2.0.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0
>
> Attachments: HBASE-15725-v4.patch, HBASE-15725-v5.patch,
> HBASE-15725.v6-addendum.patch
>
>
> When using something other than master as the base branch we should default
> to adding the branch name to the patch file name.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15725) make_patch.sh should add the branch name when -b is passed.
[ https://issues.apache.org/jira/browse/HBASE-15725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285094#comment-15285094 ] Sean Busbey commented on HBASE-15725: - sounds like a variant YETUS-274, maybe? If doing "." separated parts for branches with "-" in the name works, then +1 from me. > make_patch.sh should add the branch name when -b is passed. > --- > > Key: HBASE-15725 > URL: https://issues.apache.org/jira/browse/HBASE-15725 > Project: HBase > Issue Type: Bug > Components: tooling >Affects Versions: 2.0.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0 > > Attachments: HBASE-15725-v4.patch, HBASE-15725-v5.patch, > HBASE-15725.v6-addendum.patch > > > When using something other than master as the base branch we should default > to adding the branch name to the patch file name. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15837) More gracefully handle a negative memstoreSize
[ https://issues.apache.org/jira/browse/HBASE-15837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-15837: --- Attachment: HBASE-15837.001.patch .001 A first stab at avoiding the RS crash. The general goals are to # Determine who screwed up the memstoreSize in the first place # Avoid data loss when memstoreSize is wrong If a store does fail to flush successfully, the RS should still crash. The logic is just fixing the logic so that memstoreSize being negative doesn't prevent a Store's flush and cause the RS abort. > More gracefully handle a negative memstoreSize > -- > > Key: HBASE-15837 > URL: https://issues.apache.org/jira/browse/HBASE-15837 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: 2.0.0 > > Attachments: HBASE-15837.001.patch > > > Over in PHOENIX-2883, I've been trying to figure out how to track down the > root cause of an issue we were seeing where a negative memstoreSize was > ultimately causing an RS to abort. The tl;dr version is > * Something causes memstoreSize to be negative (not sure what is doing this > yet) > * All subsequent flushes short-circuit and don't run because they think there > is no data to flush > * The region is eventually closed (commonly, for a move). > * A final flush is attempted on each store before closing (which also > short-circuit for the same reason), leaving unflushed data in each store. > * The sanity check that each store's size is zero fails and the RS aborts. > I have a little patch which I think should improve our failure case around > this, preventing the RS abort safely (forcing a flush when memstoreSize is > negative) and logging a calltrace when an update to memstoreSize make it > negative (to find culprits in the future). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (HBASE-15837) More gracefully handle a negative memstoreSize
[ https://issues.apache.org/jira/browse/HBASE-15837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-15837 started by Josh Elser. -- > More gracefully handle a negative memstoreSize > -- > > Key: HBASE-15837 > URL: https://issues.apache.org/jira/browse/HBASE-15837 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: 2.0.0 > > Attachments: HBASE-15837.001.patch > > > Over in PHOENIX-2883, I've been trying to figure out how to track down the > root cause of an issue we were seeing where a negative memstoreSize was > ultimately causing an RS to abort. The tl;dr version is > * Something causes memstoreSize to be negative (not sure what is doing this > yet) > * All subsequent flushes short-circuit and don't run because they think there > is no data to flush > * The region is eventually closed (commonly, for a move). > * A final flush is attempted on each store before closing (which also > short-circuit for the same reason), leaving unflushed data in each store. > * The sanity check that each store's size is zero fails and the RS aborts. > I have a little patch which I think should improve our failure case around > this, preventing the RS abort safely (forcing a flush when memstoreSize is > negative) and logging a calltrace when an update to memstoreSize make it > negative (to find culprits in the future). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15837) More gracefully handle a negative memstoreSize
[
https://issues.apache.org/jira/browse/HBASE-15837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285082#comment-15285082
]
Josh Elser commented on HBASE-15837:
bq. Crashing when holding data that's unexpected seems like the correct thing
to do
Without looking at the code, I would have agreed with you; however, after
taking a look at how it's written I think it's just bad accounting. The check
is written to verify that the flush that we tried to run after grabbing the
writeLock actually ran successfully (e.g. there should be no chance that any
more data exists). The fact that we're using {{memstoreSize}} as the judge of
whether or not to run actually run the flush, but then checking the size of
each Store seems goofy as well (leading us to this split on the truth).
Given that coprocessors could be loaded which could unintentionally mess things
up (not to mention internal bugs), forcing down the RS seems very invasive to
me. I'm attaching a patch once I finish typing this -- let me know what you
think. IMO, this feels pretty safe to me given that we know we're controlling
all access to the region at this point in time.
> More gracefully handle a negative memstoreSize
> --
>
> Key: HBASE-15837
> URL: https://issues.apache.org/jira/browse/HBASE-15837
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: 2.0.0
>
>
> Over in PHOENIX-2883, I've been trying to figure out how to track down the
> root cause of an issue we were seeing where a negative memstoreSize was
> ultimately causing an RS to abort. The tl;dr version is
> * Something causes memstoreSize to be negative (not sure what is doing this
> yet)
> * All subsequent flushes short-circuit and don't run because they think there
> is no data to flush
> * The region is eventually closed (commonly, for a move).
> * A final flush is attempted on each store before closing (which also
> short-circuit for the same reason), leaving unflushed data in each store.
> * The sanity check that each store's size is zero fails and the RS aborts.
> I have a little patch which I think should improve our failure case around
> this, preventing the RS abort safely (forcing a flush when memstoreSize is
> negative) and logging a calltrace when an update to memstoreSize make it
> negative (to find culprits in the future).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15824) LocalHBaseCluster gets bind exception in master info port
[
https://issues.apache.org/jira/browse/HBASE-15824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285079#comment-15285079
]
Hudson commented on HBASE-15824:
SUCCESS: Integrated in HBase-1.3-IT #663 (See
[https://builds.apache.org/job/HBase-1.3-IT/663/])
HBASE-15824 LocalHBaseCluster gets bind exception in master info port (enis:
rev cbcc8c34b9f3d9ce82dd96f5809b705c752b4fc3)
* hbase-server/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java
> LocalHBaseCluster gets bind exception in master info port
> -
>
> Key: HBASE-15824
> URL: https://issues.apache.org/jira/browse/HBASE-15824
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: hbase-15824_v1.patch
>
>
> This is seen from Phoenix, since we already configure the ports in
> hbase-site.xml under hbase-server/src/test/resources.
> But this is also important for other {{hbase-testing-util}} consumers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15725) make_patch.sh should add the branch name when -b is passed.
[ https://issues.apache.org/jira/browse/HBASE-15725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-15725: -- Attachment: HBASE-15725.v6-addendum.patch Looks like yetus doesn't like dashes. > make_patch.sh should add the branch name when -b is passed. > --- > > Key: HBASE-15725 > URL: https://issues.apache.org/jira/browse/HBASE-15725 > Project: HBase > Issue Type: Bug > Components: tooling >Affects Versions: 2.0.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0 > > Attachments: HBASE-15725-v4.patch, HBASE-15725-v5.patch, > HBASE-15725.v6-addendum.patch > > > When using something other than master as the base branch we should default > to adding the branch name to the patch file name. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15837) More gracefully handle a negative memstoreSize
[ https://issues.apache.org/jira/browse/HBASE-15837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285049#comment-15285049 ] Elliott Clark commented on HBASE-15837: --- Crashing when holding data that's unexpected seems like the correct thing to do. If a process continues past something wholly un-expected it's more likely to get into worse state than it started in. > More gracefully handle a negative memstoreSize > -- > > Key: HBASE-15837 > URL: https://issues.apache.org/jira/browse/HBASE-15837 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: 2.0.0 > > > Over in PHOENIX-2883, I've been trying to figure out how to track down the > root cause of an issue we were seeing where a negative memstoreSize was > ultimately causing an RS to abort. The tl;dr version is > * Something causes memstoreSize to be negative (not sure what is doing this > yet) > * All subsequent flushes short-circuit and don't run because they think there > is no data to flush > * The region is eventually closed (commonly, for a move). > * A final flush is attempted on each store before closing (which also > short-circuit for the same reason), leaving unflushed data in each store. > * The sanity check that each store's size is zero fails and the RS aborts. > I have a little patch which I think should improve our failure case around > this, preventing the RS abort safely (forcing a flush when memstoreSize is > negative) and logging a calltrace when an update to memstoreSize make it > negative (to find culprits in the future). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15837) More gracefully handle a negative memstoreSize
Josh Elser created HBASE-15837: -- Summary: More gracefully handle a negative memstoreSize Key: HBASE-15837 URL: https://issues.apache.org/jira/browse/HBASE-15837 Project: HBase Issue Type: Improvement Components: regionserver Reporter: Josh Elser Assignee: Josh Elser Fix For: 2.0.0 Over in PHOENIX-2883, I've been trying to figure out how to track down the root cause of an issue we were seeing where a negative memstoreSize was ultimately causing an RS to abort. The tl;dr version is * Something causes memstoreSize to be negative (not sure what is doing this yet) * All subsequent flushes short-circuit and don't run because they think there is no data to flush * The region is eventually closed (commonly, for a move). * A final flush is attempted on each store before closing (which also short-circuit for the same reason), leaving unflushed data in each store. * The sanity check that each store's size is zero fails and the RS aborts. I have a little patch which I think should improve our failure case around this, preventing the RS abort safely (forcing a flush when memstoreSize is negative) and logging a calltrace when an update to memstoreSize make it negative (to find culprits in the future). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15615) Wrong sleep time when RegionServerCallable need retry
[
https://issues.apache.org/jira/browse/HBASE-15615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284967#comment-15284967
]
Mikhail Antonov commented on HBASE-15615:
-
[~busbey] all right, pushed to branch-1.2.
> Wrong sleep time when RegionServerCallable need retry
> -
>
> Key: HBASE-15615
> URL: https://issues.apache.org/jira/browse/HBASE-15615
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 1.0.0, 2.0.0, 1.1.0, 1.2.0, 1.3.0, 0.98.19
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Fix For: 1.3.0, 1.2.2
>
> Attachments: HBASE-15615-branch-0.98.patch,
> HBASE-15615-branch-1-v3.patch, HBASE-15615-branch-1.0-v2.patch,
> HBASE-15615-branch-1.1-v2.patch, HBASE-15615-branch-1.1-v2.patch,
> HBASE-15615-branch-1.patch, HBASE-15615-v1.patch, HBASE-15615-v1.patch,
> HBASE-15615-v2.patch, HBASE-15615-v2.patch, HBASE-15615-v3.patch,
> HBASE-15615-v4.patch, HBASE-15615-v5.patch, HBASE-15615.patch
>
>
> In RpcRetryingCallerImpl, it get pause time by expectedSleep =
> callable.sleep(pause, tries + 1); And in RegionServerCallable, it get pasue
> time by sleep = ConnectionUtils.getPauseTime(pause, tries + 1). So tries will
> be bumped up twice. And the pasue time is 3 * hbase.client.pause when tries
> is 0.
> RETRY_BACKOFF = {1, 2, 3, 5, 10, 20, 40, 100, 100, 100, 100, 200, 200}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15615) Wrong sleep time when RegionServerCallable need retry
[
https://issues.apache.org/jira/browse/HBASE-15615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-15615:
Fix Version/s: 1.2.2
> Wrong sleep time when RegionServerCallable need retry
> -
>
> Key: HBASE-15615
> URL: https://issues.apache.org/jira/browse/HBASE-15615
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 1.0.0, 2.0.0, 1.1.0, 1.2.0, 1.3.0, 0.98.19
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Fix For: 1.3.0, 1.2.2
>
> Attachments: HBASE-15615-branch-0.98.patch,
> HBASE-15615-branch-1-v3.patch, HBASE-15615-branch-1.0-v2.patch,
> HBASE-15615-branch-1.1-v2.patch, HBASE-15615-branch-1.1-v2.patch,
> HBASE-15615-branch-1.patch, HBASE-15615-v1.patch, HBASE-15615-v1.patch,
> HBASE-15615-v2.patch, HBASE-15615-v2.patch, HBASE-15615-v3.patch,
> HBASE-15615-v4.patch, HBASE-15615-v5.patch, HBASE-15615.patch
>
>
> In RpcRetryingCallerImpl, it get pause time by expectedSleep =
> callable.sleep(pause, tries + 1); And in RegionServerCallable, it get pasue
> time by sleep = ConnectionUtils.getPauseTime(pause, tries + 1). So tries will
> be bumped up twice. And the pasue time is 3 * hbase.client.pause when tries
> is 0.
> RETRY_BACKOFF = {1, 2, 3, 5, 10, 20, 40, 100, 100, 100, 100, 200, 200}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15836) The -set parameter in "hbase backup" should be in the help text
[
https://issues.apache.org/jira/browse/HBASE-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284963#comment-15284963
]
Hadoop QA commented on HBASE-15836:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color}
| {color:red} HBASE-15836 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.2.1/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804218/15836.v2.patch |
| JIRA Issue | HBASE-15836 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/1907/console |
| Powered by | Apache Yetus 0.2.1 http://yetus.apache.org |
This message was automatically generated.
> The -set parameter in "hbase backup" should be in the help text
> ---
>
> Key: HBASE-15836
> URL: https://issues.apache.org/jira/browse/HBASE-15836
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15836.v1.patch, 15836.v2.patch
>
>
> [~cartershanklin] noticed the following:
> {code}
> $ hbase backup create
> ERROR: wrong number of arguments
> Usage: hbase backup create [tables] [-s name]
> [-convert] [-silent] [-w workers][-b bandwith]
> type "full" to create a full backup image;
>"incremental" to create an incremental backup image
> backup_root_path The full root path to store the backup image,
> the prefix can be hdfs, webhdfs or gpfs
> Options:
> tables If no tables ("") are specified, all tables are backed up.
> Otherwise it is a
>comma separated list of tables.
> -s name Use the specified snapshot for full backup
> -convertFor an incremental backup, convert WAL files to HFiles
> -w number of parallel workers.
> -b bandwith per one worker (in MB sec)
> {code}
> A working backup command may look like the following (assuming the set named
> green exists):
> {code}
> sudo -u hbase hbase backup create full hdfs://ip:8020/tmp/backup -set green
> {code}
> Help text for backup command should be updated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15836) The -set parameter in "hbase backup" should be in the help text
[
https://issues.apache.org/jira/browse/HBASE-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15836:
---
Status: Patch Available (was: Open)
> The -set parameter in "hbase backup" should be in the help text
> ---
>
> Key: HBASE-15836
> URL: https://issues.apache.org/jira/browse/HBASE-15836
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15836.v1.patch, 15836.v2.patch
>
>
> [~cartershanklin] noticed the following:
> {code}
> $ hbase backup create
> ERROR: wrong number of arguments
> Usage: hbase backup create [tables] [-s name]
> [-convert] [-silent] [-w workers][-b bandwith]
> type "full" to create a full backup image;
>"incremental" to create an incremental backup image
> backup_root_path The full root path to store the backup image,
> the prefix can be hdfs, webhdfs or gpfs
> Options:
> tables If no tables ("") are specified, all tables are backed up.
> Otherwise it is a
>comma separated list of tables.
> -s name Use the specified snapshot for full backup
> -convertFor an incremental backup, convert WAL files to HFiles
> -w number of parallel workers.
> -b bandwith per one worker (in MB sec)
> {code}
> A working backup command may look like the following (assuming the set named
> green exists):
> {code}
> sudo -u hbase hbase backup create full hdfs://ip:8020/tmp/backup -set green
> {code}
> Help text for backup command should be updated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15836) The -set parameter in "hbase backup" should be in the help text
[
https://issues.apache.org/jira/browse/HBASE-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15836:
---
Attachment: 15836.v2.patch
> The -set parameter in "hbase backup" should be in the help text
> ---
>
> Key: HBASE-15836
> URL: https://issues.apache.org/jira/browse/HBASE-15836
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15836.v1.patch, 15836.v2.patch
>
>
> [~cartershanklin] noticed the following:
> {code}
> $ hbase backup create
> ERROR: wrong number of arguments
> Usage: hbase backup create [tables] [-s name]
> [-convert] [-silent] [-w workers][-b bandwith]
> type "full" to create a full backup image;
>"incremental" to create an incremental backup image
> backup_root_path The full root path to store the backup image,
> the prefix can be hdfs, webhdfs or gpfs
> Options:
> tables If no tables ("") are specified, all tables are backed up.
> Otherwise it is a
>comma separated list of tables.
> -s name Use the specified snapshot for full backup
> -convertFor an incremental backup, convert WAL files to HFiles
> -w number of parallel workers.
> -b bandwith per one worker (in MB sec)
> {code}
> A working backup command may look like the following (assuming the set named
> green exists):
> {code}
> sudo -u hbase hbase backup create full hdfs://ip:8020/tmp/backup -set green
> {code}
> Help text for backup command should be updated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15824) LocalHBaseCluster gets bind exception in master info port
[
https://issues.apache.org/jira/browse/HBASE-15824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-15824:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
I've pushed this. Thanks for review.
> LocalHBaseCluster gets bind exception in master info port
> -
>
> Key: HBASE-15824
> URL: https://issues.apache.org/jira/browse/HBASE-15824
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: hbase-15824_v1.patch
>
>
> This is seen from Phoenix, since we already configure the ports in
> hbase-site.xml under hbase-server/src/test/resources.
> But this is also important for other {{hbase-testing-util}} consumers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15816) Provide client with ability to set priority on Operations
[ https://issues.apache.org/jira/browse/HBASE-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284945#comment-15284945 ] churro morales commented on HBASE-15816: [~stack] Would you mind reviewing when you have a chance? I didn't want to make a monster patch, so I'll create another patch for removing the annotation driven priorities. Thanks. > Provide client with ability to set priority on Operations > -- > > Key: HBASE-15816 > URL: https://issues.apache.org/jira/browse/HBASE-15816 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: churro morales >Assignee: churro morales > Attachments: HBASE-15816-v1.patch, HBASE-15816.patch > > > First round will just be to expose the ability to set priorities for client > operations. For more background: > http://mail-archives.apache.org/mod_mbox/hbase-dev/201604.mbox/%3CCA+RK=_BG_o=q8HMptcP2WauAinmEsL+15f3YEJuz=qbpcya...@mail.gmail.com%3E > Next step would be to remove AnnotationReadingPriorityFunction and have the > client send priorities explicitly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15836) The -set parameter in "hbase backup" should be in the help text
[
https://issues.apache.org/jira/browse/HBASE-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284941#comment-15284941
]
Vladimir Rodionov commented on HBASE-15836:
---
Ted, can you remove "-s name Use the specified snapshot for full backup"
from helps as well. We do not support backups from snapshots.
> The -set parameter in "hbase backup" should be in the help text
> ---
>
> Key: HBASE-15836
> URL: https://issues.apache.org/jira/browse/HBASE-15836
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15836.v1.patch
>
>
> [~cartershanklin] noticed the following:
> {code}
> $ hbase backup create
> ERROR: wrong number of arguments
> Usage: hbase backup create [tables] [-s name]
> [-convert] [-silent] [-w workers][-b bandwith]
> type "full" to create a full backup image;
>"incremental" to create an incremental backup image
> backup_root_path The full root path to store the backup image,
> the prefix can be hdfs, webhdfs or gpfs
> Options:
> tables If no tables ("") are specified, all tables are backed up.
> Otherwise it is a
>comma separated list of tables.
> -s name Use the specified snapshot for full backup
> -convertFor an incremental backup, convert WAL files to HFiles
> -w number of parallel workers.
> -b bandwith per one worker (in MB sec)
> {code}
> A working backup command may look like the following (assuming the set named
> green exists):
> {code}
> sudo -u hbase hbase backup create full hdfs://ip:8020/tmp/backup -set green
> {code}
> Help text for backup command should be updated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15829) hbase.client.retries.number has different meanings in branch-1 and master
[
https://issues.apache.org/jira/browse/HBASE-15829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284933#comment-15284933
]
Ted Yu commented on HBASE-15829:
lgtm
> hbase.client.retries.number has different meanings in branch-1 and master
> -
>
> Key: HBASE-15829
> URL: https://issues.apache.org/jira/browse/HBASE-15829
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.0.0
>Reporter: Guanghao Zhang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15829.patch
>
>
> The comment of hbase.client.retries.number is:
> {code}
> /**
>* Parameter name for maximum retries, used as maximum for all retryable
>* operations such as fetching of the root region from root region server,
>* getting a cell's value, starting a row update, etc.
>*/
> public static final String HBASE_CLIENT_RETRIES_NUMBER =
> "hbase.client.retries.number";
> {code}
> In branch-1, the max attempts number equals with hbase.client.retries.number.
> But in master, the max attempts number equals with
> hbase.client.retries.number + 1.
> For RpcRetryingCaller.
> {code}
> this.retries = retries; // branch-1
> {code}
> {code}
> this.maxAttempts = retries + 1; // master
> {code}
> For AsyncProcess:
> {code}
> this.numTries = conf.getInt(HConstants.HBASE_CLIENT_RETRIES_NUMBER,
> HConstants.DEFAULT_HBASE_CLIENT_RETRIES_NUMBER); // branch-1
> {code}
> {code}
> // how many times we could try in total, one more than retry number
> this.numTries = conf.getInt(HConstants.HBASE_CLIENT_RETRIES_NUMBER,
> HConstants.DEFAULT_HBASE_CLIENT_RETRIES_NUMBER) + 1; // master
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15816) Provide client with ability to set priority on Operations
[ https://issues.apache.org/jira/browse/HBASE-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284934#comment-15284934 ] churro morales commented on HBASE-15816: I dont think that findbugs warning is related to this patch. > Provide client with ability to set priority on Operations > -- > > Key: HBASE-15816 > URL: https://issues.apache.org/jira/browse/HBASE-15816 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: churro morales >Assignee: churro morales > Attachments: HBASE-15816-v1.patch, HBASE-15816.patch > > > First round will just be to expose the ability to set priorities for client > operations. For more background: > http://mail-archives.apache.org/mod_mbox/hbase-dev/201604.mbox/%3CCA+RK=_BG_o=q8HMptcP2WauAinmEsL+15f3YEJuz=qbpcya...@mail.gmail.com%3E > Next step would be to remove AnnotationReadingPriorityFunction and have the > client send priorities explicitly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
