Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Anyone know what the refresh timeout is for the below? We seem to be in a phase where we have a bad pom and the hadoop test builds are failing. Can we force refresh of the local repository by doing something like a custom build run? Thanks, St.Ack P.S. Here is what I am talking about: https://builds.apache.org/job/PreCommit-HBASE-Build/178/artifact/patchprocess/patch-javac-2.4.0.txt which is from this run today: https://issues.apache.org/jira/browse/HBASE-15086 Thanks On Thu, Nov 5, 2015 at 10:42 AM, Sean Busbey wrote: > If Maven has trouble grabbing a pom but not an artifact, it'll > substitute in a placeholder pom that doesn't have e.g. license > information. That can result in a local repo that fails this way until > the refresh timeout hits for grabbing a pom again. > > On Wed, Nov 4, 2015 at 5:33 PM, Stack wrote: > > Thanks Andrew. Weird is that it is sporadic. Will keep an eye on it. > > St.Ack > > > > On Wed, Nov 4, 2015 at 8:14 AM, Andrew Purtell > > > wrote: > > > >> > [ERROR] Error invoking method 'get(java.lang.Integer)' in > >> java.util.ArrayList at META-INF/LICENSE.vm[line 1627, column 22] > >> > >> This means a Velocity macro for building LICENSE info about a component > >> has failed because necessary information is missing in the Maven model. > >> When I have seen this it has been due to a change in dependencies > without > >> necessary updates to supplemental-models.xml. Running the LICENSE and > >> NOTICE aggregations in debug mode will print out clues as to what > >> specifically is missing. See defines in the POMs for doing so. (I'm not > at > >> the computer yet so can't pull up the specifics.) > >> > >> > >> > On Nov 4, 2015, at 6:23 AM, Stack wrote: > >> > > >> > [ERROR] Error invoking method 'get(java.lang.Integer)' in > >> > java.util.ArrayList at META-INF/LICENSE.vm[line 1627, column 22] > >> > > > > -- > Sean >
Re: hbase git commit: HBASE-15073 Revert due to different opinion on usefulness
The commit message is wrong in the below. It misrepresents.
>From the issue:
The -1 is because no case was made for why we need this feature. Usually
folks show up w/ some numbers or a graph or some reasoning as to why
something is needed. In here there is none of that; there is 'certain use
cases' that go unspecified and then when pushed, some rambling about ambari
metrics server with no timelines or measure of what is being addressed or
fit criteria as to what would make ambari-case happy.
St.Ack
On Mon, Jan 18, 2016 at 5:11 PM, wrote:
> Repository: hbase
> Updated Branches:
> refs/heads/branch-1.2 c4bcaa3f6 -> 5ff65455c
>
>
> HBASE-15073 Revert due to different opinion on usefulness
>
>
> Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
> Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/5ff65455
> Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/5ff65455
> Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/5ff65455
>
> Branch: refs/heads/branch-1.2
> Commit: 5ff65455c60dbf56fc92630e75d93d7840dde537
> Parents: c4bcaa3
> Author: tedyu
> Authored: Mon Jan 18 17:11:32 2016 -0800
> Committer: tedyu
> Committed: Mon Jan 18 17:11:32 2016 -0800
>
> --
> .../apache/hadoop/hbase/HTableDescriptor.java | 60 +++-
> .../hbase/normalizer/NormalizationPlan.java | 45 ---
> .../org/apache/hadoop/hbase/master/HMaster.java | 20 ++-
> .../normalizer/EmptyNormalizationPlan.java | 6 --
> .../normalizer/MergeNormalizationPlan.java | 6 --
> .../master/normalizer/NormalizationPlan.java| 35
> .../master/normalizer/RegionNormalizer.java | 5 +-
> .../normalizer/SimpleRegionNormalizer.java | 11 ++--
> .../normalizer/SplitNormalizationPlan.java | 6 --
> .../normalizer/TestSimpleRegionNormalizer.java | 48
> .../TestSimpleRegionNormalizerOnCluster.java| 4 +-
> hbase-shell/src/main/ruby/hbase/admin.rb| 5 +-
> .../src/main/ruby/shell/commands/normalize.rb | 2 +-
> .../ruby/shell/commands/normalizer_switch.rb| 3 +-
> 14 files changed, 84 insertions(+), 172 deletions(-)
> --
>
>
>
> http://git-wip-us.apache.org/repos/asf/hbase/blob/5ff65455/hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
> --
> diff --git
> a/hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
> b/hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
> index dfb56e4..a6c08c3 100644
> ---
> a/hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
> +++
> b/hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
> @@ -44,8 +44,6 @@ import org.apache.hadoop.fs.Path;
> import org.apache.hadoop.hbase.client.Durability;
> import org.apache.hadoop.hbase.client.RegionReplicaUtil;
> import org.apache.hadoop.hbase.exceptions.DeserializationException;
> -import org.apache.hadoop.hbase.normalizer.NormalizationPlan;
> -import org.apache.hadoop.hbase.normalizer.NormalizationPlan.PlanType;
> import org.apache.hadoop.hbase.io.ImmutableBytesWritable;
> import org.apache.hadoop.hbase.protobuf.ProtobufUtil;
> import
> org.apache.hadoop.hbase.protobuf.generated.HBaseProtos.BytesBytesPair;
> @@ -203,14 +201,13 @@ public class HTableDescriptor implements
> WritableComparable {
>
>/**
> * INTERNAL Used by shell/rest interface to access this
> metadata
> - * attribute which denotes the allowed types of action (split/merge)
> when the table is treated
> - * by region normalizer.
> + * attribute which denotes if the table should be treated by region
> normalizer.
> *
> - * @see #getDesiredNormalizationTypes()
> + * @see #isNormalizationEnabled()
> */
> - public static final String NORMALIZATION_MODE = "NORMALIZATION_MODE";
> - private static final ImmutableBytesWritable NORMALIZATION_MODE_KEY =
> - new ImmutableBytesWritable(Bytes.toBytes(NORMALIZATION_MODE));
> + public static final String NORMALIZATION_ENABLED =
> "NORMALIZATION_ENABLED";
> + private static final ImmutableBytesWritable NORMALIZATION_ENABLED_KEY =
> +new ImmutableBytesWritable(Bytes.toBytes(NORMALIZATION_ENABLED));
>
>/** Default durability for HTD is USE_DEFAULT, which defaults to
> HBase-global default value */
>private static final Durability DEFAULT_DURABLITY =
> Durability.USE_DEFAULT;
> @@ -239,6 +236,11 @@ public class HTableDescriptor implements
> WritableComparable {
>public static final boolean DEFAULT_COMPACTION_ENABLED = true;
>
>/**
> + * Constant that denotes whether the table is normalized by default.
> + */
> + public static final boolean DEFAULT_NORMALIZATION_ENABLED = false;
> +
> + /**
> * Constant that denotes the maximum default size of the memstore after
> which
[jira] [Created] (HBASE-15128) Disable region splits and merges in HBCK
Enis Soztutar created HBASE-15128: - Summary: Disable region splits and merges in HBCK Key: HBASE-15128 URL: https://issues.apache.org/jira/browse/HBASE-15128 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Fix For: 2.0.0, 1.3.0 In large clusters where region splits are frequent, and HBCK runs take longer, the concurrent splits cause further problems in HBCK since HBCK assumes a static state for the region partition map. We have just seen a case where HBCK undo's a concurrently splitting region causing number of inconsistencies to go up. We can have a mode in master where splits and merges are disabled like the balancer and catalog janitor switches. Master will reject the split requests if regionservers decide to split. This switch can be turned on / off by the admins and also automatically by HBCK while it is running (similar to balancer switch being disabled by HBCK). HBCK should also disable the Catalog Janitor just in case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15127) NPE in RpcServer$Call.wrapWithSasl()
Enis Soztutar created HBASE-15127:
-
Summary: NPE in RpcServer$Call.wrapWithSasl()
Key: HBASE-15127
URL: https://issues.apache.org/jira/browse/HBASE-15127
Project: HBase
Issue Type: Bug
Reporter: Enis Soztutar
Fix For: 2.0.0, 1.3.0
I saw this in a log file, not sure whether it is important to fix or not:
{code}
2016-01-09 01:30:58,905 WARN [FifoRpcScheduler.handler1-thread-25]
ipc.RpcServer: FifoRpcScheduler.handler1-thread-25: caught:
java.lang.NullPointerException
at org.apache.hadoop.hbase.ipc.RpcServer$Call.wrapWithSasl(RpcServer.java:412)
at org.apache.hadoop.hbase.ipc.RpcServer$Call.setResponse(RpcServer.java:395)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:128)
at
org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.java:74)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
{code}
This is 0.98 based code base corresponding to:
{code}
synchronized (connection.saslServer) {
token = connection.saslServer.wrap(responseBytes, 0,
responseBytes.length);
}
{code}
I believe the saslserver was set to null earlier by {{disposeSasl()}} because
Connection is closing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu reopened HBASE-15073:
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Branch for HBASE-14030, HBASE-14123 etc
Thanks, Enis -Vlad On Mon, Jan 18, 2016 at 2:44 PM, Enis Söztutar wrote: > I have pushed the new branch: HBASE-7912. > > > https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=shortlog;h=refs/heads/HBASE-7912 > > Enis > > On Thu, Jan 14, 2016 at 3:35 PM, Enis Söztutar wrote: > > > I can create the branch tomorrow unless objection. > > > > Enis > > > > On Thu, Jan 14, 2016 at 3:31 PM, Ted Yu wrote: > > > >> +1 > >> > >> On Thu, Jan 14, 2016 at 3:28 PM, Matteo Bertozzi < > theo.berto...@gmail.com > >> > > >> wrote: > >> > >> > sounds good to me, it will be easier to look at and try out. > >> > > >> > > >> > Matteo > >> > > >> > > >> > On Thu, Jan 14, 2016 at 3:14 PM, Vladimir Rodionov < > >> vladrodio...@gmail.com > >> > > > >> > wrote: > >> > > >> > > Hi, folks > >> > > > >> > > I think we have to create separate branch for backup/restore work. > >> > > Any objections? > >> > > > >> > > -Vlad > >> > > > >> > > >> > > > > >
Re: Branch for HBASE-14030, HBASE-14123 etc
I have pushed the new branch: HBASE-7912. https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=shortlog;h=refs/heads/HBASE-7912 Enis On Thu, Jan 14, 2016 at 3:35 PM, Enis Söztutar wrote: > I can create the branch tomorrow unless objection. > > Enis > > On Thu, Jan 14, 2016 at 3:31 PM, Ted Yu wrote: > >> +1 >> >> On Thu, Jan 14, 2016 at 3:28 PM, Matteo Bertozzi > > >> wrote: >> >> > sounds good to me, it will be easier to look at and try out. >> > >> > >> > Matteo >> > >> > >> > On Thu, Jan 14, 2016 at 3:14 PM, Vladimir Rodionov < >> vladrodio...@gmail.com >> > > >> > wrote: >> > >> > > Hi, folks >> > > >> > > I think we have to create separate branch for backup/restore work. >> > > Any objections? >> > > >> > > -Vlad >> > > >> > >> > >
Re: 1.0.3 RC1 next Tuesday - Should we EOL 1.0?
Gentle reminder that the cut will happen tomorrow. Compat report is up at https://home.apache.org/~enis/1.0.2_1.0.3RC1_compat_report.html. Enis On Fri, Jan 15, 2016 at 2:41 PM, Andrew Purtell wrote: > +1 > > Procedurally we can just have branch RMs move on. There would be no more > planned releases but someone could come along to restart them, as you > mention. The maintenance of the branch will fall into an uncertain state. > If instead we declare we will not be supporting version X going forward > users on X will be much more clear about its status going forward. > > > > On Jan 15, 2016, at 1:45 PM, Enis Söztutar wrote: > > > > Hi, > > > > In time for the other RCs going on, let me do the next RC for 1.0.3 > > release. I'll cut the RC next Tuesday, the 19th of Jan. Let me know if > you > > want to commit anything to the branch. > > > > My original plan with 1.0 branch was to keep up monthly or semi-monthly > > release cadence, but since then, I changed my mind that having fewer but > > more tested-out releases is better. And since 1.1 releases are easily > > upgradable, and much more stable than 1.0, there is not much demand for > > continued 1.0 releases I feel. Therefore I do not plan to drive any more > > 1.0 release candidates unless there is a real need (security issue, etc). > > > > Of course, any PMC can drive the next 1.0.x release if needed, but it may > > also be worthwhile to discuss EOLing the 1.0 branch. How do you guys feel > > about that? There is already very limited bandwidth to properly test out > a > > release candidate that I think can go into maintaining 1.1 and 1.2 lines. > > > > Enis >
[jira] [Resolved] (HBASE-12557) Introduce timeout mechanism for IP to rack resolution
[ https://issues.apache.org/jira/browse/HBASE-12557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved HBASE-12557. Resolution: Later Doesn't seem necessary at the moment > Introduce timeout mechanism for IP to rack resolution > - > > Key: HBASE-12557 > URL: https://issues.apache.org/jira/browse/HBASE-12557 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Labels: loadbalancer > Attachments: 12557-v1.txt, 12557-v3.txt > > > Config parameter, "hbase.util.ip.to.rack.determiner", determines the class > which does IP to rack resolution. > The actual resolution may be lengthy. > This JIRA is continuation of HBASE-12554 where a mock DNSToSwitchMapping is > used for rack resolution. > A timeout parameter, "hbase.ip.to.rack.determiner.timeout", is proposed whose > value governs the duration which RackManager waits before rack resolution is > stopped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Successful: HBase Generate Website
Build status: Successful If successful, the website and docs have been generated. If failed, skip to the bottom of this email. Use the following commands to download the patch and apply it to a clean branch based on origin/asf-site. If you prefer to keep the hbase-site repo around permanently, you can skip the clone step. git clone https://git-wip-us.apache.org/repos/asf/hbase-site.git cd hbase-site wget -O- https://builds.apache.org/job/hbase_generate_website/104/artifact/website.patch.zip | funzip > 47fc696bc61e10386cf35644d95d2f45246d3269.patch git fetch git checkout -b asf-site-47fc696bc61e10386cf35644d95d2f45246d3269 origin/asf-site git am 47fc696bc61e10386cf35644d95d2f45246d3269.patch At this point, you can preview the changes by opening index.html or any of the other HTML pages in your local asf-site-47fc696bc61e10386cf35644d95d2f45246d3269 branch, and you can review the differences by running: git diff origin/asf-site There are lots of spurious changes, such as timestamps and CSS styles in tables. To see a list of files that have been added, deleted, renamed, changed type, or are otherwise interesting, use the following command: git diff --name-status --diff-filter=ADCRTXUB origin/asf-site To see only files that had 10 or more lines changed: git diff --stat origin/asf-site | grep -Ev "\|\s+\ [1-9]\ [\+-]+$" When you are satisfied, publish your changes to origin/asf-site using this command: git push origin asf-site-47fc696bc61e10386cf35644d95d2f45246d3269:asf-site Changes take a couple of minutes to be propagated. You can then remove your asf-site-47fc696bc61e10386cf35644d95d2f45246d3269 branch: git checkout asf-site && git branch -d asf-site-47fc696bc61e10386cf35644d95d2f45246d3269 If failed, see https://builds.apache.org/job/hbase_generate_website/104/console
Successful: hbase.apache.org HTML Checker
Successful If successful, the HTML and link-checking report for http://hbase.apache.org is available at https://builds.apache.org/job/HBase%20Website%20Link%20Ckecker/19/artifact/link_report/index.html. If failed, see https://builds.apache.org/job/HBase%20Website%20Link%20Ckecker/19/console.
