[jira] [Resolved] (HBASE-18284) Update hbasecon asia logo on home page of hbase.apache.org

2017-06-27 Thread stack (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-18284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-18284.
---
   Resolution: Fixed
 Assignee: stack
Fix Version/s: 3.0.0

Pushed new logo made by Bijieshan

> Update hbasecon asia logo on home page of hbase.apache.org
> --
>
> Key: HBASE-18284
> URL: https://issues.apache.org/jira/browse/HBASE-18284
> Project: HBase
>  Issue Type: Task
>Reporter: stack
>Assignee: stack
> Fix For: 3.0.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18284) Update hbasecon asia logo on home page of hbase.apache.org

2017-06-27 Thread stack (JIRA) 
stack created HBASE-18284:
-

 Summary: Update hbasecon asia logo on home page of hbase.apache.org
 Key: HBASE-18284
 URL: https://issues.apache.org/jira/browse/HBASE-18284
 Project: HBase
  Issue Type: Task
Reporter: stack






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-16542) Skip full backup in selected backup tests

2017-06-27 Thread Ted Yu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-16542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu resolved HBASE-16542.

Resolution: Later

> Skip full backup in selected backup tests
> -
>
> Key: HBASE-16542
> URL: https://issues.apache.org/jira/browse/HBASE-16542
> Project: HBase
>  Issue Type: Test
>Reporter: Ted Yu
>Priority: Minor
>  Labels: backup
> Fix For: HBASE-7912
>
>
> Since automatic mode is always used (HBASE-16037), some tests take longer 
> time to run:
> 1. restore full backup
> 2. restore incremental backup
> Action 2 would execute action 1 again.
> We can selectively skip full backup in backup / restore tests where 
> incremental backup is involved.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-15103) hadoopcheck test should provide diff file showing what's new

2017-06-27 Thread Ted Yu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-15103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu resolved HBASE-15103.

Resolution: Later

> hadoopcheck test should provide diff file showing what's new
> 
>
> Key: HBASE-15103
> URL: https://issues.apache.org/jira/browse/HBASE-15103
> Project: HBase
>  Issue Type: Test
>  Components: test
>Reporter: Ted Yu
>Priority: Minor
>  Labels: test
>
> Currently developer has to go to 'build artifacts' folder to read output from 
> hadoopcheck test.
> e.g.
> https://builds.apache.org/job/PreCommit-HBASE-Build/98/artifact/patchprocess/patch-javac-2.6.1.txt
> hadoopcheck test should provide diff file showing what exactly is new
> Thanks to Sean for offline discussion



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18283) Provide a construct method which accept a thread pool for AsyncAdmin

2017-06-27 Thread Guanghao Zhang (JIRA) 
Guanghao Zhang created HBASE-18283:
--

 Summary: Provide a construct method which accept a thread pool for 
AsyncAdmin
 Key: HBASE-18283
 URL: https://issues.apache.org/jira/browse/HBASE-18283
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 3.0.0, 2.0.0-alpha-2
Reporter: Guanghao Zhang
Assignee: Guanghao Zhang


Similar to AsyncTable, provide a construct method which accept a thread pool 
for normal user. User need provide a thread pool to get a AsyncAdmin instance. 
Then the callbacks registered to the returned CompletableFuture can be executed 
in that thread pool.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)

2017-06-27 Thread Stack 
On Tue, Jun 27, 2017 at 10:28 AM, Sean Busbey  wrote:

> On Tue, Jun 27, 2017 at 11:38 AM, Stack  wrote:
>
> > On Tue, Jun 27, 2017 at 9:24 AM, Sean Busbey  wrote:
> >
> > > FYI, I've updated the precommit build to use Yetus 0.4.0 (which is the
> > > current release).
> > >
> > > Shouldn't impact much. if things look off ping me.
> > >
> > >
> > Thanks Sean.
> >
> > Whats new in 0.4.0?
> >
> >
> This change was spurred by YETUS-520, which EOL's versions earlier than
> 0.4.0.
>
> Couple of things that changed:
>
>
Helps. Thanks. See below.



> * test-patch got better about handling pylint (which rarely impacts us, but
> will help the effort to adopt a pylintrc for our dev-support stuff.)
> * docker cleanup got more robust (I think we've been sheltered from this
> because Hadoop's been running a more up-to-date version of Yetus more
> frequently than us on asf infra)
> * test-patch's whitespace plugin can configured to ignore some files (but I
> can't think of any we'd care to so whitelist)
>

Generated files.


> * test-patch's nightly mode got a new email format that's not so verbose (I
> believe I'm the only one getting these emails, maybe the new format will be
> worth sending to the notification list)
>
>
Sure. Sounds good.

Thanks Sean.

S


> Here's the notes:
>
> http://yetus.apache.org/documentation/0.4.0/RELEASENOTES/
>

[jira] [Created] (HBASE-18282) ReplicationLogCleaner can delete WALs not yet replicated in case of a KeeperException

2017-06-27 Thread Ashu Pachauri (JIRA) 
Ashu Pachauri created HBASE-18282:
-

 Summary: ReplicationLogCleaner can delete WALs not yet replicated 
in case of a KeeperException
 Key: HBASE-18282
 URL: https://issues.apache.org/jira/browse/HBASE-18282
 Project: HBase
  Issue Type: Bug
  Components: Replication
Affects Versions: 2.0.0-alpha-1, 1.1.11, 1.2.6, 1.3.1
Reporter: Ashu Pachauri
Assignee: Ashu Pachauri
Priority: Critical


ReplicationStateZKBase#getListOfReplicators does not rethrow a KeeperException 
and returns null in such a case. ReplicationLogCleaner just assumes that there 
are no replicators and deletes everything.

ReplicationStateZKBase:
{code:java}
public List getListOfReplicators() {
List result = null;
try {
  result = ZKUtil.listChildrenNoWatch(this.zookeeper, this.queuesZNode);
} catch (KeeperException e) {
  this.abortable.abort("Failed to get list of replicators", e);
}
return result;
  }
{code}

ReplicationLogCleaner:
{code:java}
private Set loadWALsFromQueues() throws KeeperException {
for (int retry = 0; ; retry++) {
  int v0 = replicationQueues.getQueuesZNodeCversion();
  List rss = replicationQueues.getListOfReplicators();
  if (rss == null) {
LOG.debug("Didn't find any region server that replicates, won't prevent 
any deletions.");
return ImmutableSet.of();
  }
  ...
{code}





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [DISCUSS] More Shading

2017-06-27 Thread Stack 
Bit of an update.

I'd suggest we go ahead w/ the hbase-thirdparty project [2]. It took a
while but in its current form -- a few poms that package a few jars [1]--
it at least enables the below:

+ Allows us to skip checking in protobuf generated files (25MB!); they can
be generated inline w/ the build because the hackery patching protobuf has
been moved out to hbase-thirdparty. There is a patch up on HBASE-17056.
+ Update our guava from 12.0 to 22.0 w/o clashing w/ the guava of others.
There is a patch at HBASE-17908. It is taking a bit of wrangling getting it
to land because I pared back transitive includes from hadoop and it takes a
while to work through the failures.

Other benefits are the protobuf-util lib is on the classpath now -- its in
hbase-thirdparty relocated; depends on pb and guava -- so we have facility
to goat "HBASE-18106 Redo ProcedureInfo and LockInfo" and shading netty is
almost done so we can do with netty as we wilt independent of hadoop and
downstreamers (the hard part -- relocation of the .so -- should be done).

Let me figure how to run a vote for a couple of poms.

St.Ack

1.
https://repository.apache.org/content/groups/snapshots/org/apache/hbase/thirdparty/
(see hbase-shaded-thirdparty and hbase-shaded-protobuf)
2. https://git-wip-us.apache.org/repos/asf/hbase-thirdparty


On Tue, Jun 20, 2017 at 11:04 AM, Josh Elser  wrote:

> On 6/20/17 1:28 AM, Stack wrote:
>
>> On Thu, Apr 13, 2017 at 4:46 PM, Josh Elser  wrote:
>>
>> ...
>>>
>>> I think pushing this part forward with some code is the next logical
>>> step.
>>> Seems to be consensus about taking our known internal dependencies and
>>> performing this shade magic.
>>>
>>>
>>> I opened HBASE-18240 "Add hbase-auxillary, a project with hbase utility
>> including an hbase-shaded-thirdparty module with guava, netty, etc."
>>
>> It has a tarball attached that bundles the outline of an hbase-auxillary
>> project (groupId:org.apache.hbase.auxillary). This project is intended to
>> be standalone, in its own repository, publishing its own artifacts under
>> the aegis of this project's PMC.
>>
>> It includes the first instance of an auxillary utility, a module named
>> hbase-thirdparty-shaded (artifactId:hbase-thirdparty-shaded). Herein
>> we'll
>> pull down 3rd party libs and republish at an offset; e.g.
>> com.google.common.* from guava will be at
>> org.apache.hbase.thirdparty.shaded.com.google.common.*. Currently it
>> builds
>> a jar that includes a relocated guava 22.0.
>>
>> I then messed around making hbase-common use it (You have to build the
>> hbase-auxillary into your local repo). I put up a patch on the issue.
>> Mostly its mass find-and-replace w/ some clean up of transitive includes
>> of
>> guava from hadoop-common and some small fixup of methods renamed between
>> guava 12.0 and 22.0.
>>
>> Unless objection, I was going to press on. Sean offered to help set up new
>> repo. We can always undo and delete it if this project fails.
>>
>> When done, the hope is we are on a modern version of guava and our netty
>> and protobuf 3 will be be relocated, 'hidden' from downstream (and won't
>> clash w/ upstream). I hope to also purge the pre-build we have in our
>> modules that do protobuf moving this hackery out and under
>> hbase-thirdparty-shaded.
>>
>> St.Ack
>>
>
> Kudos on the JFDI approach :). I think having something concrete to show
> is the best way to judge success of it.
>
> Will keep an eye on HBASE-18240.
>
>

[jira] [Created] (HBASE-18281) Performance update in StoreFileWriter.java for string replacement

2017-06-27 Thread Ben Epstein (JIRA) 
Ben Epstein created HBASE-18281:
---

 Summary: Performance update in StoreFileWriter.java for string 
replacement
 Key: HBASE-18281
 URL: https://issues.apache.org/jira/browse/HBASE-18281
 Project: HBase
  Issue Type: Improvement
  Components: community
Reporter: Ben Epstein
Priority: Trivial


Change the replaceAll() function using regex with a static Pattern for 
performance upgrade



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18280) Manual Array Copy Cleanup: Automated

2017-06-27 Thread John Leach (JIRA) 
John Leach created HBASE-18280:
--

 Summary: Manual Array Copy Cleanup: Automated
 Key: HBASE-18280
 URL: https://issues.apache.org/jira/browse/HBASE-18280
 Project: HBase
  Issue Type: Improvement
Reporter: John Leach
Assignee: John Leach
Priority: Trivial


Remove Manual Array Copies: Style Issue



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18279) Manual Array to Collection Copy: Automated

2017-06-27 Thread John Leach (JIRA) 
John Leach created HBASE-18279:
--

 Summary: Manual Array to Collection Copy: Automated
 Key: HBASE-18279
 URL: https://issues.apache.org/jira/browse/HBASE-18279
 Project: HBase
  Issue Type: Improvement
Reporter: John Leach
Assignee: John Leach


Style Cleanup.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [JENKINS] [IMPORTANT] - Jenkins Migration and Upgrade (And JDK7 deprecation)

2017-06-27 Thread Josh Elser 
Sorry for the churn! I didn't update my inbox soon enough it seems and 
missed Mike's thread when sending mine :)


On 6/27/17 12:16 PM, Sean Busbey wrote:

Let's keep these on one thread:

https://lists.apache.org/thread.html/cb08670178f5c645ad2387f14ad485face0594e89d976bf31865596f@%3Cdev.hbase.apache.org%3E

On Tue, Jun 27, 2017 at 10:57 AM, Josh Elser  wrote:


tl;dr Infra is upgrading Jenkins in 2 weeks and Java7 Maven jobs
may/may-not work after this. See explanation below from [1]:


Users with jobs configured with the "Maven project" type may not be able
to use Java 7 for their Maven jobs. The correct behavior is not guaranteed
so proceed at your own risk. The Maven Project uses Jenkins Remoting to
establish "interceptors" within the Maven executable. Because of this,
Maven uses Remoting and other Jenkins core classes, and this behavior may
break an update.


[1] https://jenkins.io/blog/2017/04/10/jenkins-has-upgraded-to-java-8/

 Forwarded Message 
Subject: [JENKINS]  [IMPORTANT] - Jenkins Migration and Upgrade (And JDK7
deprecation)
Date: Tue, 27 Jun 2017 17:03:13 +1000
From: Gavin McDonald 
Reply-To: bui...@apache.org, bui...@apache.org
To: bui...@apache.org
CC: ASF Operations 

ASF Jenkins Master Migration and Upgrade on :-


LocationLocal Time
   Time Zone   UTC Offset
Melbourne (Australia - Victoria)Sunday, 16 July 2017 at 10:00:00
am AESTUTC+10 hours
New York (USA - New York)   Saturday, 15 July 2017 at 8:00:00
pm EDTUTC-4 hours
Corresponding UTC (GMT) Sunday, 16 July 2017 at 00:00:00


Hi All,

A few things are going to happen in just over 2 weeks.

1. Migration of Jenkins to a new host. A Jenkins Master module and yaml
have been puppetized and ready to go.
 What we need to do to migrate the Master away from its current host is
turn off the old service. Perform a final rsync of data and perform the
migration tasks.
 As we intend to preserve history for jobs this will take some time.
 At the same time as doing this migration to a new host, all slave
connections will be updated (see below.)
 I have no current estimate of downtime, but it will run into several
hours. We do plan to run this migration on a Sunday at the lowest part
of Jenkins usual usage.

2. Upgrade of Jenkins - Jenkins project released a new LTS release,
version 2.60.1. This is a major release and breaks Jenkins in terms of
Maven jobs for JDK 7 in the same way that it happened for Maven and JDK 6 a
few months back.

 The infra team (mainly myself) got quite some feedback on not
supplying advance notice of this breakage. That upgrade however was
necessary due to security fixes that required our upgrade.  This email
serves as advance warning of the upcoming upgrade of Jenkins, the
downtime due to the migration of the service to a new host; and notice of
the breakage to JDK 7 that the upgrade brings.

 Please familiarise yourself with the Jenkins LTS upgrade notes at [1].
 In particular please note:-

 “…2.60.1 is the first Jenkins LTS release that requires Java 8 to run.
If you're using the Maven Project type, please note that it needs to use a
JDK capable of running Jenkins, i.e. JDK 8 or up. If you configure an older
JDK in a Maven Project, Jenkins will attempt to find a newer JDK and use
that automatically. If your SSH Slaves fail to start and you have the
plugin install the JRE to run them, make sure to update SSH Slaves Plugin
to at least version 1.17 (1.20 recommended).
Changes since 2.60:
Fix for NullPointerException while initiating some SSH connections
(regression in 2.59). (issue 44120 )

Notable changes since 2.46.3:
Jenkins (master and agents) now requires Java 8 to run. (issue 27624 <
https://issues.jenkins-ci.org/browse/JENKINS-27624> <>, issue 42709 <
https://issues.jenkins-ci.org/browse/JENKINS-42709> <>, pull 2802 <
https://github.com/jenkinsci/jenkins/pull/2802>, announcement blog post <
https://jenkins.io/blog/2017/04/10/jenkins-has-upgraded-to-java-8/>)

…”

There are over 30 other enhancements/fixes since 2.46.2 which we currently
run so please do take a note of those.

Recap: In just over 2 weeks, downtime for a migration AND upgrade is
planned.
Please do not rely on Jenkins at all for that weekend if you use it in
your release workflow.

Please do take this notice back to your dev lists.
Any questions or concerns please email back to bui...@apache.org  only.
Thanks

Gav…

[1] - https://jenkins.io/changelog-stable/

[jira] [Created] (HBASE-18278) [AMv2] Enable and fix uni test hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta

2017-06-27 Thread Umesh Agashe (JIRA) 
Umesh Agashe created HBASE-18278:


 Summary: [AMv2] Enable and fix uni test 
hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta
 Key: HBASE-18278
 URL: https://issues.apache.org/jira/browse/HBASE-18278
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0-alpha-1
Reporter: Umesh Agashe
Assignee: Umesh Agashe
 Fix For: 2.0.0-alpha-2


Enable and fix uni test 
hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18277) Triage lack of branch-1.2 nightlies

2017-06-27 Thread Sean Busbey (JIRA) 
Sean Busbey created HBASE-18277:
---

 Summary: Triage lack of branch-1.2 nightlies
 Key: HBASE-18277
 URL: https://issues.apache.org/jira/browse/HBASE-18277
 Project: HBase
  Issue Type: Task
  Components: test
Affects Versions: 1.2.7
Reporter: Sean Busbey
Priority: Blocker
 Fix For: 1.2.7


Last run of either branch-1.2 job was June 22nd. Probably an infra change in 
labels.

While we're at it, either turn it back into a matrix job (using Appy's "short 
workspace name" bit) or better yet convert to yetus test-patch's nightly mode 
in anticipation of the coming loss of jdk7.

* https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2-JDK7/
* https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2-JDK8/



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18276) Release 1.2.7

2017-06-27 Thread Sean Busbey (JIRA) 
Sean Busbey created HBASE-18276:
---

 Summary: Release 1.2.7
 Key: HBASE-18276
 URL: https://issues.apache.org/jira/browse/HBASE-18276
 Project: HBase
  Issue Type: Task
  Components: community
Reporter: Sean Busbey
Assignee: Sean Busbey
 Fix For: 1.2.7


about time to get rolling on 1.2.7 for ~monthly 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)

2017-06-27 Thread Sean Busbey 
On Tue, Jun 27, 2017 at 11:38 AM, Stack  wrote:

> On Tue, Jun 27, 2017 at 9:24 AM, Sean Busbey  wrote:
>
> > FYI, I've updated the precommit build to use Yetus 0.4.0 (which is the
> > current release).
> >
> > Shouldn't impact much. if things look off ping me.
> >
> >
> Thanks Sean.
>
> Whats new in 0.4.0?
>
>
This change was spurred by YETUS-520, which EOL's versions earlier than
0.4.0.

Couple of things that changed:

* test-patch got better about handling pylint (which rarely impacts us, but
will help the effort to adopt a pylintrc for our dev-support stuff.)
* docker cleanup got more robust (I think we've been sheltered from this
because Hadoop's been running a more up-to-date version of Yetus more
frequently than us on asf infra)
* test-patch's whitespace plugin can configured to ignore some files (but I
can't think of any we'd care to so whitelist)
* test-patch's nightly mode got a new email format that's not so verbose (I
believe I'm the only one getting these emails, maybe the new format will be
worth sending to the notification list)

Here's the notes:

http://yetus.apache.org/documentation/0.4.0/RELEASENOTES/

Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)

2017-06-27 Thread Stack 
On Tue, Jun 27, 2017 at 9:24 AM, Sean Busbey  wrote:

> FYI, I've updated the precommit build to use Yetus 0.4.0 (which is the
> current release).
>
> Shouldn't impact much. if things look off ping me.
>
>
Thanks Sean.

Whats new in 0.4.0?

S


> On Wed, Mar 1, 2017 at 2:23 PM, Mikhail Antonov 
> wrote:
>
> > Ouch. Thanks Sean!
> >
> > I'm pretty sure at some point I was debugging 1.3-IT job and saw
> branch-1.3
> > getting checked out in the logs. Not sure how/when it went sideways
> though.
> >
> > Yeah, let's see how it goes.
> >
> > -Mikhail
> >
> > On Wed, Mar 1, 2017 at 5:50 AM, Sean Busbey  wrote:
> >
> > > Fun times.
> > >
> > > 1) Turns out our 1.3-IT jobs have been running against branch-1.2.
> > > Don't know how long, but as long as we have history.
> > >
> > > 2) I deleted the failing-since-august 1.2-IT job.
> > >
> > > 3) I renamed the passing 1.3-IT job that runs against branch-1.2 to be
> > > the 1.2-IT job
> > >
> > > 4) I copied the now renamed 1.2-IT job and made a 1.3-IT job that runs
> > > against branch-1.3
> > >
> > > I kicked off jobs after all this shuffling. We'll see how it goes.
> > >
> > > On Tue, Feb 21, 2017 at 5:49 PM, Sean Busbey 
> wrote:
> > > > FYI, I updated the 1.2-IT and 1.3-IT jobs today to use Appy's
> > > > suggested "custom child workspace" of "${SHORT_COMBINATION}", since
> > > > spaces in paths had caused them to fail for a v long time.
> > > >
> > > > On Fri, Oct 14, 2016 at 4:44 PM, Andrew Purtell  >
> > > wrote:
> > > >> Thanks Ted, that would be a nice contribution, thank you.
> > > >>
> > > >>
> > > >> On Fri, Oct 14, 2016 at 12:07 PM, Apekshit Sharma <
> a...@cloudera.com>
> > > wrote:
> > > >>
> > > >>> @Ted, here's the old jira, HBASE-14167. Use that.
> > > >>>
> > > >>> On Fri, Oct 14, 2016 at 12:02 PM, Ted Yu 
> > wrote:
> > > >>>
> > > >>> > I just ran the tests in hbase-spark module using 'mvn verify'.
> > > >>> >
> > > >>> > All passed.
> > > >>> >
> > > >>> > I am testing a patch locally where hbase-spark tests are run in
> > test
> > > >>> phase.
> > > >>> >
> > > >>> > If the tests pass, I will log a JIRA.
> > > >>> >
> > > >>> > Thanks
> > > >>> >
> > > >>> > > On Oct 14, 2016, at 11:41 AM, Andrew Purtell <
> > apurt...@apache.org>
> > > >>> > wrote:
> > > >>> > >
> > > >>> > > The hbase-spark integration tests run (and fail) for me locally
> > > >>> whenever
> > > >>> > I
> > > >>> > > build master with 'mvn clean install -DskipITs' .
> > > >>> > >
> > > >>> > > HBaseConnectionCacheSuite:
> > > >>> > > - all test cases *** FAILED ***
> > > >>> > >  2 did not equal 1 (HBaseConnectionCacheSuite.scala:92)
> > > >>> > >
> > > >>> > > Saw it but had to ignore/triage to get something else done.
> > > >>> > >
> > > >>> > > We have a weird situation where integration tests run when they
> > > >>> shouldn't
> > > >>> > > locally yet no tests run at all for patch process?
> > > >>> > >
> > > >>> > > I would like to see Spark behave like the other modules. I
> > remember
> > > >>> > filing
> > > >>> > > a JIRA asking that hbase-spark honor -DskipITs. It still
> doesn't.
> > > >>> > > Meanwhile, it does its own thing with '-DskipSparkTests', which
> > is
> > > not
> > > >>> > > appropriate given that none of the other modules have their own
> > > >>> distinct
> > > >>> > > control parameters. There also doesn't seem to be a distinction
> > > between
> > > >>> > > unit tests and integration tests. The 'test' target does
> nothing.
> > > >>> > > Everything happens during the 'integration-test' phase. Is
> this a
> > > Spark
> > > >>> > > limitation?
> > > >>> > >
> > > >>> > >
> > > >>> > >> On Fri, Oct 14, 2016 at 11:27 AM, Sean Busbey <
> > > bus...@cloudera.com>
> > > >>> > wrote:
> > > >>> > >>
> > > >>> > >> Do the HBase Spark tests only run during the maven verify
> > command?
> > > >>> > >> We'll need to update our personality to say that that command
> > > should
> > > >>> > >> be used for unit tests when in the hbase spark module. ugh.
> > > >>> > >>
> > > >>> > >> On Thu, Oct 13, 2016 at 7:42 PM, Apekshit Sharma <
> > > a...@cloudera.com>
> > > >>> > >> wrote:
> > > >>> > >>> Our patch process isn't running hbase-spark tests. See this
> for
> > > >>> > example:
> > > >>> > >>>
> > > >>> > >>> https://builds.apache.org/job/PreCommit-HBASE-Build/3842/
> > > >>> > >>> https://builds.apache.org/job/PreCommit-HBASE-Build/3842/
> > > >>> > >> artifact/patchprocess/patch-unit-hbase-spark.txt/*view*/
> > > >>> > >>>
> > > >>> > >>> Found it when trying to debug cause of trunk failures. Part
> of
> > > the
> > > >>> > cause
> > > >>> > >> is
> > > >>> > >>> hbase-spark's HBaseConnectionCacheSuite test failure (
> > > >>> > >>> https://builds.apache.org/view/All/job/HBase-Trunk_
> > > >>> > >> matrix/jdk=JDK%201.8%20(latest),label=yahoo-not-h2/
> > > 1776/consoleFull
> > > >>> > >>  > > >>> > matrix/jdk=JDK%201.8%20%28latest%29,label=yahoo-not-
> > > h2/1776/consoleFull>
> > > >>> > >> )
> > > >>> > >>> w

Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)

2017-06-27 Thread Sean Busbey 
FYI, I've updated the precommit build to use Yetus 0.4.0 (which is the
current release).

Shouldn't impact much. if things look off ping me.

On Wed, Mar 1, 2017 at 2:23 PM, Mikhail Antonov 
wrote:

> Ouch. Thanks Sean!
>
> I'm pretty sure at some point I was debugging 1.3-IT job and saw branch-1.3
> getting checked out in the logs. Not sure how/when it went sideways though.
>
> Yeah, let's see how it goes.
>
> -Mikhail
>
> On Wed, Mar 1, 2017 at 5:50 AM, Sean Busbey  wrote:
>
> > Fun times.
> >
> > 1) Turns out our 1.3-IT jobs have been running against branch-1.2.
> > Don't know how long, but as long as we have history.
> >
> > 2) I deleted the failing-since-august 1.2-IT job.
> >
> > 3) I renamed the passing 1.3-IT job that runs against branch-1.2 to be
> > the 1.2-IT job
> >
> > 4) I copied the now renamed 1.2-IT job and made a 1.3-IT job that runs
> > against branch-1.3
> >
> > I kicked off jobs after all this shuffling. We'll see how it goes.
> >
> > On Tue, Feb 21, 2017 at 5:49 PM, Sean Busbey  wrote:
> > > FYI, I updated the 1.2-IT and 1.3-IT jobs today to use Appy's
> > > suggested "custom child workspace" of "${SHORT_COMBINATION}", since
> > > spaces in paths had caused them to fail for a v long time.
> > >
> > > On Fri, Oct 14, 2016 at 4:44 PM, Andrew Purtell 
> > wrote:
> > >> Thanks Ted, that would be a nice contribution, thank you.
> > >>
> > >>
> > >> On Fri, Oct 14, 2016 at 12:07 PM, Apekshit Sharma 
> > wrote:
> > >>
> > >>> @Ted, here's the old jira, HBASE-14167. Use that.
> > >>>
> > >>> On Fri, Oct 14, 2016 at 12:02 PM, Ted Yu 
> wrote:
> > >>>
> > >>> > I just ran the tests in hbase-spark module using 'mvn verify'.
> > >>> >
> > >>> > All passed.
> > >>> >
> > >>> > I am testing a patch locally where hbase-spark tests are run in
> test
> > >>> phase.
> > >>> >
> > >>> > If the tests pass, I will log a JIRA.
> > >>> >
> > >>> > Thanks
> > >>> >
> > >>> > > On Oct 14, 2016, at 11:41 AM, Andrew Purtell <
> apurt...@apache.org>
> > >>> > wrote:
> > >>> > >
> > >>> > > The hbase-spark integration tests run (and fail) for me locally
> > >>> whenever
> > >>> > I
> > >>> > > build master with 'mvn clean install -DskipITs' .
> > >>> > >
> > >>> > > HBaseConnectionCacheSuite:
> > >>> > > - all test cases *** FAILED ***
> > >>> > >  2 did not equal 1 (HBaseConnectionCacheSuite.scala:92)
> > >>> > >
> > >>> > > Saw it but had to ignore/triage to get something else done.
> > >>> > >
> > >>> > > We have a weird situation where integration tests run when they
> > >>> shouldn't
> > >>> > > locally yet no tests run at all for patch process?
> > >>> > >
> > >>> > > I would like to see Spark behave like the other modules. I
> remember
> > >>> > filing
> > >>> > > a JIRA asking that hbase-spark honor -DskipITs. It still doesn't.
> > >>> > > Meanwhile, it does its own thing with '-DskipSparkTests', which
> is
> > not
> > >>> > > appropriate given that none of the other modules have their own
> > >>> distinct
> > >>> > > control parameters. There also doesn't seem to be a distinction
> > between
> > >>> > > unit tests and integration tests. The 'test' target does nothing.
> > >>> > > Everything happens during the 'integration-test' phase. Is this a
> > Spark
> > >>> > > limitation?
> > >>> > >
> > >>> > >
> > >>> > >> On Fri, Oct 14, 2016 at 11:27 AM, Sean Busbey <
> > bus...@cloudera.com>
> > >>> > wrote:
> > >>> > >>
> > >>> > >> Do the HBase Spark tests only run during the maven verify
> command?
> > >>> > >> We'll need to update our personality to say that that command
> > should
> > >>> > >> be used for unit tests when in the hbase spark module. ugh.
> > >>> > >>
> > >>> > >> On Thu, Oct 13, 2016 at 7:42 PM, Apekshit Sharma <
> > a...@cloudera.com>
> > >>> > >> wrote:
> > >>> > >>> Our patch process isn't running hbase-spark tests. See this for
> > >>> > example:
> > >>> > >>>
> > >>> > >>> https://builds.apache.org/job/PreCommit-HBASE-Build/3842/
> > >>> > >>> https://builds.apache.org/job/PreCommit-HBASE-Build/3842/
> > >>> > >> artifact/patchprocess/patch-unit-hbase-spark.txt/*view*/
> > >>> > >>>
> > >>> > >>> Found it when trying to debug cause of trunk failures. Part of
> > the
> > >>> > cause
> > >>> > >> is
> > >>> > >>> hbase-spark's HBaseConnectionCacheSuite test failure (
> > >>> > >>> https://builds.apache.org/view/All/job/HBase-Trunk_
> > >>> > >> matrix/jdk=JDK%201.8%20(latest),label=yahoo-not-h2/
> > 1776/consoleFull
> > >>> > >>  > >>> > matrix/jdk=JDK%201.8%20%28latest%29,label=yahoo-not-
> > h2/1776/consoleFull>
> > >>> > >> )
> > >>> > >>> which was added in HBASE-16638. However, to be fair, QA was
> > green and
> > >>> > >>> reported passing hbase-spark tests for that jira.
> > >>> > >>>
> > >>> >  On Mon, Sep 19, 2016 at 12:57 PM, Stack 
> > wrote:
> > >>> > 
> > >>> >  childCustomWorkspace seems to be just the ticket. Nice find
> > Appy.
> > >>> >  St.Ack
> > >>> > 
> > >>> >  On Mon, Sep 19, 2016 at 10:0

Re: [JENKINS] [IMPORTANT] - Jenkins Migration and Upgrade (And JDK7 deprecation)

2017-06-27 Thread Sean Busbey 
Let's keep these on one thread:

https://lists.apache.org/thread.html/cb08670178f5c645ad2387f14ad485face0594e89d976bf31865596f@%3Cdev.hbase.apache.org%3E

On Tue, Jun 27, 2017 at 10:57 AM, Josh Elser  wrote:

> tl;dr Infra is upgrading Jenkins in 2 weeks and Java7 Maven jobs
> may/may-not work after this. See explanation below from [1]:
>
> 
> Users with jobs configured with the "Maven project" type may not be able
> to use Java 7 for their Maven jobs. The correct behavior is not guaranteed
> so proceed at your own risk. The Maven Project uses Jenkins Remoting to
> establish "interceptors" within the Maven executable. Because of this,
> Maven uses Remoting and other Jenkins core classes, and this behavior may
> break an update.
> 
>
> [1] https://jenkins.io/blog/2017/04/10/jenkins-has-upgraded-to-java-8/
>
>  Forwarded Message 
> Subject: [JENKINS]  [IMPORTANT] - Jenkins Migration and Upgrade (And JDK7
> deprecation)
> Date: Tue, 27 Jun 2017 17:03:13 +1000
> From: Gavin McDonald 
> Reply-To: bui...@apache.org, bui...@apache.org
> To: bui...@apache.org
> CC: ASF Operations 
>
> ASF Jenkins Master Migration and Upgrade on :-
>
>
> LocationLocal Time
>   Time Zone   UTC Offset
> Melbourne (Australia - Victoria)Sunday, 16 July 2017 at 10:00:00
> am AESTUTC+10 hours
> New York (USA - New York)   Saturday, 15 July 2017 at 8:00:00
> pm EDTUTC-4 hours
> Corresponding UTC (GMT) Sunday, 16 July 2017 at 00:00:00
>
>
> Hi All,
>
> A few things are going to happen in just over 2 weeks.
>
> 1. Migration of Jenkins to a new host. A Jenkins Master module and yaml
> have been puppetized and ready to go.
> What we need to do to migrate the Master away from its current host is
> turn off the old service. Perform a final rsync of data and perform the
> migration tasks.
> As we intend to preserve history for jobs this will take some time.
> At the same time as doing this migration to a new host, all slave
> connections will be updated (see below.)
> I have no current estimate of downtime, but it will run into several
> hours. We do plan to run this migration on a Sunday at the lowest part
> of Jenkins usual usage.
>
> 2. Upgrade of Jenkins - Jenkins project released a new LTS release,
> version 2.60.1. This is a major release and breaks Jenkins in terms of
> Maven jobs for JDK 7 in the same way that it happened for Maven and JDK 6 a
> few months back.
>
> The infra team (mainly myself) got quite some feedback on not
> supplying advance notice of this breakage. That upgrade however was
> necessary due to security fixes that required our upgrade.  This email
> serves as advance warning of the upcoming upgrade of Jenkins, the
> downtime due to the migration of the service to a new host; and notice of
> the breakage to JDK 7 that the upgrade brings.
>
> Please familiarise yourself with the Jenkins LTS upgrade notes at [1].
> In particular please note:-
>
> “…2.60.1 is the first Jenkins LTS release that requires Java 8 to run.
> If you're using the Maven Project type, please note that it needs to use a
> JDK capable of running Jenkins, i.e. JDK 8 or up. If you configure an older
> JDK in a Maven Project, Jenkins will attempt to find a newer JDK and use
> that automatically. If your SSH Slaves fail to start and you have the
> plugin install the JRE to run them, make sure to update SSH Slaves Plugin
> to at least version 1.17 (1.20 recommended).
> Changes since 2.60:
> Fix for NullPointerException while initiating some SSH connections
> (regression in 2.59). (issue 44120  /browse/JENKINS-44120>)
>
> Notable changes since 2.46.3:
> Jenkins (master and agents) now requires Java 8 to run. (issue 27624 <
> https://issues.jenkins-ci.org/browse/JENKINS-27624> <>, issue 42709 <
> https://issues.jenkins-ci.org/browse/JENKINS-42709> <>, pull 2802 <
> https://github.com/jenkinsci/jenkins/pull/2802>, announcement blog post <
> https://jenkins.io/blog/2017/04/10/jenkins-has-upgraded-to-java-8/>)
>
> …”
>
> There are over 30 other enhancements/fixes since 2.46.2 which we currently
> run so please do take a note of those.
>
> Recap: In just over 2 weeks, downtime for a migration AND upgrade is
> planned.
> Please do not rely on Jenkins at all for that weekend if you use it in
> your release workflow.
>
> Please do take this notice back to your dev lists.
> Any questions or concerns please email back to bui...@apache.org  bui...@apache.org> only.
> Thanks
>
> Gav…
>
> [1] - https://jenkins.io/changelog-stable/
>

Fwd: [JENKINS] [IMPORTANT] - Jenkins Migration and Upgrade (And JDK7 deprecation)

2017-06-27 Thread Josh Elser 
tl;dr Infra is upgrading Jenkins in 2 weeks and Java7 Maven jobs 
may/may-not work after this. See explanation below from [1]:



Users with jobs configured with the "Maven project" type may not be able 
to use Java 7 for their Maven jobs. The correct behavior is not 
guaranteed so proceed at your own risk. The Maven Project uses Jenkins 
Remoting to establish "interceptors" within the Maven executable. 
Because of this, Maven uses Remoting and other Jenkins core classes, and 
this behavior may break an update.



[1] https://jenkins.io/blog/2017/04/10/jenkins-has-upgraded-to-java-8/

 Forwarded Message 
Subject: [JENKINS]  [IMPORTANT] - Jenkins Migration and Upgrade (And 
JDK7 deprecation)

Date: Tue, 27 Jun 2017 17:03:13 +1000
From: Gavin McDonald 
Reply-To: bui...@apache.org, bui...@apache.org
To: bui...@apache.org
CC: ASF Operations 

ASF Jenkins Master Migration and Upgrade on :-


Location	Local Time	 
   Time Zone	UTC Offset
Melbourne (Australia - Victoria)	Sunday, 16 July 2017 at 10:00:00 am 
AEST	UTC+10 hours
New York (USA - New York)	Saturday, 15 July 2017 at 8:00:00 pm 
EDT	UTC-4 hours

Corresponding UTC (GMT) Sunday, 16 July 2017 at 00:00:00

Hi All,

A few things are going to happen in just over 2 weeks.

1. Migration of Jenkins to a new host. A Jenkins Master module and yaml 
have been puppetized and ready to go.
What we need to do to migrate the Master away from its current host 
is turn off the old service. Perform a final rsync of data and 
perform the migration tasks.

As we intend to preserve history for jobs this will take some time.
At the same time as doing this migration to a new host, all slave 
connections will be updated (see below.)
I have no current estimate of downtime, but it will run into 
several hours. We do plan to run this migration on a Sunday at the 
lowest part of Jenkins usual usage.


2. Upgrade of Jenkins - Jenkins project released a new LTS release, 
version 2.60.1. This is a major release and breaks Jenkins in terms 
of Maven jobs for JDK 7 in the same way that it happened for Maven and 
JDK 6 a few months back.


The infra team (mainly myself) got quite some feedback on not 
supplying advance notice of this breakage. That upgrade however was 
necessary due to security fixes that required our upgrade.  This email 
serves as advance warning of the upcoming upgrade of Jenkins, the 
downtime due to the migration of the service to a new host; and notice 
of the breakage to JDK 7 that the upgrade brings.


Please familiarise yourself with the Jenkins LTS upgrade notes at [1].
In particular please note:-

“…2.60.1 is the first Jenkins LTS release that requires Java 8 to 
run. If you're using the Maven Project type, please note that it needs 
to use a JDK capable of running Jenkins, i.e. JDK 8 or up. If you 
configure an older JDK in a Maven Project, Jenkins will attempt to find 
a newer JDK and use that automatically. If your SSH Slaves fail to start 
and you have the plugin install the JRE to run them, make sure to update 
SSH Slaves Plugin to at least version 1.17 (1.20 recommended).

Changes since 2.60:
Fix for NullPointerException while initiating some SSH connections 
(regression in 2.59). (issue 44120 
)

Notable changes since 2.46.3:
Jenkins (master and agents) now requires Java 8 to run. (issue 27624 
 <>, issue 42709 
 <>, pull 2802 
, announcement blog post 
)


…”

There are over 30 other enhancements/fixes since 2.46.2 which we 
currently run so please do take a note of those.


Recap: In just over 2 weeks, downtime for a migration AND upgrade is 
planned.
Please do not rely on Jenkins at all for that weekend if you use it in 
your release workflow.


Please do take this notice back to your dev lists.
Any questions or concerns please email back to bui...@apache.org 
 only.

Thanks

Gav…

[1] - https://jenkins.io/changelog-stable/

[jira] [Created] (HBASE-18275) formatting and grammar mistakes in schemadoc chapter

2017-06-27 Thread Artem Ervits (JIRA) 
Artem Ervits created HBASE-18275:


 Summary: formatting and grammar mistakes in schemadoc chapter
 Key: HBASE-18275
 URL: https://issues.apache.org/jira/browse/HBASE-18275
 Project: HBase
  Issue Type: Improvement
  Components: documentation
Affects Versions: 2.0.0-alpha-1, 1.1.11, 1.2.6
Reporter: Artem Ervits
Priority: Trivial
 Fix For: 3.0.0, 2.0.0-alpha-2


a grammatical error and formatting error in schema design doc.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [JENKINS] [IMPORTANT] - Jenkins Migration and Upgrade (And JDK7 deprecation)

2017-06-27 Thread Sean Busbey 
We already ignore ASF infra's jdk and rely on openjdk7 via docker for
precommit on branch-1 and earlier.

Sounds like we'll need to update our nightly runs to similarly rely on
docker, which is a pretty good reason to switch to yetus' nightly check
mode.

This is what, like 2 weeks notice?

On Tue, Jun 27, 2017 at 9:49 AM, Mike Drob  wrote:

> Gavin of the infra team just sent this missive over.
>
> It looks like JDK 7 will no longer be available. Do we need to do anything
> this affect our branch-1, or are we sufficiently confident that JDK8 with
> compatibility mode targeting 1.7 will be enough for our needs?
>
> Mike
>
> -- Forwarded message --
> From: Gavin McDonald 
> Date: Tue, Jun 27, 2017 at 2:03 AM
> Subject: [JENKINS] [IMPORTANT] - Jenkins Migration and Upgrade (And JDK7
> deprecation)
> To: bui...@apache.org
> Cc: ASF Operations 
>
>
> ASF Jenkins Master Migration and Upgrade on :-
>
>
> LocationLocal Time
> Time Zone   UTC Offset
> Melbourne (Australia - Victoria)Sunday, 16 July 2017 at 10:00:00
> am AESTUTC+10 hours
> New York (USA - New York)   Saturday, 15 July 2017 at 8:00:00
> pmEDT UTC-4 hours
> Corresponding UTC (GMT) Sunday, 16 July 2017 at 00:00:00
>
>
> Hi All,
>
> A few things are going to happen in just over 2 weeks.
>
> 1. Migration of Jenkins to a new host. A Jenkins Master module and yaml
> have been puppetized and ready to go.
> What we need to do to migrate the Master away from its current host is
> turn off the old service. Perform a final
> rsync of data and perform the migration tasks.
>
> As we intend to preserve history for jobs this will take some time.
> At the same time as doing this migration to a new host, all slave
> connections will be updated (see below.)
> I have no current estimate of downtime, but it will run into several
> hours. We do plan to run this migration on a
> Sunday at the lowest part of Jenkins usual usage.
>
> 2. Upgrade of Jenkins - Jenkins project released a new LTS release, version
> 2.60.1. This is a major release and breaks
> Jenkins in terms of Maven jobs for JDK 7 in the same way that it
> happened for Maven and JDK 6 a few months back.
>
> The infra team (mainly myself) got quite some feedback on not supplying
> advance notice of this breakage. That upgrade
> however was necessary due to security fixes that required our upgrade.
> This email serves as advance warning of the
> upcoming upgrade of Jenkins, the downtime due to the migration of the
> service to a new host; and notice of the breakage
> to JDK 7 that the upgrade brings.
>
> Please familiarise yourself with the Jenkins LTS upgrade notes at [1].
> In particular please note:-
>
> “…2.60.1 is the first Jenkins LTS release that requires Java 8 to run.
> If you're using the Maven Project type, please note that it needs to use a
> JDK capable of running Jenkins, i.e. JDK 8 or up. If you configure an older
> JDK in a Maven Project, Jenkins will attempt to find a newer JDK and use
> that automatically. If your SSH Slaves fail to start and you have the
> plugin install the JRE to run them, make sure to update SSH Slaves Plugin
> to at least version 1.17 (1.20 recommended).
> Changes since 2.60:
> Fix for NullPointerException while initiating some SSH connections
> (regression in 2.59). (issue 44120  org/browse/JENKINS-44120>)
> Notable changes since 2.46.3:
> Jenkins (master and agents) now requires Java 8 to run. (issue 27624 <
> https://issues.jenkins-ci.org/browse/JENKINS-27624> <>, issue 42709 <
> https://issues.jenkins-ci.org/browse/JENKINS-42709> <>, pull 2802 <
> https://github.com/jenkinsci/jenkins/pull/2802>, announcement blog post <
> https://jenkins.io/blog/2017/04/10/jenkins-has-upgraded-to-java-8/>)
>
> …”
>
> There are over 30 other enhancements/fixes since 2.46.2 which we currently
> run so please do take a note of those.
>
> Recap: In just over 2 weeks, downtime for a migration AND upgrade is
> planned.
>
> Please do not rely on Jenkins at all for that weekend if you use it in your
> release workflow.
>
> Please do take this notice back to your dev lists.
>
> Any questions or concerns please email back to bui...@apache.org  bui...@apache.org> only.
>
> Thanks
>
> Gav…
>
> [1] - https://jenkins.io/changelog-stable/
>

Fwd: [JENKINS] [IMPORTANT] - Jenkins Migration and Upgrade (And JDK7 deprecation)

2017-06-27 Thread Mike Drob 
Gavin of the infra team just sent this missive over.

It looks like JDK 7 will no longer be available. Do we need to do anything
this affect our branch-1, or are we sufficiently confident that JDK8 with
compatibility mode targeting 1.7 will be enough for our needs?

Mike

-- Forwarded message --
From: Gavin McDonald 
Date: Tue, Jun 27, 2017 at 2:03 AM
Subject: [JENKINS] [IMPORTANT] - Jenkins Migration and Upgrade (And JDK7
deprecation)
To: bui...@apache.org
Cc: ASF Operations 


ASF Jenkins Master Migration and Upgrade on :-


LocationLocal Time
Time Zone   UTC Offset
Melbourne (Australia - Victoria)Sunday, 16 July 2017 at 10:00:00
am AESTUTC+10 hours
New York (USA - New York)   Saturday, 15 July 2017 at 8:00:00
pmEDT UTC-4 hours
Corresponding UTC (GMT) Sunday, 16 July 2017 at 00:00:00


Hi All,

A few things are going to happen in just over 2 weeks.

1. Migration of Jenkins to a new host. A Jenkins Master module and yaml
have been puppetized and ready to go.
What we need to do to migrate the Master away from its current host is
turn off the old service. Perform a final
rsync of data and perform the migration tasks.

As we intend to preserve history for jobs this will take some time.
At the same time as doing this migration to a new host, all slave
connections will be updated (see below.)
I have no current estimate of downtime, but it will run into several
hours. We do plan to run this migration on a
Sunday at the lowest part of Jenkins usual usage.

2. Upgrade of Jenkins - Jenkins project released a new LTS release, version
2.60.1. This is a major release and breaks
Jenkins in terms of Maven jobs for JDK 7 in the same way that it
happened for Maven and JDK 6 a few months back.

The infra team (mainly myself) got quite some feedback on not supplying
advance notice of this breakage. That upgrade
however was necessary due to security fixes that required our upgrade.
This email serves as advance warning of the
upcoming upgrade of Jenkins, the downtime due to the migration of the
service to a new host; and notice of the breakage
to JDK 7 that the upgrade brings.

Please familiarise yourself with the Jenkins LTS upgrade notes at [1].
In particular please note:-

“…2.60.1 is the first Jenkins LTS release that requires Java 8 to run.
If you're using the Maven Project type, please note that it needs to use a
JDK capable of running Jenkins, i.e. JDK 8 or up. If you configure an older
JDK in a Maven Project, Jenkins will attempt to find a newer JDK and use
that automatically. If your SSH Slaves fail to start and you have the
plugin install the JRE to run them, make sure to update SSH Slaves Plugin
to at least version 1.17 (1.20 recommended).
Changes since 2.60:
Fix for NullPointerException while initiating some SSH connections
(regression in 2.59). (issue 44120 )
Notable changes since 2.46.3:
Jenkins (master and agents) now requires Java 8 to run. (issue 27624 <
https://issues.jenkins-ci.org/browse/JENKINS-27624> <>, issue 42709 <
https://issues.jenkins-ci.org/browse/JENKINS-42709> <>, pull 2802 <
https://github.com/jenkinsci/jenkins/pull/2802>, announcement blog post <
https://jenkins.io/blog/2017/04/10/jenkins-has-upgraded-to-java-8/>)

…”

There are over 30 other enhancements/fixes since 2.46.2 which we currently
run so please do take a note of those.

Recap: In just over 2 weeks, downtime for a migration AND upgrade is
planned.

Please do not rely on Jenkins at all for that weekend if you use it in your
release workflow.

Please do take this notice back to your dev lists.

Any questions or concerns please email back to bui...@apache.org  only.

Thanks

Gav…

[1] - https://jenkins.io/changelog-stable/

[jira] [Created] (HBASE-18274) hbase autorestart will overwirte the gc log

2017-06-27 Thread Fangyuan Deng (JIRA) 
Fangyuan Deng created HBASE-18274:
-

 Summary: hbase autorestart will overwirte the gc log
 Key: HBASE-18274
 URL: https://issues.apache.org/jira/browse/HBASE-18274
 Project: HBase
  Issue Type: Bug
  Components: hbase
Affects Versions: 2.0.0-alpha-1, 1.1.11, 1.2.6
Reporter: Fangyuan Deng


While using hbase autorestart ,  the gc log will be overwirited after the 
process(hmaster or hregionserver) retarting.

This is because the autorestart loop is in internal_autostart function ( in 
hbase-daemon.sh), but we only rotate the gc log in autorestart function.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18273) hbase_rotate_log in hbase script not working for some JDK

2017-06-27 Thread Fangyuan Deng (JIRA) 
Fangyuan Deng created HBASE-18273:
-

 Summary: hbase_rotate_log in hbase script not working for some JDK
 Key: HBASE-18273
 URL: https://issues.apache.org/jira/browse/HBASE-18273
 Project: HBase
  Issue Type: Bug
  Components: hbase
Affects Versions: 2.0.0-alpha-1, 1.1.11, 1.2.6
Reporter: Fangyuan Deng
Priority: Minor


When restarting a hbase process,  hbase_rotate_log $HBASE_LOGGC will rotate GC 
logs.
the code looks like this,

 if [ -f "$log" ]; then # rotate logs
while [ $num -gt 1 ]; do
prev=`expr $num - 1`
[ -f "$log.$prev" ] && mv -f "$log.$prev" "$log.$num"
num=$prev
done

But, some version JDK will add a postfix .0 to the gc file, like 
hbase-xxx.gc.0,  rather than hbase-xxx.gc.

So I add a check before rotate,

 if [ ! -f "$log" ]; then #for some jdk, gc log has a postfix 0
  if [ -f "$log.0" ]; then
mv -f "$log.0" "$log";
  fi
fi




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-17846) [JDK8] Use Optional instead of Nullable parameter in async client

2017-06-27 Thread Guanghao Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-17846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang resolved HBASE-17846.

Resolution: Duplicate

As HBASE-18234 has been resloved, this can be closed, too.

> [JDK8] Use Optional instead of Nullable parameter in async client
> -
>
> Key: HBASE-17846
> URL: https://issues.apache.org/jira/browse/HBASE-17846
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 2.0.0
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>
> For master branch, we use a lot of java 8 features in async client, like 
> lambda, stream, default method and so on. And java 8 support Optional, we can 
> update some method to use Optional instead of Nullable parameter. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)