After some googling, I discovered we need the sign profile to make deploy work:
mvn deploy -Psign -DskipTests -DskipShading Updated HowToRelease accordingly. On Fri, Jan 20, 2017 at 11:15 AM, Andrew Wang <andrew.w...@cloudera.com> wrote: > What I meant is that you can mvn deploy the specific missing files to fix > up the repo state. Though we should also figure out why the deploy plugin > didn't pick up the signature files the first time. > > Best, > Andrew > > On Fri, Jan 20, 2017 at 11:12 AM, Junping Du <j...@hortonworks.com> wrote: > >> Yes. I did maven deploy in root directory before close the staging >> repository. If this is the only suspect, I can drop the repository and do >> mvn deploy again. >> >> >> Thanks, >> >> >> Junping >> ------------------------------ >> *From:* Andrew Wang <andrew.w...@cloudera.com> >> *Sent:* Friday, January 20, 2017 10:48 AM >> *To:* Junping Du >> *Cc:* common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; >> mapreduce-...@hadoop.apache.org; yarn-dev@hadoop.apache.org >> >> *Subject:* Re: [Continued] [Release thread] 2.8.0 release activities >> >> You can check the error message by clicking on it, a bunch like this: >> >> Missing Signature: '/org/apache/hadoop/hadoop-map >> reduce-client-jobclient/2.8.0/hadoop-mapreduce-client-jobclient-2.8.0-tests.jar.asc' >> does not exist for 'hadoop-mapreduce-client-jobclient-2.8.0-tests.jar'. >> >> Did you maven deploy all the signature files? >> >> On Fri, Jan 20, 2017 at 2:04 AM, Junping Du <j...@hortonworks.com> wrote: >> >>> Hi, >>> I have successfully built the release bit on branch-2.8.0 by >>> following: https://wiki.apache.org/hadoop/HowToRelease step by step. >>> However, when try to close the "Staging Repositories" at Nexus page ( >>> https://repository.apache.org/#stagingRepositories), I found our >>> repository - orgapachehadoop-1051 cannot be closed due to some signature >>> validation failed and one Apache rule failed. I never met this problem >>> before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent >>> changes on release process/tools (docker based)? Any ideas or thoughts on >>> how to fix the problem? >>> >>> Thanks, >>> >>> Junping >>> >>> ________________________________________ >>> From: Junping Du <j...@hortonworks.com> >>> Sent: Thursday, January 19, 2017 6:46 PM >>> To: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; >>> mapreduce-...@hadoop.apache.org; yarn-dev@hadoop.apache.org >>> Cc: Varun Vasudev >>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities >>> >>> According to Varun's offline email, the security fixes has landed on >>> branch-2, 2.8 and 2.8.0 branch. >>> I was kicking off a new RC build (RC1), and will publish it for vote >>> soon. In the mean time, please mark fix version as 2.8.1 for any new >>> commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at >>> this moment. Thanks! >>> >>> Cheers, >>> >>> Junping >>> >>> ________________________________________ >>> From: Junping Du <j...@hortonworks.com> >>> Sent: Wednesday, January 18, 2017 3:26 PM >>> To: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; >>> mapreduce-...@hadoop.apache.org; yarn-dev@hadoop.apache.org >>> Cc: Varun Vasudev >>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities >>> >>> Hi folks, >>> In the passed one or two weeks, we found some new blockers get >>> coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM >>> restart with work preserving enabled) and YARN-6072 (RM unable to start in >>> secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I >>> fixed YARN-6068), and I was starting the RC build process since early this >>> week. As we have significant build tools/process change (docker based) >>> since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with >>> it and finally get a successful build on 2.8.0-RC0 last night. >>> I already push RC0 tag into public which is prerequisite step >>> before RC voting. However, in the mean while, I was pinged by Varun >>> Vasudev that there are a known vulnerability issues on container_executor >>> get identified and discussed in hadoop security email threads - looks like >>> YARN-5704 fixed part of it, but left part - the privilege escalation via >>> /proc/self/environ is not fixed yet. So most likely, I have to withdraw our >>> 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait >>> this issue get fixed to prepare a new release candidate. As RC0 tag cannot >>> be reverted after push into apache, our next release candidate will start >>> from RC1. >>> As I mentioned in early email, 2.8.0 is a very big release (2300+ >>> commits since 2.7.3) and I am glad that we are almost there. Thanks >>> everyone for being patient and contributing our release work. Please let me >>> know if you have more comments or suggestions. >>> >>> Thanks, >>> >>> Junping >>> ________________________________________ >>> From: Junping Du <j...@hortonworks.com> >>> Sent: Wednesday, January 04, 2017 1:41 AM >>> To: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; >>> mapreduce-...@hadoop.apache.org; yarn-dev@hadoop.apache.org >>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities >>> >>> Hi all Hadoopers, >>> I just commit YARN-3866 which is the last blocker for branch-2.8, >>> so since tomorrow I will kick off process to prepare the first RC for our >>> 2.8.0 release. This release will include at least 2374 commits that never >>> landed before in previous 2.x releases. Please check >>> https://s.apache.org/RW5k for details. There are also 352 fixes marked >>> in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 ( >>> https://s.apache.org/RKti) that I need to double check all are landed >>> in branch-2.8 and fix the versions before kicking off the first RC. >>> Our progress is slightly behind my estimation weeks ago. However, >>> considering we just go through holidays and Jenkins cleanup issues linger >>> on for several projects (YARN, etc.), our achievement here is still great! >>> Thanks everyone for keep calm and carry on the help for the release. Kudos >>> to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on >>> blockers during this time. Also, special thanks to Vinod and Andrew for >>> sharing knowledge and practice (include scripts for auto version check) for >>> releasing effort. >>> Will try to prepare RC0 of 2.8 release for vote within this week. >>> Stay tuned! >>> >>> Thanks, >>> >>> Junping >>> >>> ________________________________________ >>> From: Junping Du <j...@hortonworks.com> >>> Sent: Wednesday, December 07, 2016 11:31 AM >>> To: Akira Ajisaka; common-...@hadoop.apache.org; >>> hdfs-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org; >>> yarn-dev@hadoop.apache.org >>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities >>> >>> Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well >>> for several runs before last weekend, but it get failed for latest several >>> runs, probably affected by recently Jenkins down. >>> However, from my recent manual kick off, it seems to be good again: >>> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. >>> I will keep an eye for today's nightly run to see if it back to normal. >>> >>> >>> Thanks, >>> >>> Junping >>> >>> ________________________________________ >>> From: Akira Ajisaka <ajisa...@apache.org> >>> Sent: Wednesday, December 07, 2016 12:12 AM >>> To: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; >>> mapreduce-...@hadoop.apache.org; yarn-dev@hadoop.apache.org >>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities >>> >>> Thanks Junping and Andrew! >>> >>> HADOOP-2.8-JACC is not working well, so I manually kicked a job to >>> compare 2.8 with 2.7.3. >>> >>> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-tr >>> unk-JACC/24/artifact/target/compat-check/report.html >>> >>> Regards, >>> Akira >>> >>> On 2016/12/02 2:08, Junping Du wrote: >>> > Thanks Andrew! That's also a nice suggestion. I already create a >>> similar job: https://builds.apache.org/view >>> /H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs >>> manually. Will monitor incompatible status from there. >>> > >>> > >>> > Thanks, >>> > >>> > >>> > Junping >>> > >>> > >>> > ________________________________ >>> > From: Andrew Wang <andrew.w...@cloudera.com> >>> > Sent: Wednesday, November 30, 2016 4:18 PM >>> > To: Junping Du >>> > Cc: Sangjin Lee; common-...@hadoop.apache.org; >>> hdfs-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org; >>> yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; >>> sj...@twitter.com >>> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities >>> > >>> > I recommend giving the JACC report another look. I set up a >>> parameterized jenkins job which you can trigger manually for branch-2.8: >>> > >>> > https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/ >>> > >>> > On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <j...@hortonworks.com >>> <mailto:j...@hortonworks.com>> wrote: >>> > Hi Sangjin and all, >>> > >>> > That sounds good. If anyone have priority fix to backport , >>> please nominate it by following this email thread or ping me on JIRA >>> directly. And I agree that we should keep in mind that our bar for 2.8 >>> release should be high now as we want to move quickly on this release. >>> Non-critical fixes can wait for next one. >>> > >>> > Any other comments and suggestions? >>> > >>> > >>> > Thanks, >>> > >>> > >>> > Junping >>> > >>> > >>> > ________________________________ >>> > From: sjl...@gmail.com<mailto:sjl...@gmail.com> <sjl...@gmail.com >>> <mailto:sjl...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org >>> <mailto:sj...@apache.org>> >>> > Sent: Wednesday, November 30, 2016 3:24 PM >>> > To: Junping Du >>> > Cc: common-...@hadoop.apache.org<mailto:common-...@hadoop.apache.org>; >>> hdfs-...@hadoop.apache.org<mailto:hdfs-...@hadoop.apache.org>; >>> mapreduce-...@hadoop.apache.org<mailto:mapreduce-...@hadoop.apache.org>; >>> yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org>; Vinod >>> Vavilapalli; Jian He; Wangda Tan; sj...@twitter.com<mailto:sjlee >>> @twitter.com> >>> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities >>> > >>> > Thanks for picking this up Junping! >>> > >>> > I heard on email threads and offline discussions that there may be >>> interests in porting some fixes from branch-2 to branch-2.8. I still feel >>> that the bar should be pretty high to do that, but it might be good to have >>> folks suggest some critical fixes that need to be released sooner. Thoughts? >>> > >>> > On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <j...@hortonworks.com >>> <mailto:j...@hortonworks.com><mailto:j...@hortonworks.com<mailto:jdu@hor >>> tonworks.com>>> wrote: >>> > Hi folks, >>> > Per asked by Vinod offline, I would like to continue the effort >>> to push 2.8 release out in next several weeks. The plan is as following: >>> > - We only have 5 blockers and 3 critical issues ( >>> https://s.apache.org/6kwx) so far, and most of them are in good >>> progress now. I put them all on apache ciwiki ( >>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) >>> and will do daily monitor and update on these issues in next couple of >>> weeks. >>> > - When all blocker/critical issues are gone (target EOD is Dec. >>> 10), I will push out all major/minor issues that is open. So if you are >>> working on some major issues and target for release in 2.8, please keep >>> track of our release status and your work progress. >>> > - I will create RC immediately once our 2.8 target issues are >>> clean (especially for blocker and critical issues) for voting process. >>> > >>> > I hope we can make it within year 2016, best before Xmas holiday. >>> And I need help from all of you. :) >>> > >>> > >>> > Thanks, >>> > >>> > Junping >>> > >>> > ________________________________________ >>> > From: Vinod Kumar Vavilapalli <vino...@apache.org<mailto:vin >>> o...@apache.org><mailto:vino...@apache.org<mailto:vino...@apache.org>>> >>> > Sent: Monday, July 25, 2016 2:57 PM >>> > To: Jian He >>> > Cc: mapreduce-...@hadoop.apache.org<mailto:mapreduce-dev@hadoop. >>> apache.org><mailto:mapreduce-...@hadoop.apache.org<mailto:ma >>> preduce-...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<mailto: >>> yarn-dev@hadoop.apache.org><mailto:yarn-dev@hadoop.apache.org<mailto: >>> yarn-dev@hadoop.apache.org>>; hdfs-...@hadoop.apache.org<mailto: >>> hdfs-...@hadoop.apache.org><mailto:hdfs-...@hadoop.apache.org<mailto: >>> hdfs-...@hadoop.apache.org>>; common-...@hadoop.apache.org<mailto: >>> common-...@hadoop.apache.org><mailto:common-...@hadoop.apache.org >>> <mailto:common-...@hadoop.apache.org>> >>> > Subject: Re: [Release thread] 2.8.0 release activities >>> > >>> > Thanks for all that great help moving this release forward, Jian, >>> Sangjin, Wangda et al! Appreciate it. >>> > >>> > The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 >>> last week. >>> > >>> > I only see 9 blocker / critical fixes pending for 2.8.0 - >>> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this! >>> > >>> > Thanks >>> > +Vinod >>> > >>> >> On May 11, 2016, at 6:04 PM, Jian He <j...@hortonworks.com<mailto:jh >>> e...@hortonworks.com><mailto:j...@hortonworks.com<mailto:j...@hortonworks.com>>> >>> wrote: >>> >> >>> >> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs >>> attention for 2.8 >>> >> >>> >> MAPREDUCE-6288 >>> >> YARN-1815 >>> >> YARN-4685 >>> >> YARN-4844 >>> >> >>> >> The rest are either improvements or long-standing issues and does not >>> qualify release blocker, IMO. >>> >> I think we’ll try to get these 4 jiras in asap. The rest will be on >>> best effort, resolve as much as possible and move them out if not resolved >>> in time. >>> >> >>> >> Jian >>> >> >>> >> On May 11, 2016, at 5:37 PM, Wangda Tan <wheele...@gmail.com<mailto: >>> wheele...@gmail.com><mailto:wheele...@gmail.com<mailto:wheelea >>> s...@gmail.com>><mailto:wheele...@gmail.com<mailto:wheele...@gmail.com >>> ><mailto:wheele...@gmail.com<mailto:wheele...@gmail.com>>>> wrote: >>> >> >>> >> Sounds good to me :). >>> >> >>> >> Jian and I have looked at all existing 2.8.0 blockers and criticals >>> today. >>> >> To me more than half of MR/YARN blockers/criticals of 2.8 should be >>> moved >>> >> out. Left comments on these JIRAs asked original owners, plan to >>> update >>> >> target version of these JIRAs early next week. >>> >> >>> >> Will keep this thread updated. >>> >> >>> >> Thanks, >>> >> Wangda >>> >> >>> >> >>> >> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org >>> <mailto:sj...@apache.org><mailto:sj...@apache.org<mailto: >>> sj...@apache.org>><mailto:sj...@apache.org<mailto:sj...@apache.org >>> ><mailto:sj...@apache.org<mailto:sj...@apache.org>>>> wrote: >>> >> >>> >> How about this? I'll review the HADOOP/HDFS bugs in that list to come >>> up >>> >> with true blockers for 2.8.0 or JIRAs that are close to being ready. >>> I'll >>> >> report the list here. Then folks can chime in if you agree >>> >> >>> >> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan? >>> >> >>> >> Thanks, >>> >> Sangjin >>> >> >>> >> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheele...@gmail.com >>> <mailto:wheele...@gmail.com><mailto:wheele...@gmail.com<mailto:wheelea >>> s...@gmail.com>><mailto:wheele...@gmail.com<mailto:wheele...@gmail.com >>> ><mailto:wheele...@gmail.com<mailto:wheele...@gmail.com>>>> wrote: >>> >> >>> >> +1, we should close such staled JIRAs to avoid doing unnecessary >>> checks >>> >> for >>> >> every releases. >>> >> >>> >> I'm working on reviewing YARN/MR critical/blocker patches currently, >>> it >>> >> gonna very helpful if someone else can help with reviewing Common/HDFS >>> >> JIRAs. >>> >> >>> >> Thanks, >>> >> Wangda >>> >> >>> >> >>> >> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org >>> <mailto:sj...@apache.org><mailto:sj...@apache.org<mailto: >>> sj...@apache.org>><mailto:sj...@apache.org<mailto:sj...@apache.org >>> ><mailto:sj...@apache.org<mailto:sj...@apache.org>>>> wrote: >>> >> >>> >> Where do we stand in terms of closing out blocker/critical issues for >>> >> 2.8.0? I still see 50 open JIRAs in Vinod's list: >>> >> https://issues.apache.org/jira/issues/?filter=12334985 >>> >> >>> >> But I see a lot of JIRAs with no patches or very stale patches. It >>> >> would be >>> >> a good exercise to come up with the list of JIRAs that we need to >>> block >>> >> 2.8.0 for and focus our attention on closing them out. Thoughts? >>> >> >>> >> Thanks, >>> >> Sangjin >>> >> >>> >> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran < >>> ste...@hortonworks.com<mailto:ste...@hortonworks.com><mailto: >>> ste...@hortonworks.com<mailto:ste...@hortonworks.com>><mailto: >>> ste...@hortonworks.com<mailto:ste...@hortonworks.com><mailto: >>> ste...@hortonworks.com<mailto:ste...@hortonworks.com>>> >>> >> >>> >> wrote: >>> >> >>> >> >>> >> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli < >>> >> vino...@apache.org<mailto:vino...@apache.org><mailto:vinodkv >>> @apache.org<mailto:vino...@apache.org>><mailto:vino...@apache.org >>> <mailto:vino...@apache.org><mailto:vino...@apache.org<mailto: >>> vino...@apache.org>>>> >>> >> wrote: >>> >> >>> >> We are not converging - there’s still 58 more. I need help from the >>> >> community in addressing / review 2.8.0 blockers. If folks can start >>> >> with >>> >> reviewing Patch available tickets, that’ll be great. >>> >> >>> >> >>> >> >>> >> >>> >> I'm still doing the s3a stuff, other people testing and reviewing this >>> >> stuff welcome. >>> >> >>> >> in particular, I could do with others playing with this patch of mine, >>> >> which adds counters and things into S3a, based on the azure >>> >> instrumentation >>> >> >>> >> https://issues.apache.org/jira/browse/HADOOP-13028 >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> > >>> > >>> > --------------------------------------------------------------------- >>> > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org<mailto: >>> yarn-dev-unsubscr...@hadoop.apache.org><mailto: >>> yarn-dev-unsubscr...@hadoop.apache.org<mailto:yarn-dev- >>> unsubscr...@hadoop.apache.org>> >>> > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org >>> <mailto:yarn-dev-h...@hadoop.apache.org><mailto:yarn-dev-he >>> l...@hadoop.apache.org<mailto:yarn-dev-h...@hadoop.apache.org>> >>> > >>> > >>> > --------------------------------------------------------------------- >>> > To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org >>> <mailto:mapreduce-dev-unsubscr...@hadoop.apache.org><mailto: >>> mapreduce-dev-unsubscr...@hadoop.apache.org<mailto: >>> mapreduce-dev-unsubscr...@hadoop.apache.org>> >>> > For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org >>> <mailto:mapreduce-dev-h...@hadoop.apache.org><mailto:m >>> apreduce-dev-h...@hadoop.apache.org<mailto:mapreduce-dev- >>> h...@hadoop.apache.org>> >>> > >>> > >>> > >>> > >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org >>> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org >>> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org >>> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org >>> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org >>> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org >>> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org >>> >>> >> >
