[jira] [Created] (HADOOP-13998) initial s3guard preview
Steve Loughran created HADOOP-13998: --- Summary: initial s3guard preview Key: HADOOP-13998 URL: https://issues.apache.org/jira/browse/HADOOP-13998 Project: Hadoop Common Issue Type: Sub-task Reporter: Steve Loughran JIRA to link in all the things we think are needed for a preview/merge into trunk -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [jira] [Created] (HADOOP-13998) initial s3guard preview
Mmm On Wed, Jan 18, 2017, 7:27 PM Steve Loughran (JIRA) wrote: > Steve Loughran created HADOOP-13998: > --- > > Summary: initial s3guard preview > Key: HADOOP-13998 > URL: https://issues.apache.org/jira/browse/HADOOP-13998 > Project: Hadoop Common > Issue Type: Sub-task > Reporter: Steve Loughran > > > JIRA to link in all the things we think are needed for a preview/merge > into trunk > > > > -- > This message was sent by Atlassian JIRA > (v6.3.4#6332) > > - > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > >
shading killing trunk build times
a clean build with no tests, javadocs, is now taking 7+ min locally, with most of that time going into the packaging work [INFO] Apache Hadoop Client API ... SUCCESS [01:50 min] [INFO] Apache Hadoop Client Runtime ... SUCCESS [01:19 min] [INFO] Apache Hadoop Client Packaging Invariants .. SUCCESS [ 0.332 s] [INFO] Apache Hadoop Client Test Minicluster .. SUCCESS [02:52 min] This really hurts when you are trying to build and test across modules, where I want to do a change in say, a hadoop-common test, then run that test in hdfs and aws modules. I don't want or need this client stuff, and all it is doing is taking up time and forcing me into contrived bits where I have to explicitly only build the modules I want, which gets dangerous once you start switching branches Can we make these a profile? Even if it's on by default and something that those of us try - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/290/ [Jan 17, 2017 7:05:28 AM] (arp) HDFS-11339. Fix import. [Jan 17, 2017 8:23:51 AM] (aajisaka) HADOOP-13989. Fix typo in hadoop-client shade configuration. Contributed [Jan 17, 2017 8:46:00 AM] (bibinchundatt) YARN-6057. yarn.scheduler.minimum-allocation-* descriptions are [Jan 17, 2017 8:56:10 AM] (aajisaka) MAPREDUCE-6819. Replace UTF8 with Text in MRBench. Contributed by Peter [Jan 17, 2017 4:01:42 PM] (jlowe) MAPREDUCE-6831. Flaky test TestJobImpl.testKilledDuringKillAbort. [Jan 17, 2017 8:55:47 PM] (templedf) YARN-6071. Fix incompatible API change on AM-RM protocol due to [Jan 17, 2017 9:10:24 PM] (kihwal) HADOOP-13976. Path globbing does not match newlines. Contributed by Eric [Jan 17, 2017 9:56:06 PM] (wang) HADOOP-13978. Update project release notes for 3.0.0-alpha2. Contributed [Jan 17, 2017 10:33:26 PM] (xyao) HDFS-11209. SNN can't checkpoint when rolling upgrade is not finalized. [Jan 17, 2017 10:48:03 PM] (subru) YARN-6016. Fix minor bugs in handling of local AMRMToken in AMRMProxy. [Jan 18, 2017 12:23:38 AM] (wang) Revert "HADOOP-13989. Fix typo in hadoop-client shade configuration. [Jan 18, 2017 1:01:31 AM] (kasha) YARN-5831. FairScheduler: Propagate allowPreemptionFrom flag all the way -1 overall The following subsystems voted -1: asflicense unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.yarn.server.timeline.webapp.TestTimelineWebServices hadoop.yarn.server.resourcemanager.TestRMRestart hadoop.yarn.server.TestContainerManagerSecurity hadoop.yarn.server.TestMiniYarnClusterNodeUtilization hadoop.yarn.client.api.impl.TestAMRMProxy cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/290/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/290/artifact/out/diff-compile-javac-root.txt [160K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/290/artifact/out/diff-checkstyle-root.txt [16M] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/290/artifact/out/diff-patch-pylint.txt [20K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/290/artifact/out/diff-patch-shellcheck.txt [24K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/290/artifact/out/diff-patch-shelldocs.txt [16K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/290/artifact/out/whitespace-eol.txt [11M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/290/artifact/out/whitespace-tabs.txt [1.3M] javadoc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/290/artifact/out/diff-javadoc-javadoc-root.txt [2.2M] unit: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/290/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt [12K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/290/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt [56K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/290/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt [324K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/290/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt [16K] asflicense: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/290/artifact/out/patch-asflicense-problems.txt [4.0K] Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13999) Add maven profile to dissable jar shading to reduce compile time
Arun Suresh created HADOOP-13999: Summary: Add maven profile to dissable jar shading to reduce compile time Key: HADOOP-13999 URL: https://issues.apache.org/jira/browse/HADOOP-13999 Project: Hadoop Common Issue Type: Bug Reporter: Arun Suresh Assignee: Arun Suresh Adding a maven profile to disable client jar shading -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
ApacheCon CFP closing soon (11 February)
Hello, fellow Apache enthusiast. Thanks for your participation, and interest in, the projects of the Apache Software Foundation. I wanted to remind you that the Call For Papers (CFP) for ApacheCon North America, and Apache: Big Data North America, closes in less than a month. If you've been putting it off because there was lots of time left, it's time to dig for that inspiration and get those talk proposals in. It's also time to discuss with your developer and user community whether there's a track of talks that you might want to propose, so that you have more complete coverage of your project than a talk or two. We're looking for talks directly, and indirectly, related to projects at the Apache Software Foundation. These can be anything from in-depth technical discussions of the projects you work with, to talks about community, documentation, legal issues, marketing, and so on. We're also very interested in talks about projects and services built on top of Apache projects, and case studies of how you use Apache projects to solve real-world problems. We are particularly interested in presentations from Apache projects either in the Incubator, or recently graduated. ApacheCon is where people come to find out what technology they'll be using this time next year. Important URLs are: To submit a talk for Apache: Big Data - http://events.linuxfoundation.org/events/apache-big-data-north-america/program/cfp To submit a talk for ApacheCon - http://events.linuxfoundation.org/events/apachecon-north-america/program/cfp To register for Apache: Big Data - http://events.linuxfoundation.org/events/apache-big-data-north-america/attend/register- To register for ApacheCon - http://events.linuxfoundation.org/events/apachecon-north-america/attend/register- Early Bird registration rates end March 12th, but if you're a committer on an Apache project, you get the low committer rate, which is less than half of the early bird rate! For further updated about ApacheCon, follow us on Twitter, @ApacheCon, or drop by our IRC channel, #apachecon on the Freenode IRC network. Or contact me - rbo...@apache.org - with any questions or concerns. Thanks! Rich Bowen, VP Conferences, Apache Software Foundation -- (You've received this email because you're on a dev@ or users@ mailing list of an Apache Software Foundation project. For subscription and unsubscription information, consult the headers of this email message, as this varies from one list to another.) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: shading killing trunk build times
For most purposes, you can get by with "mvn clean process-resources process-test-resources compile -DskipTests -T 1C". (Hat tip to Karthik) Daniel On 1/18/17 6:27 AM, Steve Loughran wrote: a clean build with no tests, javadocs, is now taking 7+ min locally, with most of that time going into the packaging work [INFO] Apache Hadoop Client API ... SUCCESS [01:50 min] [INFO] Apache Hadoop Client Runtime ... SUCCESS [01:19 min] [INFO] Apache Hadoop Client Packaging Invariants .. SUCCESS [ 0.332 s] [INFO] Apache Hadoop Client Test Minicluster .. SUCCESS [02:52 min] This really hurts when you are trying to build and test across modules, where I want to do a change in say, a hadoop-common test, then run that test in hdfs and aws modules. I don't want or need this client stuff, and all it is doing is taking up time and forcing me into contrived bits where I have to explicitly only build the modules I want, which gets dangerous once you start switching branches Can we make these a profile? Even if it's on by default and something that those of us try - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13489) DistCp may incorrectly return success status when the underlying Job failed
[
https://issues.apache.org/jira/browse/HADOOP-13489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved HADOOP-13489.
-
Resolution: Won't Fix
After adjusting hbase code, the problem is gone.
> DistCp may incorrectly return success status when the underlying Job failed
> ---
>
> Key: HADOOP-13489
> URL: https://issues.apache.org/jira/browse/HADOOP-13489
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Labels: distcp
> Attachments: HADOOP-13489.v1.patch, HADOOP-13489.v2.patch,
> HADOOP-13489.v3.patch, MapReduceBackupCopyService.java,
> testIncrementalBackup-8-12-rethrow.txt, testIncrementalBackup-8-12.txt,
> TestIncrementalBackup-output.txt
>
>
> I was troubleshooting HBASE-14450 where at the end of BackupdistCp#execute(),
> distcp job was marked unsuccessful (BackupdistCp is a wrapper of DistCp).
> Yet in IncrementalTableBackupProcedure#incrementalCopy(), the return value
> from copyService.copy() was 0.
> Here is related code from DistCp:
> {code}
> try {
> execute();
> } catch (InvalidInputException e) {
> LOG.error("Invalid input: ", e);
> return DistCpConstants.INVALID_ARGUMENT;
> } catch (DuplicateFileException e) {
> LOG.error("Duplicate files in input path: ", e);
> return DistCpConstants.DUPLICATE_INPUT;
> } catch (AclsNotSupportedException e) {
> LOG.error("ACLs not supported on at least one file system: ", e);
> return DistCpConstants.ACLS_NOT_SUPPORTED;
> } catch (XAttrsNotSupportedException e) {
> LOG.error("XAttrs not supported on at least one file system: ", e);
> return DistCpConstants.XATTRS_NOT_SUPPORTED;
> } catch (Exception e) {
> LOG.error("Exception encountered ", e);
> return DistCpConstants.UNKNOWN_ERROR;
> }
> return DistCpConstants.SUCCESS;
> {code}
> We don't check whether the Job returned by execute() was successful - we rely
> on all failure cases going through the last catch clause but there may be
> special case.
> Even if the Job fails, DistCpConstants.SUCCESS is returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: shading killing trunk build times
Just filed https://issues.apache.org/jira/browse/HADOOP-13999 To add a mvn flag and profile that can disable shading On Wed, Jan 18, 2017 at 9:46 AM, Daniel Templeton wrote: > For most purposes, you can get by with "mvn clean process-resources > process-test-resources compile -DskipTests -T 1C". (Hat tip to Karthik) > > Daniel > > > On 1/18/17 6:27 AM, Steve Loughran wrote: > >> a clean build with no tests, javadocs, is now taking 7+ min locally, with >> most of that time going into the packaging work >> >> [INFO] Apache Hadoop Client API ... SUCCESS >> [01:50 min] >> [INFO] Apache Hadoop Client Runtime ... SUCCESS >> [01:19 min] >> [INFO] Apache Hadoop Client Packaging Invariants .. SUCCESS [ >> 0.332 s] >> [INFO] Apache Hadoop Client Test Minicluster .. SUCCESS >> [02:52 min] >> >> This really hurts when you are trying to build and test across modules, >> where I want to do a change in say, a hadoop-common test, then run that >> test in hdfs and aws modules. I don't want or need this client stuff, and >> all it is doing is taking up time and forcing me into contrived bits where >> I have to explicitly only build the modules I want, which gets dangerous >> once you start switching branches >> >> >> Can we make these a profile? Even if it's on by default and something >> that those of us try >> >> - >> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org >> For additional commands, e-mail: common-dev-h...@hadoop.apache.org >> >> > > - > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > >
[jira] [Created] (HADOOP-14000) s3guard metadata stores to support millons of children
Steve Loughran created HADOOP-14000:
---
Summary: s3guard metadata stores to support millons of children
Key: HADOOP-14000
URL: https://issues.apache.org/jira/browse/HADOOP-14000
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/s3
Affects Versions: HADOOP-13345
Reporter: Steve Loughran
S3 repos can have millions of child entries
Currently {{DirListingMetaData}} can't and {{MetadataStore.listChildren(Path
path)}} won't be able to handle directories that big, for listing, deleting or
naming.
We will need a paged response from the listing operation, something which can
be iterated over.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [VOTE] Release cadence and EOL
Thanks Sangjin for pushing this forward! I have a few questions: 1. What is the definition of end-of-life for a release in the hadoop project? My current understanding is as follows: When a release line reaches end-of-life, there are no more planned releases for that line. Committers are no longer responsible for back-porting bug fixes to the line (including fixed security vulnerabilities) and it is essentially unmaintained. 2. How do major releases affect the end-of-life proposal? For example, how does a new minor release in the next major release affect the end-of-life of minor releases in a previous major release? Is it possible to have a maintained 2.x release if there is a 3.3 release? Thanks! On Tue, Jan 17, 2017 at 10:32 AM, Karthik Kambatla wrote: > +1 > > I would also like to see some process guidelines. I should have brought > this up on the discussion thread, but I thought of them only now :( > >- Is an RM responsible for all maintenance releases against a minor >release, or finding another RM to drive a maintenance release? In the > past, >this hasn't been a major issue. >- When do we pick/volunteer to RM a minor release? IMO, this should be >right after the previous release goes out. For example, Junping is > driving >2.8.0 now. As soon as that is done, we need to find a volunteer to RM > 2.9.0 >6 months after. >- The release process has multiple steps, based on >major/minor/maintenance. It would be nice to capture/track how long each >step takes so the RM can be prepared. e.g. herding the cats for an RC > takes >x weeks, compatibility checks take y days of work. > > > On Tue, Jan 17, 2017 at 10:05 AM, Sangjin Lee wrote: > > > Thanks for correcting me! I left out a sentence by mistake. :) > > > > To correct the proposal we're voting for: > > > > A minor release on the latest major line should be every 6 months, and a > > maintenance release on a minor release (as there may be concurrently > > maintained minor releases) every 2 months. > > > > A minor release line is end-of-lifed 2 years after it is released or > there > > are 2 newer minor releases, whichever is sooner. The community reserves > the > > right to extend or shorten the life of a release line if there is a good > > reason to do so. > > > > Sorry for the snafu. > > > > Regards, > > Sangjin > > > > On Tue, Jan 17, 2017 at 9:58 AM, Daniel Templeton > > wrote: > > > > > Thanks for driving this, Sangjin. Quick question, though: the subject > > line > > > is "Release cadence and EOL," but I don't see anything about cadence in > > the > > > proposal. Did I miss something? > > > > > > Daniel > > > > > > > > > On 1/17/17 8:35 AM, Sangjin Lee wrote: > > > > > >> Following up on the discussion thread on this topic ( > > >> https://s.apache.org/eFOf), I'd like to put the proposal for a vote > for > > >> the > > >> release cadence and EOL. The proposal is as follows: > > >> > > >> "A minor release line is end-of-lifed 2 years after it is released or > > >> there > > >> are 2 newer minor releases, whichever is sooner. The community > reserves > > >> the > > >> right to extend or shorten the life of a release line if there is a > good > > >> reason to do so." > > >> > > >> This also entails that we the Hadoop community commit to following > this > > >> practice and solving challenges to make it possible. Andrew Wang laid > > out > > >> some of those challenges and what can be done in the discussion thread > > >> mentioned above. > > >> > > >> I'll set the voting period to 7 days. I understand a majority rule > would > > >> apply in this case. Your vote is greatly appreciated, and so are > > >> suggestions! > > >> > > >> Thanks, > > >> Sangjin > > >> > > >> > > > > > > - > > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > > > > > > > > >
Re: [Continued] [Release thread] 2.8.0 release activities
Hi folks, In the passed one or two weeks, we found some new blockers get coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart with work preserving enabled) and YARN-6072 (RM unable to start in secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed YARN-6068), and I was starting the RC build process since early this week. As we have significant build tools/process change (docker based) since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with it and finally get a successful build on 2.8.0-RC0 last night. I already push RC0 tag into public which is prerequisite step before RC voting. However, in the mean while, I was pinged by Varun Vasudev that there are a known vulnerability issues on container_executor get identified and discussed in hadoop security email threads - looks like YARN-5704 fixed part of it, but left part - the privilege escalation via /proc/self/environ is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait this issue get fixed to prepare a new release candidate. As RC0 tag cannot be reverted after push into apache, our next release candidate will start from RC1. As I mentioned in early email, 2.8.0 is a very big release (2300+ commits since 2.7.3) and I am glad that we are almost there. Thanks everyone for being patient and contributing our release work. Please let me know if you have more comments or suggestions. Thanks, Junping From: Junping Du Sent: Wednesday, January 04, 2017 1:41 AM To: common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: Re: [Continued] [Release thread] 2.8.0 release activities Hi all Hadoopers, I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC. Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort. Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned! Thanks, Junping From: Junping Du Sent: Wednesday, December 07, 2016 11:31 AM To: Akira Ajisaka; common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: Re: [Continued] [Release thread] 2.8.0 release activities Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down. However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal. Thanks, Junping From: Akira Ajisaka Sent: Wednesday, December 07, 2016 12:12 AM To: common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: Re: [Continued] [Release thread] 2.8.0 release activities Thanks Junping and Andrew! HADOOP-2.8-JACC is not working well, so I manually kicked a job to compare 2.8 with 2.7.3. https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html Regards, Akira On 2016/12/02 2:08, Junping Du wrote: > Thanks Andrew! That's also a nice suggestion. I already create a similar job: > https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 > and kick off several runs manually. Will monitor incompatible status from > there. > > > Thanks, > > > Junping > > > > From: Andrew Wang > Sent: Wednesday, November 30, 2016 4:18 PM > To: Junping Du > Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; > mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org; Vinod > Vavilapalli; Jian He; Wangda Tan; sj...@twitter.com > Subject: Re: [Continued] [R
Re: [VOTE] Release cadence and EOL
> On Jan 18, 2017, at 11:21 AM, Chris Trezzo wrote: > > Thanks Sangjin for pushing this forward! I have a few questions: These are great questions, because I know I'm not seeing a whole lot of substance in this vote. The way to EOL software in the open source universe is with new releases and aging it out. If someone wants to be a RE for a new branch-1 release, more power to them. As volunteers to the ASF, we're not on the hook to provide much actual support. This feels more like a vendor play than a community one. But if the PMC want to vote on it, whatever. It won't be first bylaw that doesn't really mean much. > 1. What is the definition of end-of-life for a release in the hadoop > project? My current understanding is as follows: When a release line > reaches end-of-life, there are no more planned releases for that line. > Committers are no longer responsible for back-porting bug fixes to the line > (including fixed security vulnerabilities) and it is essentially > unmaintained. Just a point of clarification. There is no policy that says that committers must back port. It's up to the individual committers to push a change onto any particular branch. Therefore, this vote doesn't really change anything in terms of committer responsibilities here. > 2. How do major releases affect the end-of-life proposal? For example, how > does a new minor release in the next major release affect the end-of-life > of minor releases in a previous major release? Is it possible to have a > maintained 2.x release if there is a 3.3 release? I'm looking forward to seeing this answer too, given that 2.7.0 is probably past the 2 year mark, 2.8.0 has seemingly been in a holding pattern for over a year, and the next 3.0.0 alpha should be RSN - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14001) Improve delegation token validity checking
Akira Ajisaka created HADOOP-14001: -- Summary: Improve delegation token validity checking Key: HADOOP-14001 URL: https://issues.apache.org/jira/browse/HADOOP-14001 Project: Hadoop Common Issue Type: Bug Reporter: Akira Ajisaka Assignee: Akira Ajisaka In AbstractDelegationSecretManager#verifyToken, MessageDigest.isEqual should be used instead of Arrays.equals to compare byte arrays. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [VOTE] Release cadence and EOL
+1 on Sangjin's proposal - "A minor release line is end-of-lifed 2 years after it is released or there are 2 newer minor releases, whichever is sooner. The community reserves the right to extend or shorten the life of a release line if there is a good reason to do so." I also noticed Karthik bring up some new proposals - some of them looks interesting to me and I have some ideas as well. Karthik, can you bring it out in a separated discussion threads so that we can discuss from there? About Chris Trezzo's question about definition of EOL of hadoop release, I think potentially changes could be: 1. For users of Apache hadoop, they would expect to upgrade to a new minor/major releases after EOL of their current release because there is no guarantee of new maintenance release. 2. For release effort, apache law claim that committer can volunteer RM for any release. With this release EOL proposal passes and written into hadoop bylaw, anyone want to call for a release which is EOL then she/he have to provide a good reason to community and get voted before to start release effort. We don't want to waste community time/resource to verify/vote a narrow interested release. 3. About committer's responsibility, I think the bottom line is committer should commit patch contributor's target release and her/his own interest release which I conservatively agree with Allen's point that this vote doesn't change anything. But if a committer want to take care more interest from the whole community like most committers are doing today, he/she should understand which branches can benefit more people and could skip some EOL release branches for backport effort. About major release EOL, this could be more complicated and I think we should discuss separately. Thanks, Junping From: Allen Wittenauer Sent: Wednesday, January 18, 2017 3:30 PM To: Chris Trezzo Cc: common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; yarn-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org Subject: Re: [VOTE] Release cadence and EOL > On Jan 18, 2017, at 11:21 AM, Chris Trezzo wrote: > > Thanks Sangjin for pushing this forward! I have a few questions: These are great questions, because I know I'm not seeing a whole lot of substance in this vote. The way to EOL software in the open source universe is with new releases and aging it out. If someone wants to be a RE for a new branch-1 release, more power to them. As volunteers to the ASF, we're not on the hook to provide much actual support. This feels more like a vendor play than a community one. But if the PMC want to vote on it, whatever. It won't be first bylaw that doesn't really mean much. > 1. What is the definition of end-of-life for a release in the hadoop > project? My current understanding is as follows: When a release line > reaches end-of-life, there are no more planned releases for that line. > Committers are no longer responsible for back-porting bug fixes to the line > (including fixed security vulnerabilities) and it is essentially > unmaintained. Just a point of clarification. There is no policy that says that committers must back port. It's up to the individual committers to push a change onto any particular branch. Therefore, this vote doesn't really change anything in terms of committer responsibilities here. > 2. How do major releases affect the end-of-life proposal? For example, how > does a new minor release in the next major release affect the end-of-life > of minor releases in a previous major release? Is it possible to have a > maintained 2.x release if there is a 3.3 release? I'm looking forward to seeing this answer too, given that 2.7.0 is probably past the 2 year mark, 2.8.0 has seemingly been in a holding pattern for over a year, and the next 3.0.0 alpha should be RSN - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/291/ [Jan 18, 2017 8:54:33 AM] (aajisaka) HDFS-11290. TestFSNameSystemMBean should wait until JMX cache is [Jan 18, 2017 9:11:33 AM] (aajisaka) HADOOP-13955. Replace deprecated HttpServer2 and SSLFactory constants. [Jan 18, 2017 8:46:32 PM] (shv) HDFS-10733. NameNode terminated after full GC thinking QJM is [Jan 18, 2017 9:31:05 PM] (wang) HDFS-10759. Change fsimage bool isStriped from boolean to an enum. [Jan 18, 2017 9:31:33 PM] (wangda) YARN-5556. CapacityScheduler: Support deleting queues without requiring [Jan 18, 2017 10:35:51 PM] (aw) HADOOP-13964. Remove vestigal templates directories [Jan 18, 2017 10:39:05 PM] (aw) HADOOP-13673. Update scripts to be smarter when running with privilege [Jan 18, 2017 11:18:48 PM] (wang) HADOOP-13996. Fix some release build issues. [Jan 19, 2017 1:42:56 AM] (yqlin) HADOOP-13965. Groups should be consistent in using default group mapping [Jan 19, 2017 4:05:13 AM] (yqlin) HDFS-11316. - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
