[jira] [Resolved] (MAPREDUCE-5458) Jobhistory server (and probably others) throws HTTP 500 error if keytab fails

2018-09-01 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-5458. - Resolution: Won't Fix > Jobhistory server (and probably others) thr

[jira] [Resolved] (MAPREDUCE-6313) Audit/optimize tests in hadoop-mapreduce-client-jobclient

2018-09-01 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-6313. - Resolution: Won't Fix > Audit/optimize tests in hadoop-mapreduc

[jira] [Resolved] (MAPREDUCE-6378) convergence error in hadoop-streaming during mvn install

2018-09-01 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-6378. - Resolution: Won't Fix > convergence error in hadoop-streaming du

[jira] [Resolved] (MAPREDUCE-6469) libnativetask lacks header files and documentation

2018-09-01 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-6469. - Resolution: Won't Fix Target Version/s: (was: ) > libna

[jira] [Resolved] (MAPREDUCE-6699) hadoop-mapred unit tests for dynamic commands

2018-09-01 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-6699. - Resolution: Won't Fix > hadoop-mapred unit tests for dynamic

[jira] [Resolved] (MAPREDUCE-6691) move the shell code out of hadoop-mapreduce-project

2018-09-01 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-6691. - Resolution: Won't Fix > move the shell code out of hadoop-mapreduce

[jira] [Resolved] (MAPREDUCE-6934) downlink.data is written to CWD

2018-09-01 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-6934. - Resolution: Won't Fix > downlink.data is writte

Re: [DISCUSS] Alpha Release of Ozone

2018-08-08 Thread Allen Wittenauer
> On Aug 8, 2018, at 12:56 PM, Anu Engineer wrote: > >> Has anyone verified that a Hadoop release doesn't have _any_ of the extra >> ozone bits that are sprinkled outside the maven modules? > As far as I know that is the state, we have had multiple Hadoop releases > after ozone has been merg

Re: [DISCUSS] Alpha Release of Ozone

2018-08-08 Thread Allen Wittenauer
Given that there are some Ozone components spread out past the core maven modules, is the plan to release a Hadoop Trunk + Ozone tar ball or is more work going to go into segregating the Ozone components prior to release? Has anyone verified that a Hadoop release doesn't have _any_ of the extr

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

2018-06-07 Thread Allen Wittenauer
> On Jun 7, 2018, at 11:47 AM, Steve Loughran wrote: > > Actually, Yongjun has been really good at helping me get set up for a 2.7.7 > release, including "things you need to do to get GPG working in the docker > image” *shrugs* I use a different release script after some changes broke

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

2018-06-07 Thread Allen Wittenauer
> On Jun 7, 2018, at 3:46 AM, Lokesh Jain wrote: > > Hi Yongjun > > I followed Nanda’s steps and I see the same issues as reported by Nanda. This situation is looking like an excellent opportunity for PMC members to mentor people on how the build works since it’s apparent that three days la

Re: Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

2018-05-15 Thread Allen Wittenauer
> On May 15, 2018, at 10:16 AM, Chris Douglas wrote: > > They've been failing for a long time. It can't install bats, and > that's fatal? -C The bats error is new and causes the build to fail enough that it produces the email output. For the past few months, it hasn’t been producing

Re: Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

2018-05-15 Thread Allen Wittenauer
FYI: I’m going to disable the branch-2 nightly jobs. - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: [NOTIFICATION] Hadoop trunk rebased

2018-04-27 Thread Allen Wittenauer
Did the patch that fixes the mountain of maven warnings get missed? > On Apr 26, 2018, at 11:52 PM, Akira Ajisaka wrote: > > + common-dev and mapreduce-dev > > On 2018/04/27 6:23, Owen O'Malley wrote: >> As we discussed in hdfs-dev@hadoop, I did a force push to Hadoop's trunk to >> replace the

Fwd: Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64

2018-03-15 Thread Allen Wittenauer
For my part of the HDFS bug bash, I’ve gotten the ASF Windows build working again. Starting tomorrow, results will be sent to the *-dev lists. A few notes: * It only runs the unit tests. There’s not much point in running the other Yetus plugins since those are covered by the Linux one

Re: [ANNOUNCE] Apache Hadoop 3.0.0 GA is released

2017-12-18 Thread Allen Wittenauer
It’s significantly more concerning that 3.0.0-beta1 doesn’t show up here: http://hadoop.apache.org/docs/r3.0.0/hadoop-project-dist/hadoop-common/release/index.html It looks like they are missing from the source tag too. I wonder what else is missing. > On Dec 18, 2017, at 11:15 AM, Andrew Wa

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

2017-11-30 Thread Allen Wittenauer
> On Nov 30, 2017, at 1:07 AM, Rohith Sharma K S > wrote: > > > >. If ATSv1 isn’t replaced by ATSv2, then why is it marked deprecated? > Ideally it should not be. Can you point out where it is marked as deprecated? > If it is in historyserver daemon start, that change made very long back when

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

2017-11-25 Thread Allen Wittenauer
> On Nov 21, 2017, at 2:16 PM, Vinod Kumar Vavilapalli > wrote: > >>> - $HADOOP_YARN_HOME/sbin/yarn-daemon.sh start historyserver doesn't even >>> work. Not just deprecated in favor of timelineserver as was advertised. >> >> This works for me in trunk and the bash code doesn’t appear to

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

2017-11-20 Thread Allen Wittenauer
The original release script and instructions broke the build up into three or so steps. When I rewrote it, I kept that same model. It’s probably time to re-think that. In particular, it should probably be one big step that even does the maven deploy. There’s really no harm in doing th

Re: [DISCUSSION] Merging HDFS-7240 Object Store (Ozone) to trunk

2017-11-03 Thread Allen Wittenauer
> On Nov 3, 2017, at 12:08 PM, Stack wrote: > > On Sat, Oct 28, 2017 at 2:00 PM, Konstantin Shvachko > wrote: > >> It is an interesting question whether Ozone should be a part of Hadoop. > > I don't see a direct answer to this question. Is there one? Pardon me if > I've not seen it but I'm in

Re: Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

2017-10-24 Thread Allen Wittenauer
> On Oct 24, 2017, at 4:10 PM, Andrew Wang wrote: > > FWIW we've been running branch-3.0 unit tests successfully internally, though > we have separate jobs for Common, HDFS, YARN, and MR. The failures here are > probably a property of running everything in the same JVM, which I've found > pro

Re: Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

2017-10-24 Thread Allen Wittenauer
t tell which daemons/components of HDFS consumes unexpected high >> memory. Don't sounds like a solid bug report to me. >> >> >> >> Thanks,? >> >> >> Junping >> >> >> >> From: Sean Bus

Re: Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

2017-10-24 Thread Allen Wittenauer
> On Oct 23, 2017, at 12:50 PM, Allen Wittenauer > wrote: > > > > With no other information or access to go on, my current hunch is that one of > the HDFS unit tests is ballooning in memory size. The easiest way to kill a > Linux machine is to eat all of the RAM,

Re: Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

2017-10-23 Thread Allen Wittenauer
. > > Thanks, > Subru > > On Mon, Oct 23, 2017 at 11:26 AM, Vrushali C wrote: > Hi Allen, > > I have filed https://issues.apache.org/jira/browse/YARN-7380 for the > timeline service findbugs warnings. > > thanks > Vrushali > > > On Mon, Oct 23, 2017

Re: Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

2017-10-23 Thread Allen Wittenauer
I’m really confused why this causes the Yahoo! QA boxes to go catatonic (!?!) during the run. As in, never come back online, probably in a kernel panic. It’s pretty consistently in hadoop-hdfs, so something is going wrong there… is branch-2 hdfs behaving badly? Someone needs to run the hadoop

Re: Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

2017-10-21 Thread Allen Wittenauer
To whoever set this up: There was a job config problem where the Jenkins branch parameter wasn’t passed to Yetus. Therefore both of these reports have been against trunk. I’ve fixed this job (as well as the other jobs) to honor that parameter. I’ve kicked off a new run with these changes.

YARN native services Re: 2017-10-06 Hadoop 3 release status update

2017-10-09 Thread Allen Wittenauer
> On Oct 6, 2017, at 5:51 PM, Eric Yang wrote: > yarn application -deploy –f spec.json > yarn application -stop > yarn application -restart > yarn application -remove > > and > > yarn application –list will display both application list from RM as well as > docker services?

Re: 2017-10-06 Hadoop 3 release status update

2017-10-06 Thread Allen Wittenauer
> On Oct 6, 2017, at 1:31 PM, Andrew Wang wrote: > > - Still waiting on Allen to review YARN native services feature. Fake news. I’m still -1 on it, at least prior to a patch that posted late yesterday. I’ll probably have a chance to play with it early next week. Key pro

Re: qbt is failiing///RE: Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2017-09-19 Thread Allen Wittenauer
> On Sep 19, 2017, at 6:35 AM, Brahma Reddy Battula > wrote: > > qbt is failing from two days with following errors, any idea on this..? Nothing to be too concerned about. This is what it looks like when a build server gets bounced or crashed. INFRA team knows our jobs take

Re: [VOTE] Merge yarn-native-services branch into trunk

2017-09-13 Thread Allen Wittenauer
> On Sep 8, 2017, at 9:25 AM, Jian He wrote: > > Hi Allen, > The documentations are committed. Please check QuickStart.md and others in > the same folder. > YarnCommands.md doc is updated to include new commands. > DNS default port is also documented. > Would you like to give a look and see if

Re: [VOTE] Merge yarn-native-services branch into trunk

2017-09-06 Thread Allen Wittenauer
> On Sep 5, 2017, at 6:23 PM, Jian He wrote: > >> If it doesn’t have all the bells and whistles, then it shouldn’t be on >> port 53 by default. > Sure, I’ll change the default port to not use 53 and document it. >> *how* is it getting launched on a privileged port? It sounds like the

Re: [VOTE] Merge yarn-native-services branch into trunk

2017-09-05 Thread Allen Wittenauer
> On Sep 5, 2017, at 3:12 PM, Gour Saha wrote: > > 2) Lots of markdown problems in the NativeServicesDiscovery.md document. > This includes things like Œyarnsite.xml¹ (missing a dash.) > > The md patch uploaded to YARN-5244 had some special chars. I fixed those > in YARN-7161. It’s a

Re: [VOTE] Merge yarn-native-services branch into trunk

2017-09-05 Thread Allen Wittenauer
> On Sep 5, 2017, at 2:53 PM, Jian He wrote: > >> Based on the documentation, this doesn’t appear to be a fully function DNS >> server as an admin would expect (e.g., BIND, Knot, whatever). Where’s >> forwarding? How do I setup notify? Are secondaries even supported? etc, etc. > > It seems l

Re: [VOTE] Merge yarn-native-services branch into trunk

2017-09-05 Thread Allen Wittenauer
> On Aug 31, 2017, at 8:33 PM, Jian He wrote: > I would like to call a vote for merging yarn-native-services to trunk. 1) Did I miss it or is there no actual end-user documentation on how to use this? I see hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/native-serv

YARN javadoc failures Re: [DISCUSS] Branches and versions for Hadoop 3

2017-09-01 Thread Allen Wittenauer
> On Aug 28, 2017, at 9:58 AM, Allen Wittenauer > wrote: > The automation only goes so far. At least while investigating Yetus > bugs, I've seen more than enough blatant and purposeful ignored errors and > warnings that I'm not convinced it will be effectiv

Re: [DISCUSS] Branches and versions for Hadoop 3

2017-08-28 Thread Allen Wittenauer
> On Aug 28, 2017, at 12:41 PM, Jason Lowe wrote: > > I think this gets back to the "if it's worth committing" part. This brings us back to my original question: "Doesn't this place an undue burden on the contributor with the first incompatible patch to prove worthiness? What

Re: [DISCUSS] Branches and versions for Hadoop 3

2017-08-28 Thread Allen Wittenauer
> On Aug 25, 2017, at 1:23 PM, Jason Lowe wrote: > > Allen Wittenauer wrote: > > > Doesn't this place an undue burden on the contributor with the first > > incompatible patch to prove worthiness? What happens if it is decided that > > it's not g

Re: [DISCUSS] Branches and versions for Hadoop 3

2017-08-25 Thread Allen Wittenauer
> On Aug 25, 2017, at 10:36 AM, Andrew Wang wrote: > Until we need to make incompatible changes, there's no need for > a Hadoop 4.0 version. Some questions: Doesn't this place an undue burden on the contributor with the first incompatible patch to prove worthiness? What happens if it

Re: Branch merges and 3.0.0-beta1 scope

2017-08-22 Thread Allen Wittenauer
We should avoid turning this into a replay of Apache Hadoop 2.6.0 (and to a lesser degree, 2.7.0 and 2.8.0) where a bunch of last minute “experimental” features derail stability for a significantly long period of time. -

[jira] [Created] (MAPREDUCE-6934) downlink.data is written to CWD

2017-08-04 Thread Allen Wittenauer (JIRA)
Allen Wittenauer created MAPREDUCE-6934: --- Summary: downlink.data is written to CWD Key: MAPREDUCE-6934 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6934 Project: Hadoop Map/Reduce

[jira] [Resolved] (MAPREDUCE-2320) RAID DistBlockFixer should limit pending jobs instead of pending files

2017-08-01 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-2320. - Resolution: Won't Fix MR RAID has been replaced by HDFS EC in modern ver

[jira] [Resolved] (MAPREDUCE-2267) Parallelize reading of blocks within a stripe

2017-08-01 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-2267. - Resolution: Won't Fix MR RAID has been replaced by HDFS EC in modern ver

[jira] [Resolved] (MAPREDUCE-2189) RAID Parallel traversal needs to synchronize stats

2017-08-01 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-2189. - Resolution: Won't Fix MR RAID has been replaced by HDFS EC in modern ver

Re: Are binary artifacts are part of a release?

2017-07-31 Thread Allen Wittenauer
-general/20.mbox/%3C4EB0827C.6040204%40apache.org%3E > > Doug Cutting: > "Folks should not primarily evaluate binaries when voting. The ASF primarily > produces and publishes source-code > so voting artifacts should be optimized for evaluation of that." > > Thanks, &g

Re: Are binary artifacts are part of a release?

2017-07-31 Thread Allen Wittenauer
> On Jul 31, 2017, at 4:18 PM, Andrew Wang wrote: > > Forking this off to not distract from release activities. > > I filed https://issues.apache.org/jira/browse/LEGAL-323 to get clarity on the > matter. I read the entire webpage, and it could be improved one way or the > other. IAN

Re: [VOTE] Release Apache Hadoop 2.7.4 (RC0)

2017-07-31 Thread Allen Wittenauer
> On Jul 31, 2017, at 11:20 AM, Konstantin Shvachko > wrote: > > https://wiki.apache.org/hadoop/HowToReleasePreDSBCR FYI: If you are using ASF Jenkins to create an ASF release artifact, it's pretty much an automatic vote failure as any such release is in violation of

Re: Pre-Commit build is failing

2017-07-25 Thread Allen Wittenauer
; >> > >> Should we add "patchprocess/" to .gitignore, is that the problem for 2.7? > >> > >> Thanks, > >> --Konstantin > >> > >> On Fri, Jul 21, 2017 at 6:24 PM, Konstantin Shvachko < > >> shv.had...@gmail.com>

Re: About 2.7.4 Release

2017-05-02 Thread Allen Wittenauer
Is there any reason to not Close -alpha1+resolved state JIRAs? It's been quite a while and those definitely should not getting re-opened anymore. What about -alpha2's that are also resolved? - To unsubscribe, e-mail: mapreduce

Re: About 2.7.4 Release

2017-05-01 Thread Allen Wittenauer
> On May 1, 2017, at 2:27 PM, Andrew Wang wrote: > I believe I asked about this on dev-yetus a while back. I'd prefer that the > presence of the fix version be sufficient to indicate whether a JIRA is > included in a release branch. Yetus requires that the JIRA be resolved as > "Fixed" to show

Re: About 2.7.4 Release

2017-04-26 Thread Allen Wittenauer
> On Apr 25, 2017, at 12:35 AM, Akira Ajisaka wrote: > > Maybe we should create a jira to track this? > > I think now either way (reopen or create) is fine. > > Release doc maker creates change logs by fetching information from JIRA, so > reopening the tickets should be avoided when a release

Re: HADOOP-14316: Switching from Findbugs to Spotbugs

2017-04-19 Thread Allen Wittenauer
> On Apr 19, 2017, at 10:52 AM, Wei-Chiu Chuang wrote: > That sounds scary. Would you mind to share the list of bugs that spotbugs > found? Sounds like some of them may warrant new blockers jiras for Hadoop 3. I've added the list to the JIRA. ---

HADOOP-14316: Switching from Findbugs to Spotbugs

2017-04-19 Thread Allen Wittenauer
Hey gang. HADOOP-14316 enables the spotbugs back-end for the findbugs front-end. Spotbugs (https://spotbugs.github.io/) is the fork of findbugs that the community and some of the major contributors have made to move findbugs forward. It is geared towards JDK8 and JDK9. Befor

Re: Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2017-04-17 Thread Allen Wittenauer
Looks like someone reset HEAD back to Mar 31. Sent from my iPad > On Apr 16, 2017, at 12:08 AM, Apache Jenkins Server > wrote: > > For more details, see > https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/378/ > > > > > > -1 overall > > > The following subsystems voted -1

[jira] [Created] (MAPREDUCE-6875) mapred-site.xml in bin tarball lacks a license

2017-04-05 Thread Allen Wittenauer (JIRA)
Allen Wittenauer created MAPREDUCE-6875: --- Summary: mapred-site.xml in bin tarball lacks a license Key: MAPREDUCE-6875 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6875 Project: Hadoop

Re: [DISCUSS] Changing the default class path for clients

2017-04-03 Thread Allen Wittenauer
;s the current contract for `hadoop classpath`? Would it be safer to > introduce `hadoop userclasspath` or similar for this behavior? > > I'm betting that changing `hadoop classpath` will lead to some breakages, > so I'd prefer to make this new behavior opt-in. > > Best,

[DISCUSS] Changing the default class path for clients

2017-04-03 Thread Allen Wittenauer
This morning I had a bit of a shower thought: With the new shaded hadoop client in 3.0, is there any reason the default classpath should remain the full blown jar list? e.g., shouldn’t ‘hadoop classpath’ just return configuration, user supplied bits (e.g., HADOOP_USER_CLASSPAT

Re: Can we update protobuf's version on trunk?

2017-03-28 Thread Allen Wittenauer
> On Mar 28, 2017, at 5:09 PM, Chris Douglas wrote: > > I haven't seen data identifying PB as a bottleneck, but the > non-x86/non-Linux and dev setup arguments may make this worthwhile. -C FWIW, we have the same problem with leveldbjni-all. (See the ASF PowerPC build logs) I keep meani

Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

2017-03-23 Thread Allen Wittenauer
Just a heads up. Looks like some removed the Finish Date off of 2.8.0 in JIRA. It needs to be put back to match what is in the artifacts that we voted on. - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org Fo

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

2017-03-21 Thread Allen Wittenauer
> On Mar 21, 2017, at 10:12 AM, Andrew Wang wrote: > > I poked around a bit. The 3.0.0-alpha2 binary tarball is only 246M and has > more changes than 2.8.0. Not to disclaim any other potential issues, but it's worth noting 3.x de-dupes jar files as part of the packaging process. So it's not

Re: About 2.7.4 Release

2017-03-08 Thread Allen Wittenauer
> On Mar 8, 2017, at 1:54 PM, Allen Wittenauer > wrote: > > This is already possible: > * don’t use —asfrelease > * use —sign, —native, and, if appropriate for your platform, > —docker and —dockercache Oh yeah, I forg

Re: About 2.7.4 Release

2017-03-08 Thread Allen Wittenauer
> On Mar 8, 2017, at 10:55 AM, Marton Elek wrote: > > I think the main point here is the testing of the release script, not the > creation of the official release. … except the Hadoop PMC was doing exactly this from 2.3.0 up until recently. Which means we have a few years worth of rel

Re: About 2.7.4 Release

2017-03-07 Thread Allen Wittenauer
> On Mar 7, 2017, at 2:51 PM, Andrew Wang wrote: > I think it'd be nice to > have a nightly Jenkins job that builds an RC, Just a reminder that any such build cannot be used for an actual release: http://www.apache.org/legal/release-policy.html#owned-controlled-hardware

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha2 RC0

2017-01-23 Thread Allen Wittenauer
> On Jan 23, 2017, at 8:50 PM, Chris Douglas wrote: > > Thanks for all your work on this, Andrew. It's great to see the 3.x > series moving forward. > > If you were willing to modify the release notes and add the LICENSE to > the jar, we don't need to reset the clock on the VOTE, IMO. FWIW, I

Re: [VOTE] Release cadence and EOL

2017-01-23 Thread Allen Wittenauer
> On Jan 21, 2017, at 7:08 PM, Karthik Kambatla wrote: > > 3. RM: some method to madness. Junping, for instance, is trying to roll > a release with 2300 patches. It is a huge time investment. (Thanks again, > Junping.) Smaller releases are easier to manage. A target release cadence, > co

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha2 RC0

2017-01-22 Thread Allen Wittenauer
> On Jan 22, 2017, at 9:05 PM, Allen Wittenauer > wrote: > > > > > >> On Jan 20, 2017, at 2:36 PM, Andrew Wang wrote: >> >> http://home.apache.org/~wang/3.0.0-alpha2-RC0/ > > There are quite a few JIRA issues that need release notes.

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha2 RC0

2017-01-22 Thread Allen Wittenauer
> On Jan 20, 2017, at 2:36 PM, Andrew Wang wrote: > > http://home.apache.org/~wang/3.0.0-alpha2-RC0/ There are quite a few JIRA issues that need release notes. - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@ha

Re: [Continued] [Release thread] 2.8.0 release activities

2017-01-20 Thread Allen Wittenauer
If you ran mvn clean at any point in your repo between create-release and mvn deploy, you'll need to start at running create-release again. create-release leaves things in a state that mvn deploy should be ready to go, with no clean necessary. > On Jan 20, 2017, at 11:12 AM, Junping Du wrote

Re: [VOTE] Release cadence and EOL

2017-01-18 Thread Allen Wittenauer
> On Jan 18, 2017, at 11:21 AM, Chris Trezzo wrote: > > Thanks Sangjin for pushing this forward! I have a few questions: These are great questions, because I know I'm not seeing a whole lot of substance in this vote. The way to EOL software in the open source universe is with new rel

Re: [VOTE] Release Apache Hadoop 2.6.5 (RC1)

2016-10-06 Thread Allen Wittenauer
> On Oct 6, 2016, at 1:39 PM, Akira Ajisaka wrote: > > > It wasn't 'renamed' to jenkins, prior releases were actually built by and > > on the Jenkins infrastructure. Which was a very very bad idea: it's > > insecure and pretty much against ASF policy. > > Sorry for the confusion. I should no

Re: [VOTE] Release Apache Hadoop 2.6.5 (RC1)

2016-10-06 Thread Allen Wittenauer
> On Oct 5, 2016, at 10:35 PM, Akira Ajisaka wrote: > Can we rename it? > > AFAIK, hadoop releases were built by hortonmu in 2014 and was renamed to > jenkins. That's not how that works. It's literally storing the id of the person who built the classes. It wasn't 'renamed' t

Re: Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2016-09-13 Thread Allen Wittenauer
> On Sep 13, 2016, at 7:31 AM, Apache Jenkins Server > wrote: > > For more details, see > https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/163/ > > unit: > > > > https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/163/artifact/out/patch-unit-hadoop-mapreduc

[RESULTS] Re: [VOTE] Merge HADOOP-13341

2016-09-12 Thread Allen Wittenauer
The vote passes with 3 +1 binding votes. I'll be merging this later today. Thanks everyone! > On Sep 7, 2016, at 6:44 AM, Allen Wittenauer > wrote: > > > I’d like to call for a vote to run for 5 days (ending Mon 12, 2016 at > 7AM PT) to merge the HADOOP-13

Re: [VOTE] Merge HADOOP-13341

2016-09-09 Thread Allen Wittenauer
> On Sep 9, 2016, at 2:15 PM, Anu Engineer wrote: > > +1, Thanks for the effort. It brings in a world of consistency to the hadoop > vars; and as usual reading your bash code was very educative. Thanks! There's still a handful of HDFS and MAPRED vars that begin with HADOOP, b

Re: [VOTE] Merge HADOOP-13341

2016-09-08 Thread Allen Wittenauer
> On Sep 8, 2016, at 2:50 AM, Steve Loughran wrote: > > I'm trying to do the review effort here even though I don't know detailed > bash, as I expect I don't know any less than others, and what better way to > learn than reviewing code written by people that do know bash? Just a head

[VOTE] Merge HADOOP-13341

2016-09-07 Thread Allen Wittenauer
I’d like to call for a vote to run for 5 days (ending Mon 12, 2016 at 7AM PT) to merge the HADOOP-13341 feature branch into trunk. This branch was developed exclusively by me. As usual with large shell script changes, it's been broken up into several smaller commits to make it easier

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0

2016-09-01 Thread Allen Wittenauer
> On Sep 1, 2016, at 3:18 PM, Allen Wittenauer > wrote: > > >> On Sep 1, 2016, at 2:57 PM, Andrew Wang wrote: >> >> Steve requested a git hash for this release. This led us into a brief >> discussion of our use of git tags, wherein we realized that al

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0

2016-09-01 Thread Allen Wittenauer
> On Sep 1, 2016, at 2:57 PM, Andrew Wang wrote: > > Steve requested a git hash for this release. This led us into a brief > discussion of our use of git tags, wherein we realized that although > release tags are immutable (start with "rel/"), RC tags are not. This is > based on the HowToRelease

[DISCUSS] HADOOP-13341 Merge Request

2016-08-31 Thread Allen Wittenauer
Before requesting a merge vote, I'd like for folks to take a look at HADOOP-13341. This branch changes how the vast majority of the _OPTS variables work in various ways, making things easier for devs and users by helping to make the rules consistent. It also clarifies/cleans up how the _USER

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0

2016-08-30 Thread Allen Wittenauer
> On Aug 30, 2016, at 2:20 PM, Eric Badger wrote: > > Well that's embarrassing. I had accidentally slightly renamed my > log4j.properties file in my conf directory, so it was there, just not being > read. Nah. You were just testing out the shell rewrite's ability to detect a common

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0

2016-08-30 Thread Allen Wittenauer
> On Aug 30, 2016, at 2:06 PM, Eric Badger > wrote: > > > WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete. ^^ > > After running the above command, the RM UI showed a successful job, but as > you can see, I did not have anything pri

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0

2016-08-30 Thread Allen Wittenauer
> On Aug 30, 2016, at 10:17 AM, Zhe Zhang wrote: > > Thanks Andrew for the great work! It's really exciting to finally see a > Hadoop 3 RC. > > I noticed CHANGES and RELEASENOTES markdown files which were not in > previous RCs like 2.7.3. What are good tools to verify them? I tried > reading th

Re: Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2016-08-26 Thread Allen Wittenauer
> On Aug 26, 2016, at 7:55 AM, Apache Jenkins Server > wrote: > > >Failed CTEST tests : > > test_test_libhdfs_threaded_hdfs_static > test_test_libhdfs_zerocopy_hdfs_static Something here likely broke these tests: [Aug 24, 2016 7:47:52 AM] (aajisaka) HADOOP-13538. Deprecat

Re: [VOTE] Release Apache Hadoop 2.7.3 RC1

2016-08-17 Thread Allen Wittenauer
t; exception cases. I think it still belongs to 2.7.3 scope. > Kuhu and Kihwal, any comments here? > > > Thanks, > > Junping > > From: Allen Wittenauer > Sent: Wednesday, August 17, 2016 5:29 AM > To: common-...@hadoop.apache

Re: [VOTE] Release Apache Hadoop 2.7.3 RC1

2016-08-16 Thread Allen Wittenauer
-1 HDFS-9395 is an incompatible change: a) Why is not marked as such in the changes file? b) Why is an incompatible change in a micro release, much less a minor? c) Where is the release note for this change? > On Aug 12, 2016, at 9:45 AM, Vinod Kumar Vavilapalli > wrote: > > Hi all, > > I

Re: [Release thread] 2.6.5 release activities

2016-08-15 Thread Allen Wittenauer
> On Aug 12, 2016, at 8:19 AM, Junping Du wrote: > > In this community, we are so aggressive to drop Java 7 support in 3.0.x > release. Here, why we are so conservative to keep releasing new bits to > support Java 6? I don't view a group of people putting bug fixes into a micro relea

Re: [Release thread] 2.6.5 release activities

2016-08-11 Thread Allen Wittenauer
> On Aug 11, 2016, at 8:10 AM, Junping Du wrote: > > Allen, to be clear, I am not against any branch release effort here. However, "I'm not an X but " > as RM for previous releases 2.6.3 and 2.6.4, I feel to have responsibility to > take care branch-2.6 together with other

Re: [Release thread] 2.6.5 release activities

2016-08-11 Thread Allen Wittenauer
> On Aug 11, 2016, at 5:59 AM, Junping Du wrote: > > These comments are more like wishes but not giving more clarifications > on the needs. I would like to hear more specific reasons to not move to 2.7.x > releases but prefer to upgrade to 2.6.5. If the only reason is about > expectation

Re: [DISCUSS] The order of classpath isolation work and updating/shading dependencies on trunk

2016-07-25 Thread Allen Wittenauer
> On Jul 25, 2016, at 1:16 PM, Sangjin Lee wrote: > > Also: right now, the non-Linux and/or non-x86 platforms have to supply their > own leveldbjni jar (or at least the C level library?) in order to make YARN > even functional. How is that going to work with the class path manipulation? > >

Re: [DISCUSS] The order of classpath isolation work and updating/shading dependencies on trunk

2016-07-22 Thread Allen Wittenauer
> On Jul 22, 2016, at 5:47 PM, Zheng, Kai wrote: > > For the leveldb thing, wouldn't we have an alternative option in Java for the > platforms where leveldb isn't supported yet due to whatever reasons. IMO, > native library would be best to be used for optimization and production for > perfor

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-22 Thread Allen Wittenauer
> On Jul 22, 2016, at 7:16 PM, Andrew Wang wrote: > > Does this mean you find our current system of listing a JIRA as being fixed > in both a 2.6.x and 2.7.x to be confusing? Nope. I'm only confused when there isn't a .0 release in the fix line. When I see 2.6.x and 2.7.x I know tha

Re: [DISCUSS] The order of classpath isolation work and updating/shading dependencies on trunk

2016-07-22 Thread Allen Wittenauer
nly* helps processes that sit outside of YARN. :) > > On Fri, Jul 22, 2016 at 10:48 AM, Allen Wittenauer > wrote: > > > > Does any of this work actually help processes that sit outside of YARN? > > > >> On Jul 21, 2016, at 12:29 PM, Sean Busbey wrote: > >

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-22 Thread Allen Wittenauer
From the perspective of an end user who is reading multiple versions' listings at once, listing the same JIRA being fixed in multiple releases is totally confusing, especially now that release notes are actually readable. "So which version was it ACTUALLY fixed in?" is going to be the

Re: [DISCUSS] The order of classpath isolation work and updating/shading dependencies on trunk

2016-07-22 Thread Allen Wittenauer
Does any of this work actually help processes that sit outside of YARN? > On Jul 21, 2016, at 12:29 PM, Sean Busbey wrote: > > thanks for bringing this up! big +1 on upgrading dependencies for 3.0. > > I have an updated patch for HADOOP-11804 ready to post this week. I've > been updating HBase

[jira] [Created] (MAPREDUCE-6743) nativetask unit tests need to provide usable output

2016-07-22 Thread Allen Wittenauer (JIRA)
Allen Wittenauer created MAPREDUCE-6743: --- Summary: nativetask unit tests need to provide usable output Key: MAPREDUCE-6743 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6743 Project

[jira] [Resolved] (MAPREDUCE-6742) Test

2016-07-22 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-6742. - Resolution: Not A Problem > Test > > > Ke

Re: [DICUSS] Upgrading Guice to 4.0(HADOOP-12064)

2016-06-29 Thread Allen Wittenauer
> On Jun 29, 2016, at 10:16 AM, Tsuyoshi Ozawa wrote: > > No objections here? I talked to a handful of non-committer ISV-type folks yesterday and the general consensus was that there is an expectation that 3.x will have all the dependencies updated to something relatively modern if we can't

Re: Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2016-06-18 Thread Allen Wittenauer
> On Jun 17, 2016, at 7:04 AM, Apache Jenkins Server > wrote: > > For more details, see > https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/66/ > I suspect this change: > [Jun 16, 2016 11:17:06 AM] (vinayakumarb) HDFS-10256. Use > GenericTestUtils.getTestDir method i

ASF OS X Build Infrastructure

2016-05-19 Thread Allen Wittenauer
Some of you may not know that the ASF actually does have an OS X machine (a Mac mini, so it’s not a speed demon) in the build infrastructure. While messing around with getting all? of the trunk jobs reconfigured to do Java 8 and separate maven repos, I noticed that this box ten

Re: Hadoop CI with alternate architectures.

2016-05-18 Thread Allen Wittenauer
That’s really a question for infrastructure-...@apache.org . They manage the ASF build infrastructure which Apache Hadoop and lots of other projects utilize. (Bigtop uses something custom, which I think is funded by Cloudera.) Once it is registered with builds.apache.org, it’

Re: [DISCUSS] Set minimum version of Hadoop 3 to JDK8 (HADOOP-11858)

2016-05-17 Thread Allen Wittenauer
t 8:02 PM, Allen Wittenauer wrote: > > OK, it looks like if someone commits HADOOP-13161, then trunk only uses > JDK8 and branch-2 will use JDK7 and JDK8 during precommit with no changes > required to Apache Yetus. :D > > >> On May 16, 2016, at 5:38 PM,

  1   2   3   4   5   6   7   8   >