Hi Gavin, Perfect!
Thank you very much for your dedicated support of the ASF! Regards, Dave Sent from my iPhone > On Feb 14, 2017, at 4:20 PM, Gavin McDonald <[email protected]> wrote: > > >> On 10 Feb 2017, at 11:46 am, Dave Fisher <[email protected]> wrote: >> >> Thanks for the explanation. Is there documentation anywhere about Apache >> infrastructure's standards and requirements for external slaves? > > There wasn’t one but now there is ;) > > https://reference.apache.org/committer/node-hosting > > Please let me know if you have any further questions. > > Gav… > > > > > >> >> Regards, >> Dave >> >> Sent from my iPhone >> >>> On Feb 9, 2017, at 4:32 PM, Greg Stein <[email protected]> wrote: >>> >>> On Thu, Feb 9, 2017 at 5:53 PM, Allen Wittenauer >>> <[email protected]> >>> wrote: >>>> ... >>> >>>> The Mac OS X host was shut down literally a day after I sent out >>>> an email to common-dev@hadoop announcing I had full build and patch >>>> testing working. I had spent quite a bit of time getting Apache Yetus >>>> ported over to work on Apache's OS X machine, then spent over a month on >>>> working out the Hadoop specifics, running build after build after build. >>>> Competing with the Apache Mesos jobs that also ran on that box. The reason >>>> I was told it was killed was: "no one was using it". (Umm, what? Clearly >>>> no one bothered looking at the build log.) >>>> >>> >>> This occurred before I started working as the Infrastructure Administrator >>> (last Fall). I don't know the full background, other than a PMC requested >>> that buildbot, then never used it. Yeah: maybe the build logs weren't >>> examined to see that other projects had hopped onto it. >>> >>> I also believe we had to pay for that box, and it wasn't cheap. >>> >>> Today, our preferred model for non-Ubuntu boxes is to have other people >>> own/run/manage those buildbots and hook them into our buildmaster. For >>> example, people on the Apache Subversion project have several such 'bots. >>> >>> We are concentrating our in-house experience on the Ubuntu platform, from >>> both an operational and a cost angle. Four years ago, the Infra team had >>> many fewer projects to support. Today, we have hundreds of projects and >>> many thousands of committers to support. We've had to reallocate in order >>> to meet the incredible growth of the ASF. >>> >>> Unfortunately, especially for yourself and some others, the "smoothing down >>> the edges" has been detrimental. >>> >>> In parallel, I started working on the Solaris box.... which was >>>> then promptly shutdown not too long after I had filed a jira to see if we >>>> could get the base CA certificates upgraded. (which was pretty much all I >>>> needed, after that I could have finished getting the Hadoop builds working >>>> on it as well). >>>> >>> >>> We're still shutting down Solaris. Only one guy has experience with it, and >>> he's also got a ton of other stuff to do. >>> >>> Our hardware that runs Solaris is also *very* old. Worse: we could never >>> get a support contract for it. They wouldn't sell us one (messed up, but >>> there it is). We really need to get that box fully shut down, unracked, and >>> thrown out. >>> >>> These were huge blows to Apache Hadoop, as one of the common >>>> complaints amongst committers is the lack of resources to do cross platform >>>> testing. Given the ASF had that infrastructure in place, being in this >>>> position was kind of dumb of the project. Now the machines are gone and as >>>> a result, the portability of the code is still very hit or miss and the ASF >>>> is worse for it. >>>> >>> >>> Apache Hadoop is worse for it. As Gavin has noted, just in the past year, >>> we've increased our build farm dramatically. I believe the ASF is better >>> for it. We also have a team better focused to support the growth of the ASF. >>> >>> We can all agree that turning off services sucks for some projects and >>> people. But our growth has made demands upon the Foundation and its Infra >>> team that have forced our hand. We also have a funding model that just >>> doesn't support us hiring a team large enough to retain the disparate array >>> of services that we offered in the past. >>> >>> >>>> Since that time, I've helped get the PowerPC build up and running, >>>> but that's been about it... and even then, I spend little-to-no time on the >>>> ASF-side of the build bits for those projects I'm interested simply because >>>> I have no idea if I'll be wasting my time because "whoops, we've changed >>>> direction again". >>> >>> >>> Again, we'll happily link any buildbot into our buildmaster, so you can >>> automate builds on your special bots. As you can see from above, we won't >>> be doing PowerPC. Just Ubuntu for all machines and services from now on. >>> This allows us (via Puppet) to easily reallocate, move, upgrade, and >>> maintain our services. Years ago, each machine was manually configured, and >>> when it went down, the Foundation suffered. Today, if a machine goes down, >>> we can spin it back up in an hour or two due to the consistency. >>> >>> I do sympathize that our service reduction is painful. But I hope you can >>> understand where the Foundation (and its Infra team) is coming from. We >>> have vastly more projects to support today, meaning more uniformity is >>> required. >>> >>> Sincerely, >>> Greg Stein, >>> Infrastructure Administrator, ASF >> >
