Re: ASF OS X Build Infrastructure
Hi Ravi, Something certainly seems off about that bootstrapping problem you encountered. :-) When I've done this, the artifact I downloaded was an .iso file, which I could then use to install a VirtualBox VM. I'm now tuned into the discussion Sean referenced about the ASF MSDN program. I'll send another update when I have something more specific to share. --Chris Nauroth From: Ravi Prakash <ravihad...@gmail.com<mailto:ravihad...@gmail.com>> Date: Friday, May 20, 2016 at 4:56 PM To: Sean Busbey <bus...@cloudera.com<mailto:bus...@cloudera.com>> Cc: Chris Nauroth <cnaur...@hortonworks.com<mailto:cnaur...@hortonworks.com>>, Steve Loughran <ste...@hortonworks.com<mailto:ste...@hortonworks.com>>, Hadoop Common <common-...@hadoop.apache.org<mailto:common-...@hadoop.apache.org>>, "mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.apache.org>" <mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.apache.org>>, "hdfs-...@hadoop.apache.org<mailto:hdfs-...@hadoop.apache.org>" <hdfs-...@hadoop.apache.org<mailto:hdfs-...@hadoop.apache.org>>, "yarn-...@hadoop.apache.org<mailto:yarn-...@hadoop.apache.org>" <yarn-...@hadoop.apache.org<mailto:yarn-...@hadoop.apache.org>> Subject: Re: ASF OS X Build Infrastructure FWIW, I was able to get a response from the form last month. I was issued a new MSDN subscriber ID using which I could have downloaded Microsoft Visual Studio (and some other products, I think). I was interested in downloading an image of Windows to run in a VM, but the downloader is. wait for it. an exe file :-) Haven't gotten around to begging someone with a Windows OS to run that image downloader. On Fri, May 20, 2016 at 10:39 AM, Sean Busbey <bus...@cloudera.com<mailto:bus...@cloudera.com>> wrote: Some talk about the MSDN-for-committers program recently passed by on a private list. It's still active, it just changed homes within Microsoft. The info should still be in the committer repo. If something is amiss please let me know and I'll pipe up to the folks already plugged in to confirming it's active. On Fri, May 20, 2016 at 12:13 PM, Chris Nauroth <cnaur...@hortonworks.com<mailto:cnaur...@hortonworks.com>> wrote: > It's very disappointing to see that vanish. I'm following up to see if I > can learn more about what happened or if I can do anything to help > reinstate it. > > --Chris Nauroth > > > > > On 5/20/16, 6:11 AM, "Steve Loughran" > <ste...@hortonworks.com<mailto:ste...@hortonworks.com>> wrote: > >> >>> On 20 May 2016, at 10:40, Lars Francke >>> <lars.fran...@gmail.com<mailto:lars.fran...@gmail.com>> wrote: >>> >>>> >>>> Regarding lack of personal access to anything but Linux, I'll take >>>>this as >>>> an opportunity to remind everyone that ASF committers (not just >>>>limited to >>>> Hadoop committers) are entitled to a free MSDN license, which can get >>>>you >>>> a Windows VM for validating Windows issues and any patches that touch >>>> cross-platform concerns, like the native code. Contributors who are >>>>not >>>> committers still might struggle to get access to Windows, but all of us >>>> reviewing and committing patches do have access. >>>> >>> >>> Actually, from all I can tell this MSDN offer has been discontinued for >>> now. All the information has been removed from the committers repo. Do >>>you >>> have any more up to date information on this? >>> >> >> >>That's interesting. >> >>I did an SVN update and it went away..looks like something happened on >>April 26 >> >>No idea, though the svn log has a bit of detail >> >>- >>To unsubscribe, e-mail: >>mapreduce-dev-unsubscr...@hadoop.apache.org<mailto:mapreduce-dev-unsubscr...@hadoop.apache.org> >>For additional commands, e-mail: >>mapreduce-dev-h...@hadoop.apache.org<mailto:mapreduce-dev-h...@hadoop.apache.org> >> >> > > > - > To unsubscribe, e-mail: > common-dev-unsubscr...@hadoop.apache.org<mailto:common-dev-unsubscr...@hadoop.apache.org> > For additional commands, e-mail: > common-dev-h...@hadoop.apache.org<mailto:common-dev-h...@hadoop.apache.org> > -- busbey - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org<mailto:mapreduce-dev-unsubscr...@hadoop.apache.org> For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org<mailto:mapreduce-dev-h...@hadoop.apache.org>
Re: ASF OS X Build Infrastructure
FWIW, I was able to get a response from the form last month. I was issued a new MSDN subscriber ID using which I could have downloaded Microsoft Visual Studio (and some other products, I think). I was interested in downloading an image of Windows to run in a VM, but the downloader is. wait for it. an exe file :-) Haven't gotten around to begging someone with a Windows OS to run that image downloader. On Fri, May 20, 2016 at 10:39 AM, Sean Busbeywrote: > Some talk about the MSDN-for-committers program recently passed by on a > private > list. It's still active, it just changed homes within Microsoft. The > info should still be in the committer repo. If something is amiss > please let me know and I'll pipe up to the folks already plugged in to > confirming it's active. > > On Fri, May 20, 2016 at 12:13 PM, Chris Nauroth > wrote: > > It's very disappointing to see that vanish. I'm following up to see if I > > can learn more about what happened or if I can do anything to help > > reinstate it. > > > > --Chris Nauroth > > > > > > > > > > On 5/20/16, 6:11 AM, "Steve Loughran" wrote: > > > >> > >>> On 20 May 2016, at 10:40, Lars Francke wrote: > >>> > > Regarding lack of personal access to anything but Linux, I'll take > this as > an opportunity to remind everyone that ASF committers (not just > limited to > Hadoop committers) are entitled to a free MSDN license, which can get > you > a Windows VM for validating Windows issues and any patches that touch > cross-platform concerns, like the native code. Contributors who are > not > committers still might struggle to get access to Windows, but all of > us > reviewing and committing patches do have access. > > >>> > >>> Actually, from all I can tell this MSDN offer has been discontinued for > >>> now. All the information has been removed from the committers repo. Do > >>>you > >>> have any more up to date information on this? > >>> > >> > >> > >>That's interesting. > >> > >>I did an SVN update and it went away..looks like something happened on > >>April 26 > >> > >>No idea, though the svn log has a bit of detail > >> > >>- > >>To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org > >>For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org > >> > >> > > > > > > - > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > > > > > > -- > busbey > > - > To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org > >
Re: ASF OS X Build Infrastructure
Some talk about the MSDN-for-committers program recently passed by on a private list. It's still active, it just changed homes within Microsoft. The info should still be in the committer repo. If something is amiss please let me know and I'll pipe up to the folks already plugged in to confirming it's active. On Fri, May 20, 2016 at 12:13 PM, Chris Naurothwrote: > It's very disappointing to see that vanish. I'm following up to see if I > can learn more about what happened or if I can do anything to help > reinstate it. > > --Chris Nauroth > > > > > On 5/20/16, 6:11 AM, "Steve Loughran" wrote: > >> >>> On 20 May 2016, at 10:40, Lars Francke wrote: >>> Regarding lack of personal access to anything but Linux, I'll take this as an opportunity to remind everyone that ASF committers (not just limited to Hadoop committers) are entitled to a free MSDN license, which can get you a Windows VM for validating Windows issues and any patches that touch cross-platform concerns, like the native code. Contributors who are not committers still might struggle to get access to Windows, but all of us reviewing and committing patches do have access. >>> >>> Actually, from all I can tell this MSDN offer has been discontinued for >>> now. All the information has been removed from the committers repo. Do >>>you >>> have any more up to date information on this? >>> >> >> >>That's interesting. >> >>I did an SVN update and it went away..looks like something happened on >>April 26 >> >>No idea, though the svn log has a bit of detail >> >>- >>To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org >>For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org >> >> > > > - > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > -- busbey - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
Re: ASF OS X Build Infrastructure
It's very disappointing to see that vanish. I'm following up to see if I can learn more about what happened or if I can do anything to help reinstate it. --Chris Nauroth On 5/20/16, 6:11 AM, "Steve Loughran"wrote: > >> On 20 May 2016, at 10:40, Lars Francke wrote: >> >>> >>> Regarding lack of personal access to anything but Linux, I'll take >>>this as >>> an opportunity to remind everyone that ASF committers (not just >>>limited to >>> Hadoop committers) are entitled to a free MSDN license, which can get >>>you >>> a Windows VM for validating Windows issues and any patches that touch >>> cross-platform concerns, like the native code. Contributors who are >>>not >>> committers still might struggle to get access to Windows, but all of us >>> reviewing and committing patches do have access. >>> >> >> Actually, from all I can tell this MSDN offer has been discontinued for >> now. All the information has been removed from the committers repo. Do >>you >> have any more up to date information on this? >> > > >That's interesting. > >I did an SVN update and it went away..looks like something happened on >April 26 > >No idea, though the svn log has a bit of detail > >- >To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org >For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org > > - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
Re: ASF OS X Build Infrastructure
> > Regarding lack of personal access to anything but Linux, I'll take this as > an opportunity to remind everyone that ASF committers (not just limited to > Hadoop committers) are entitled to a free MSDN license, which can get you > a Windows VM for validating Windows issues and any patches that touch > cross-platform concerns, like the native code. Contributors who are not > committers still might struggle to get access to Windows, but all of us > reviewing and committing patches do have access. > Actually, from all I can tell this MSDN offer has been discontinued for now. All the information has been removed from the committers repo. Do you have any more up to date information on this? It has long been on my TODO list to set up similar Jenkins jobs for > Windows, but it keeps slipping. I'll try once again to bump up priority. > > --Chris Nauroth > > > > > On 5/19/16, 9:41 AM, "Allen Wittenauer"wrote: > > > > > Some of you may not know that the ASF actually does have an OS X > machine > >(a Mac mini, so it¹s not a speed demon) in the build infrastructure. > >While messing around with getting all? of the trunk jobs reconfigured to > >do Java 8 and separate maven repos, I noticed that this box tends to sit > >idle most of the day. Why take advantage of it? Therefore, I also setup > >two jobs for us to use to help alleviate the ³I don¹t have access to > >anything but Linux² excuse when writing code that may not work in a > >portable manner. > > > >Jobs #1: > > > > > https://builds.apache.org/view/H-L/view/Hadoop/job/Precommit-HADOOP-OSX > > > > This basically runs Apache Yetus precommit with quite a few of the > >unnecessary tests disabled. For example, there¹s no point in running > >checkstyle. Note that this job takes the *full* JIRA issue id as input. > >So ŒHADOOP-9902¹ not Œ9902¹. This allows for one Jenkins job to be used > >for all the Hadoop sub-projects (HADOOP, HDFS, MR, YARN). ³But my code > >is on github and I don¹t want to upload a patch!² I haven¹t tested it, > >but it should also take a URL, so just add a .diff to the end of your > >github compare URL and put that in the issue box. It hypothetically > >should work. > > > >Job #2: > > > > I¹m still hammering on this one because the email notifications > aren¹t > >working to my satisfaction plus we have some extremely Linux-specific > >code in YARNŠ but > > > > > https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-trunk-osx-java8 > >/ > > > > Š is a ³build the world² job similar to what is currently running > under > >the individual sub projects. (This actually makes it one of the few > >³build everything² jobs we have running. Most of the other jobs only > >build that particular sub project.). It does not run the full unit test > >suite and it also does not build all of the native code. This gives us a > >place to start on our journey of making Hadoop actually, truly run > >everywhere. (Interesting side note: It¹s been *extremely* consistent in > >what fails vs. the Linux build hosts.) > > > > At some point, likely after YETUS-390 is complete, I¹ll switch > this job > >over to be run by Apache Yetus in qbt mode so that it¹s actually easier > >to track failures across all dirs. A huge advantage over raw maven > >commands. > > > > Happy testing everyone. > > > > NOTE: if you don¹t have access to launch jobs on builds.apache.org > , > >you¹ll need to send a request to private@. The Apache Hadoop PMC has the > >keys to give access to folks. > > > > > > > >- > >To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org > >For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org > > > > > > > - > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org > >
Re: ASF OS X Build Infrastructure
> On 19 May 2016, at 18:08, Chris Naurothwrote: > > Allen, thank you for doing this. > > Regarding lack of personal access to anything but Linux, I'll take this as > an opportunity to remind everyone that ASF committers (not just limited to > Hadoop committers) are entitled to a free MSDN license, which can get you > a Windows VM for validating Windows issues and any patches that touch > cross-platform concerns, like the native code. Contributors who are not > committers still might struggle to get access to Windows, but all of us > reviewing and committing patches do have access. > It gives you Azure credentials too, which lets you test the hadoop-azure module *from the OS of your choice* - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
Re: ASF OS X Build Infrastructure
Allen, thank you for doing this. Regarding lack of personal access to anything but Linux, I'll take this as an opportunity to remind everyone that ASF committers (not just limited to Hadoop committers) are entitled to a free MSDN license, which can get you a Windows VM for validating Windows issues and any patches that touch cross-platform concerns, like the native code. Contributors who are not committers still might struggle to get access to Windows, but all of us reviewing and committing patches do have access. It has long been on my TODO list to set up similar Jenkins jobs for Windows, but it keeps slipping. I'll try once again to bump up priority. --Chris Nauroth On 5/19/16, 9:41 AM, "Allen Wittenauer"wrote: > > Some of you may not know that the ASF actually does have an OS X machine >(a Mac mini, so it¹s not a speed demon) in the build infrastructure. >While messing around with getting all? of the trunk jobs reconfigured to >do Java 8 and separate maven repos, I noticed that this box tends to sit >idle most of the day. Why take advantage of it? Therefore, I also setup >two jobs for us to use to help alleviate the ³I don¹t have access to >anything but Linux² excuse when writing code that may not work in a >portable manner. > >Jobs #1: > > https://builds.apache.org/view/H-L/view/Hadoop/job/Precommit-HADOOP-OSX > > This basically runs Apache Yetus precommit with quite a few of the >unnecessary tests disabled. For example, there¹s no point in running >checkstyle. Note that this job takes the *full* JIRA issue id as input. >So ŒHADOOP-9902¹ not Œ9902¹. This allows for one Jenkins job to be used >for all the Hadoop sub-projects (HADOOP, HDFS, MR, YARN). ³But my code >is on github and I don¹t want to upload a patch!² I haven¹t tested it, >but it should also take a URL, so just add a .diff to the end of your >github compare URL and put that in the issue box. It hypothetically >should work. > >Job #2: > > I¹m still hammering on this one because the email notifications aren¹t >working to my satisfaction plus we have some extremely Linux-specific >code in YARNŠ but > > > https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-trunk-osx-java8 >/ > > Š is a ³build the world² job similar to what is currently running under >the individual sub projects. (This actually makes it one of the few >³build everything² jobs we have running. Most of the other jobs only >build that particular sub project.). It does not run the full unit test >suite and it also does not build all of the native code. This gives us a >place to start on our journey of making Hadoop actually, truly run >everywhere. (Interesting side note: It¹s been *extremely* consistent in >what fails vs. the Linux build hosts.) > > At some point, likely after YETUS-390 is complete, I¹ll switch this job >over to be run by Apache Yetus in qbt mode so that it¹s actually easier >to track failures across all dirs. A huge advantage over raw maven >commands. > > Happy testing everyone. > > NOTE: if you don¹t have access to launch jobs on builds.apache.org, >you¹ll need to send a request to private@. The Apache Hadoop PMC has the >keys to give access to folks. > > > >- >To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org >For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org > > - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org