Re: Travis Builds
Getting back to this, the issue was resolved. On Sun, Jan 2, 2022 at 10:55 AM Gavin McDonald wrote: > Hi All. > > Infra is aware of an issue with our Travis account where we have currently > Zero users, as such no builds can be performed. I have emailed support and > await their response. > > Regards > > -- > > *Gavin McDonald* > Systems Administrator > ASF Infrastructure Team > -- *Gavin McDonald* Systems Administrator ASF Infrastructure Team
Re: Possibility of using ci-hadoop.a.o for Nutch integration tests
Thank you for the information. I agree with you. I totally misunderstood what ci-hadoop.a.o was. Sorry for the noise. lewismc On Wed, Jan 5, 2022 at 21:56 Akira Ajisaka wrote: > (Adding builds@) > > Hi Lewis, > > Nutch is already using ci-builds.apache.org, so I think Nutch can > continue using it. ci-hadoop.apache.org provides almost the same > functionality as ci-builds.apache.org and there is no non-production > Hadoop cluster running there. Therefore moving to ci-hadoop does not make > sense. > > Short history: In the past there were some jenkins hosts that were labeled > for Hadoop and its related projects. After the migration to cloudbees, the > labeled hosts are moved under ci-hadoop.apache.org. > > Thanks, > Akira > > > On Thu, Jan 6, 2022 at 2:20 PM lewis john mcgibbney > wrote: > >> Thank you for the response and for directing the conversation to the >> correct places. >> I may have misunderstood what ci-hadoop.apache.org actually is. We are >> looking for a non-production Hadoop cluster which we can use to simulate >> Nutch jobs. I am not sure if this is what ci-hadoop.apache.org actually >> is... >> Instead it looks like lots of compute resources used to perform Jenkins >> CI/CD tasks for Hadoop and associated projects rather than test things >> on-top of Hadoop (and associated projects). >> Any clarity on what ci-hadoop.apache.org actually is would be greatly >> appreciated. >> >> Let me also clarify my language, rather than have the integration tests >> run >> on every PR, we could trigger the integration tests to be run by tagging a >> Github bot i.e., "@nutchbot integration-test". Similar to what is done >> with >> Dependabot or conda-forge for anyon familiar with those mechanisms. >> >> Thanks for any advice or comments. >> lewismc >> >> On Wed, Jan 5, 2022 at 9:05 PM Ayush Saxena wrote: >> >> > Moved to Dev lists. >> > >> > Not sure about this though: >> > when a PR is submitted to Nutch project it will run some MR job in >> Hadoop >> > CI. >> > >> > Whatever that PR requires should run as part of Nutch Infra. Why in >> Hadoop >> > CI? >> > Our CI is already loaded with our own workloads. >> > If by any chance the above assertion gets a pass, then secondly we have >> > very less number of people managing work related to CI and Infra. I >> don’t >> > think most of the people won’t have context or say in the Nutch project, >> > neither bandwidth to fix stuff if it gets broken. >> > >> > Just my thoughts. Looped in the dev lists, if others have any feedback. >> As >> > for the process, this would require a consensus from the Hadoop PMC >> > >> > -Ayush >> > >> > > On 06-Jan-2022, at 7:02 AM, lewis john mcgibbney >> > wrote: >> > > >> > > Hi general@, >> > > >> > > Not sure if this is the correct mailing list. Please redirect me if >> there >> > > is a more suitable location. Thank you >> > > >> > > I am PMC over on the Nutch project (https://nutch.apache.org). I >> would >> > like >> > > to investigate whether we can build an integration testing capability >> for >> > > the project. This would involve running a Nutch integration test suite >> > > (collection of MR jobs) in a Hadoop CI environment. For example >> whenever >> > a >> > > pull request is submitted to the Nutch project. This could easily be >> > > automated through Jenkins. >> > > >> > > I’m not sure if this is something the Hadoop PMC would consider. Thank >> > you >> > > for the consideration. >> > > >> > > lewismc >> > > -- >> > > http://home.apache.org/~lewismc/ >> > > http://people.apache.org/keys/committer/lewismc >> > >> >> >> -- >> http://home.apache.org/~lewismc/ >> http://people.apache.org/keys/committer/lewismc >> > -- http://home.apache.org/~lewismc/ http://people.apache.org/keys/committer/lewismc
Re: Possibility of using ci-hadoop.a.o for Nutch integration tests
(Adding builds@) Hi Lewis, Nutch is already using ci-builds.apache.org, so I think Nutch can continue using it. ci-hadoop.apache.org provides almost the same functionality as ci-builds.apache.org and there is no non-production Hadoop cluster running there. Therefore moving to ci-hadoop does not make sense. Short history: In the past there were some jenkins hosts that were labeled for Hadoop and its related projects. After the migration to cloudbees, the labeled hosts are moved under ci-hadoop.apache.org. Thanks, Akira On Thu, Jan 6, 2022 at 2:20 PM lewis john mcgibbney wrote: > Thank you for the response and for directing the conversation to the > correct places. > I may have misunderstood what ci-hadoop.apache.org actually is. We are > looking for a non-production Hadoop cluster which we can use to simulate > Nutch jobs. I am not sure if this is what ci-hadoop.apache.org actually > is... > Instead it looks like lots of compute resources used to perform Jenkins > CI/CD tasks for Hadoop and associated projects rather than test things > on-top of Hadoop (and associated projects). > Any clarity on what ci-hadoop.apache.org actually is would be greatly > appreciated. > > Let me also clarify my language, rather than have the integration tests run > on every PR, we could trigger the integration tests to be run by tagging a > Github bot i.e., "@nutchbot integration-test". Similar to what is done with > Dependabot or conda-forge for anyon familiar with those mechanisms. > > Thanks for any advice or comments. > lewismc > > On Wed, Jan 5, 2022 at 9:05 PM Ayush Saxena wrote: > > > Moved to Dev lists. > > > > Not sure about this though: > > when a PR is submitted to Nutch project it will run some MR job in > Hadoop > > CI. > > > > Whatever that PR requires should run as part of Nutch Infra. Why in > Hadoop > > CI? > > Our CI is already loaded with our own workloads. > > If by any chance the above assertion gets a pass, then secondly we have > > very less number of people managing work related to CI and Infra. I don’t > > think most of the people won’t have context or say in the Nutch project, > > neither bandwidth to fix stuff if it gets broken. > > > > Just my thoughts. Looped in the dev lists, if others have any feedback. > As > > for the process, this would require a consensus from the Hadoop PMC > > > > -Ayush > > > > > On 06-Jan-2022, at 7:02 AM, lewis john mcgibbney > > wrote: > > > > > > Hi general@, > > > > > > Not sure if this is the correct mailing list. Please redirect me if > there > > > is a more suitable location. Thank you > > > > > > I am PMC over on the Nutch project (https://nutch.apache.org). I would > > like > > > to investigate whether we can build an integration testing capability > for > > > the project. This would involve running a Nutch integration test suite > > > (collection of MR jobs) in a Hadoop CI environment. For example > whenever > > a > > > pull request is submitted to the Nutch project. This could easily be > > > automated through Jenkins. > > > > > > I’m not sure if this is something the Hadoop PMC would consider. Thank > > you > > > for the consideration. > > > > > > lewismc > > > -- > > > http://home.apache.org/~lewismc/ > > > http://people.apache.org/keys/committer/lewismc > > > > > -- > http://home.apache.org/~lewismc/ > http://people.apache.org/keys/committer/lewismc >
