(Adding builds@) Hi Lewis,
Nutch is already using ci-builds.apache.org, so I think Nutch can continue using it. ci-hadoop.apache.org provides almost the same functionality as ci-builds.apache.org and there is no non-production Hadoop cluster running there. Therefore moving to ci-hadoop does not make sense. Short history: In the past there were some jenkins hosts that were labeled for Hadoop and its related projects. After the migration to cloudbees, the labeled hosts are moved under ci-hadoop.apache.org. Thanks, Akira On Thu, Jan 6, 2022 at 2:20 PM lewis john mcgibbney <lewi...@apache.org> wrote: > Thank you for the response and for directing the conversation to the > correct places. > I may have misunderstood what ci-hadoop.apache.org actually is. We are > looking for a non-production Hadoop cluster which we can use to simulate > Nutch jobs. I am not sure if this is what ci-hadoop.apache.org actually > is... > Instead it looks like lots of compute resources used to perform Jenkins > CI/CD tasks for Hadoop and associated projects rather than test things > on-top of Hadoop (and associated projects). > Any clarity on what ci-hadoop.apache.org actually is would be greatly > appreciated. > > Let me also clarify my language, rather than have the integration tests run > on every PR, we could trigger the integration tests to be run by tagging a > Github bot i.e., "@nutchbot integration-test". Similar to what is done with > Dependabot or conda-forge for anyon familiar with those mechanisms. > > Thanks for any advice or comments. > lewismc > > On Wed, Jan 5, 2022 at 9:05 PM Ayush Saxena <ayush...@gmail.com> wrote: > > > Moved to Dev lists. > > > > Not sure about this though: > > when a PR is submitted to Nutch project it will run some MR job in > Hadoop > > CI. > > > > Whatever that PR requires should run as part of Nutch Infra. Why in > Hadoop > > CI? > > Our CI is already loaded with our own workloads. > > If by any chance the above assertion gets a pass, then secondly we have > > very less number of people managing work related to CI and Infra. I don’t > > think most of the people won’t have context or say in the Nutch project, > > neither bandwidth to fix stuff if it gets broken. > > > > Just my thoughts. Looped in the dev lists, if others have any feedback. > As > > for the process, this would require a consensus from the Hadoop PMC > > > > -Ayush > > > > > On 06-Jan-2022, at 7:02 AM, lewis john mcgibbney <lewi...@apache.org> > > wrote: > > > > > > Hi general@, > > > > > > Not sure if this is the correct mailing list. Please redirect me if > there > > > is a more suitable location. Thank you > > > > > > I am PMC over on the Nutch project (https://nutch.apache.org). I would > > like > > > to investigate whether we can build an integration testing capability > for > > > the project. This would involve running a Nutch integration test suite > > > (collection of MR jobs) in a Hadoop CI environment. For example > whenever > > a > > > pull request is submitted to the Nutch project. This could easily be > > > automated through Jenkins. > > > > > > I’m not sure if this is something the Hadoop PMC would consider. Thank > > you > > > for the consideration. > > > > > > lewismc > > > -- > > > http://home.apache.org/~lewismc/ > > > http://people.apache.org/keys/committer/lewismc > > > > > -- > http://home.apache.org/~lewismc/ > http://people.apache.org/keys/committer/lewismc >