> On Aug 11, 2019, at 11:55 PM, Zhenyu Zheng <zhengzhenyul...@gmail.com> wrote:
> So, is here the correct place to ask for help and propose to add this CI?
This is the place to come ask questions related to Apache Yetus, not
necessarily any particular project’s usage of it.
> We are willing to provide resources for the CI and man power to setup the
> CI and keep it green in the future. We have already found some issues and
> started to working on fixing them in the community, such as
> https://issues.apache.org/jira/browse/HADOOP-13363 , but it will be better
> to show the process of what we have fixed and overall view of what is still
> wrong if we can have an ARM CI in upstream. Here is an example for what we
> have recently done for Apache Flink:
> https://issues.apache.org/jira/browse/INFRA-18822
There are two routes to take:
#1: Getting an ARM machine hooked up to the ASF Jenkins system
INFRA is your best contact for doing that. After that, you’ll need to
work with the Hadoop community on getting jobs/processes in place to make ARM a
first class citizen. (One of the big ones will be making an ARM-specific
Dockerfile!)
Unfortunately, the folks in charge of Hadoop have a track record of not
being particularly friendly to contributions outside of the clique or if it
doesn’t further goals by their employers. The only way to really grab their
attention is to start an email campaign on common-...@hadoop.apache.org. Just
be aware that you’ll need to be extremely persistent to get anywhere. It’s
probably also worth mentioning that attempts to get other architectures/OSes
(Windows, PowerPC, OS X, etc) as first class CI tests have failed multiple
times because, again, the majority of the community’s employers don’t work with
that architecture and do not have a vested interested in making it succeed.
#2: Going a completely different route with OpenLab CI
If you plan on following the same route as you did with Apache Flink
(bypass Jenkins and do something completely custom), then you’ll likely be
interested in integrating Apache Yetus with OpenLab CI. This is where we can
help. But to get started, you’ll want to create some robot support:
Code:
https://github.com/apache/yetus/tree/master/precommit/src/main/shell/robots.d
Docs:
http://yetus.apache.org/documentation/in-progress/precommit-robots/
I’d recommend starting with a source tree you know well or is
relatively simple while building the support before tackling Hadoop straight
on. Yetus tends to shed a lot of light on short-signed CI system design so
there may be some workarounds required and/or new Yetus APIs we’ll need to add.
To make matters worse, Hadoop is notoriously broken in lots of ways, so has a
tendency to light up quality test results that will make it seem like your
robot code is broken.
After you get the robot support, you’re likely back to talking with the
Hadoop community about how to integrate your CI system with their contribution
procedures. So ultimately, you’ll be back to convincing them that this is
something important to work on. You really need to find a champion on the PMC
to help make it happen.
Good luck!
