Hello, This is a mail I sent a month or so ago to the Release Engineering internal mailing list. Kunal Mehta pointed out it is worth publishing for later reference.
The topic are: * Accessing the Jenkins API * Trigger a job via the API * Zuul gearman protocol (Zuul triggers jobs in Jenkins via Gearman) * Fetching a patch from Zuul merger == Access Jenkins API == Jenkins has a REST API with entry points for almost everything you might need. The base URL is https://integration.wikimedia.org/ci/ To interact you will need some credentials that grants you appropriate rights such as triggering a build. For Zuul I created a user local to Jenkins 'zuul-bot': https://integration.wikimedia.org/ci/user/zuul-bot/ You might want to create one in wikitech instead so it ends up being registered in labs LDAP. To get your user API key head to: https://integration.wikimedia.org/ci/user/<USERNAME>/configure Click the [Show API Token...] button and you will be enlightened with an API key Then set rights for that user: * left bar [Manage Jenkins] * [Configure Global Security] https://integration.wikimedia.org/ci/configureSecurity/ There is an "Authorization" section. Add your user to the list with the input box at the bottom then check the same boxes as the 'zuul-bot' user. To create new jobs you can use JJB. The tutorial is at: https://www.mediawiki.org/wiki/CI/JJB Or create a new job via the Jenkins web GUI [New Item]. Rest of team can assist there. == Trigger a job via Jenkins API == Once job is created, the footer has a link [REST API] which describe what can be done with the job. Example: https://integration.wikimedia.org/ci/job/hashar-test-gradle/api/ You will want to build with parameters by POSTING a form data to: https://integration.wikimedia.org/ci/job/hashar-test-gradle/buildWithParameters Jenkins will send you back a Location: pointing to the representation of the build in Jenkins running queue. You might want to poll that item to get its status, and once it is executed you will get the job build number. That is a bit problematic though. When Zuul used the Jenkins API, they would generate a UUID passed as a job parameter The build request is inserted in a queue, and until it is processed no build number is assigned to it. You will need it to retrieve the build status and result I found out all of that fairly recently while reviewing a change in python-jenkins, see my comment on: https://review.openstack.org/#/c/201794/2/jenkins/__init__.py,unified When Zuul used the Jenkins API, it had an HTTP end point to receive notifications from Jenkins and each Jenkins job emitted a notification at start and end of job with the build status. Turns out the plugin as some NPE and it is no more used by Zuul so I got rid of it: https://phabricator.wikimedia.org/T93321 python-jenkins has some documentation: https://python-jenkins.readthedocs.org/en/latest/example.html == Here comes Gearman == Nowadays there is a Gearman server between Zuul and Jenkins that is spawned by Zuul. When Jenkins starts, all its jobs are registered as Gearman functions and Zuul simply ask the function to be executed passing it a bunch of parameters such as the GIT repo, ref, commit sha1 etc. Maybe harbormaster can trigger jobs directly via Gearman ? The protocol is fairly simple and other TCP. It has a few pure PHP implementations: https://packagist.org/search/?q=gearman And a couple native extensions: http://gearman.org/download/#php The Zuul Gearman protocol is described on: http://docs.openstack.org/infra/zuul/launchers.html#zuul-gearman-protocol So in short trigger a Gearman function: build:name_of_jenkins_job I am not sure how the Zuul scheduler will handle functions being triggered by another process. Might have some side effects that confuse Zuul. == To conclude == Zuul pass the git repo url / sha1 / ref to fetch to the Jenkins job as parameter. You can then use git to retrieve the patch with something that looks like: git clone $ZUUL_URL/$ZUUL_PROJECT git fetch $ZUUL_REF git checkout -b $ZUUL_CHANGE/$ZUUL_PATCHSET $ZUUL_COMMIT That is more or less explained at http://docs.openstack.org/infra/zuul/launchers.html#jenkins-git-plugin-configuration Run tests! -- Antoine Musso _______________________________________________ RelEng mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/releng -- Antoine "hashar" Musso _______________________________________________ QA mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/qa
