Nevermind I fixed the apt issue, but the current problem is that the COMPONENTS of the smoke test cannot be grouped together. IIUC we want to be able to kick off a run with COMPONENTS set to all or a subset of daemons (yarn/hdfs/etc..), that will bootstrap the hadoop cluster via Docker. I'll open a task and work on it, hopefully there is an easy solution :)
On Thu, Jan 11, 2024 at 12:11 PM Luca Toscano <[email protected]> wrote: > > I have also created: https://ci.bigtop.apache.org/job/pipeline-smoke-test/ > > It currently doesn't work since there seems to be an error when apt > runs inside the Docker container, but it is a generic script to give > you the idea. Release version and branch have a separate parameter, at > the moment we have 3.2.0 and 3.3.0, so a single script could in theory > be used multiple times with minimal changes (IIUC at the moment for > every release we create new smoke tests etc..).\ > > To test it: > > 1) Run build > 2) Open the build run's "Console output" > 3) Hit "Input Requested" and you'll see a series of dropdown options. > Once you select the right ones, the build should start. > > If you want to see an example: > https://ci.bigtop.apache.org/job/pipeline-smoke-test/5/console > > This is just an idea to gather feedback, it can surely be improved, > but it is a starting point :) > > Thanks! > > Luca > > On Wed, Jan 10, 2024 at 9:44 PM Luca Toscano <[email protected]> wrote: > > > > Hi folks, > > > > I created https://ci.bigtop.apache.org/job/test-elukey as an example > > of how Jenkins pipelines can be used to replace the Matrix Reloaded > > plugin. The prototype is limited for the moment to: > > - It uses scripted pipelines, not declarative pipelines, since the > > former is more flexible in my opinion. > > - It mimics the Smoke Tests for Debian OSes, used in our release > > process IIUC (like > > https://ci.bigtop.apache.org/view/3.2.0-smoke-tests/job/Bigtop-3.2.0-debian-10-smoke-tests/1/) > > - It only echoes values, no build is done. > > - It allows dynamic parameters (namely to restrict OS/ARCH/COMPONENT > > at runtime), but the inputs need to be added in the build's console > > output rather than having specific "Build with params" in Jenkins UI. > > This is probably something that we can circumvent, but the current > > solution avoids to copy/paste values multiple times etc.. > > - Compared to Matrix reloaded, the flexibility of selecting dynamic > > parameters is less, but it should be sufficient for the release use > > case. > > > > We'd need to deprecate Matrix Reloaded > > (https://plugins.jenkins.io/matrix-reloaded/) since the plugin was > > abandoned since a long time ago, and no more fixes are provided from > > upstream. > > > > Do you think that this process is viable? If so we could use pipelines > > as experimental step during the next release, to see if they can be > > adopted. Let me know your thoughts and if I am missing something big > > (probably happening, apologies in advance). > > > > Thanks! > > > > Luca > > > > On Sun, Jan 7, 2024 at 3:38 PM Luca Toscano <[email protected]> wrote: > > > > > > Hi folks, > > > > > > just letting you know that I have installed the Pipelines plugin in > > > the Jenkins' master, to test if we can define our jobs in there and > > > avoid the Matrix reloaded plugin. It would be also very nice to have > > > all pipelines checked out in the BigTop's repo eventually. > > > > > > I'll open a jira to track all the work after some quick tests :) > > > > > > Let me know if you see anything weird, I've just restarted Jenkins to > > > pick up the new plugin. > > > > > > Luca
