Hi folks, I just wanted to write up an update of the work I’ve been doing over the last few days so everyone was aware of the changes.
We currently use GitLab CI to test all software changes, and this was a combination of GitLab cloud runners (Linux Docker images created by ourselves) and AppVeyor (Windows MSVC/Cygwin/MinGW using their provided images. These have worked but were lacking in a few aspects. The first was a limited quota of CI minutes for the GitLab runners which we were exceeding. The second was that the AppVeyor builds were very slow, and the images were not updated regularly. The other aspect was a lack of support for other platforms such as MacOS and BSDs, as well as others. The change is that I’ve replaced these with self-hosted machines. This comes with some benefits and some downsides. The machines: 1) Apple M4 Mac Mini. This hosts a MacOS 26 virtual machine using UTM; AppleClang toolchain with build dependencies provided by homebrew 2) Minisforum MS-A1 with an AMD Ryzen 7 8700G, 64 GB RAM and 2TB mirrored storage. This runs FreeBSD 15.0-RELEASE with all the filesystems on ZFS, so they can be automatically snapshotted and backed up on a continuous basis. This hosts a mixture of FreeBSD jails and Bhyve virtual machines on a common ethernet bridge: a) FreeBSD 15.0-RELEASE runner (jail) b) Alpine Linux runner using docker (vm); while the base is a minimal Alpine image with docker and the runner, all CI jobs run in docker using any image of choice c) Windows 11 runner (vm) with Visual Studio 2026, Visual Studio 2022, Cygwin and MinGW64/MSYS2. MSVC build dependencies provided by vcpkg; Cygwin and pacman packages used for Cygwin and MinGW/MSYS It hosts other things as well, but these are the resources allocated for libtiff CI. This is a short overview of the configuration: # iocage list +-----+-------------------------+-------+--------------+------+ | JID | NAME | STATE | RELEASE | IP4 | +=====+=========================+=======+==============+======+ ... | 4 | erith.codelibre.net | up | 15.0-RELEASE | DHCP | +-----+-------------------------+-------+--------------+———+ # vm list NAME DATASTORE LOADER CPU MEMORY VNC AUTO STATE ablar default uefi 2 8G 0.0.0.0:5900 Yes [2] Running (20483) emarin default uefi 2 8G - Yes [1] Running (47430) # zfs list -r zvirt NAME USED AVAIL REFER MOUNTPOINT zvirt 92.1G 807G 96K /zvirt zvirt/bhyve 58.8G 807G 7.44G /zvirt/bhyve zvirt/bhyve/ablar 42.1G 807G 42.1G /zvirt/bhyve/ablar zvirt/bhyve/emarin 9.23G 807G 9.23G /zvirt/bhyve/emarin zvirt/iocage 33.2G 807G 144K /zvirt/iocage zvirt/iocage/download 458M 807G 96K /zvirt/iocage/download zvirt/iocage/download/15.0-RELEASE 458M 807G 458M /zvirt/iocage/download/15.0-RELEASE zvirt/iocage/images 8.60G 807G 8.60G /zvirt/iocage/images zvirt/iocage/jails 22.8G 807G 128K /zvirt/iocage/jails ... zvirt/iocage/jails/erith.codelibre.net 7.60G 807G 108K /zvirt/iocage/jails/erith.codelibre.net zvirt/iocage/jails/erith.codelibre.net/data 96K 807G 96K /zvirt/iocage/jails/erith.codelibre.net/data zvirt/iocage/jails/erith.codelibre.net/root 7.60G 807G 4.52G /zvirt/iocage/jails/erith.codelibre.net/root zvirt/iocage/log 144K 807G 144K /zvirt/iocage/log zvirt/iocage/releases 1.43G 807G 96K /zvirt/iocage/releases zvirt/iocage/releases/15.0-RELEASE 1.43G 807G 96K /zvirt/iocage/releases/15.0-RELEASE zvirt/iocage/releases/15.0-RELEASE/root 1.43G 807G 1.43G /zvirt/iocage/releases/15.0-RELEASE/root zvirt/iocage/templates 96K 807G 96K /zvirt/iocage/templates The registered runners can be seen here: https://gitlab.com/groups/libtiff/-/runners You can see an example CI pipeline here: https://gitlab.com/libtiff/libtiff/-/pipelines/2218553416 This shows all of the newly added platforms (MacOS, FreeBSD) as well as all of the previous GitLab and AppVeyor builds all consolidated. The AppVeyor build is marked as failed because its configuration has been deleted; if everyone is happy with this approach we could turn the AppVeyor integration off. The benefit of this approach is that we have added platforms which are not supported by the cloud providers, and they run faster and with effectively unlimited time. We could potentially also expand the CI testing to test libtiff against a large corpus of test images; there is ample storage for a lot of image data. This would be useful to test a lot of different corner-cases and non-standard TIFFs which need their quirks handling. The downside is that I have to maintain six systems (hosts and virtual combined), and also keep them up-to-date and running 24/7, and they are on my home ADSL. So the availability might not be as good. We’ll need to see how that works out. The other downside is that these systems are libtiff group runners; they are only accessible to members of projects within the libtiff GitLab group, so external or internal contributors who open merge requests from forked repositories won’t have the CI builds run successfully. Only branches pushed to the libtiff repository will run. It would be good if we could get libtiff approved by GitLab as an open source project, then the limit on the group membership number would be lifted and it would be possible for more contributors to have access. That’s essentially it. Any comments or feedback appreciated. And if this doesn’t work out, it’s a quick task to revert things back to how they were. Kind regards, Roger _______________________________________________ Tiff mailing list [email protected] https://lists.osgeo.org/mailman/listinfo/tiff
