FreeBSD Status Report Fourth Quarter 2023 Here is the fourth 2023 status report, with 18 entries.
This is the last 2023 quarter. As you have probably noticed, this status report comes later than usual and with fewer reports than the preceding quarter. Indeed, please keep in mind that the last quarter of every year is for many members of our community the quarter of the celebrations for Christmas and for the New Year, which implies that those members will spend more time with their families and will have less time for their favorite voluntary software projects. Thus there is less to report and reports tend to arrive later. But finally, here they are. Have a nice read. Lorenzo Salvadore, on behalf of the Status Team. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ A rendered version of this report is available here: https://www.freebsd.org/status/report-2023-10-2023-12/ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Table of Contents • FreeBSD Team Reports □ FreeBSD Core Team □ FreeBSD Foundation □ FreeBSD Release Engineering Team □ Cluster Administration Team □ Continuous Integration □ Ports Collection □ Bugmeister Team and Bugzilla • Userland □ Service jails — Automatic jailing of rc.d services • Kernel □ Packrat - NFS client caching on non-volatile storage • Architectures □ armv7 Ports Quality Assurance □ SIMD enhancements for amd64 • Cloud □ OpenStack on FreeBSD □ FreeBSD on Microsoft HyperV and Azure □ FreeBSD on EC2 • Documentation □ Documentation Engineering Team □ FreeBSD Online Editor and Man Page Editor □ FreeBSD Wiki • Ports □ KDE on FreeBSD □ State of GNOME 44 □ GCC on FreeBSD • Third Party Projects □ Containers and FreeBSD: Pot, Potluck and Potman ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ FreeBSD Team Reports Entries from the various official and semi-official teams, as found in the Administration Page. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ FreeBSD Core Team Contact: FreeBSD Core Team <c...@freebsd.org> The FreeBSD Core Team is the governing body of FreeBSD. Along the release engineering team, the project dedicates the 14.0-RELEASE to the memory of Hans Petter Selasky. 14.0-RELEASE FreeBSD 14.0 was released at the end of 2023Q4. The release notes can be found at https://www.freebsd.org/releases/14.0R/relnotes/ New Release Engineering Team After years of serving as the release engineer gjb@ stepped down. cperciva@ took over as the new release engineer. karels@ is serving as the new deputy release engineer. Core would like to thank gjb@ for his long tenure and the many timely releases he created. FreeBSD 2024 Community Survey In the end of 2023, Core Team works with the Foundation to do the 2024 community survey. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ FreeBSD Foundation Links: FreeBSD Foundation URL: https://freebsdfoundation.org/ Technology Roadmap URL: https://freebsdfoundation.org/blog/technology-roadmap/ Donate URL: https://freebsdfoundation.org/donate/ Foundation Partnership Program URL: https://freebsdfoundation.org/our-donors/ freebsd-foundation-partnership-program/ FreeBSD Journal URL: https://freebsdfoundation.org/journal/ Foundation Events URL: https://freebsdfoundation.org/our-work/events/ Contact: Deb Goodkin <d...@freebsdfoundation.org> The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the FreeBSD Project and worldwide community, and helping to advance the state of FreeBSD. We do this in both technical and non-technical ways. We are 100% supported by donations from individuals and corporations and those investments help us fund the: • Software development projects to implement features and functionality in FreeBSD • Sponsor and organize conferences and developer summits to provide collaborative opportunities and promote FreeBSD • Purchase and support of hardware to improve and maintain FreeBSD infrastructure, • Resources to improve security, quality assurance, and continuous integration efforts. • Materials and staff needed to promote, educate, and advocate for FreeBSD, • Collaboration between commercial vendors and FreeBSD developers, • Representation of the FreeBSD Project in executing contracts, license agreements, and other legal arrangements that require a recognized legal entity. We supported FreeBSD in the following ways during the last quarter of 2023: OS Improvements During the fourth quarter of 2023, 236 src, 47 ports, and 33 doc tree commits identified The FreeBSD Foundation as a sponsor. Some of this Foundation-sponsored work is described in separate report entries: • OpenStack on FreeBSD • SIMD enhancements for amd64. Three new contractors started. Cheng Cui began working full-time on wireless networking. A main goal for Cheng’s project is to assist Bjoern Zeeb with 802.11ac support in iwlwifi. Tom Jones began work to port the Vector Packet Processor (VPP) to FreeBSD. VPP is an open-source, high-performance user space networking stack that provides fast packet processing suitable for software-defined networking and network function virtualization applications. Olivier Certner joined the FreeBSD Foundation as a general FreeBSD developer. Some of Olivier’s contributions so far include: • reviewing, fixing, and hardening several security policies aimed at limiting process visibility, policies that are based on user identity, group membership, or sub-jail membership • committing fixes in the login class code, including one that allowed unprivileged users to bypass resource limits • implementing a secure hardware fix for the Zenbleed issue affecting AMD Zen2 processors. Here is a sampling of other Foundation-sponsored work completed over the last quarter of 2023: • arm64: Add Armv8 rndr random number provider • net80211, LinuxKPI, and iwlwifi fixes and improvements • OpenSSL: updates to 3.0.11 and 3.0.12 • Various freebsd-update fixes in preparation for 14.0 • ssh: Update to OpenSSH 9.5p1 • Various iommu fixes • Various makefs/zfs fixes Learn more about our software development work for all of 2023 at https://freebsdfoundation.org/blog/2023-in-review-software-development/. FreeBSD Infrastructure We approved over $100,000 for a cluster refresh that began in late 2023 and will carry over into the new year by purchasing and shipping 15 new servers to 4 racks generously donated by NYI in their new Chicago facility. The systems specifications were determined by the Cluster Administration team and consist of: • 5 package builders • 3 web servers • 2 package mirrors • 2 CI servers • 2 firewall/router • 1 admin bastion More on our 2023 infrastructure support can be found at: https://freebsdfoundation.org/blog/2023-in-review-infrastructure/. Continuous Integration and Workflow Improvement As part of our continued support of the FreeBSD Project, the Foundation supports a full-time staff member dedicated to improving the Project’s continuous integration system and the test infrastructure. The full update can be found within the quarterly status report. Partnerships and Research In Q4 I connected with the following people, companies, and organizations: Phil Shafer, who works at Juniper Networks, and I met at All Things Open. He told me about the libxo library and his continuing work on related issues, like rewriting and filtering output to allow richer options that regular expressions provide. Sticking with Juniper, I also met Simon Gerraty at the Vendor Summit and heard his talk on SecureBoot. In alphabetical order, I also met with AMD, Ampere, Center for Internet Security (CIS), Innovate UK, Michael Dexter, Metify, Microsoft, several people at NetApp when I attended their annual conference (Thank you for the invitation!!), NetScaler, NIST, Nozomi Networks, NVIDIA, members of the Open Container Initiative community, OpenSSF, RG Nets, Doug Rabson. I greatly appreciated the opportunity to attend NetApp’s annual conference in October. I heard from and connected with experts at NetApp and their partners and customers on topics such as AI and seamless AI data pipelines, hybrid cloud, and green computing. I took the opportunity to hand out some FreeBSD lapel pins 🙂 and I connected with a FreeBSD user and member of the Enterprise WG whose company is a NetApp Customer. In Q4 we announced the new FreeBSD SSDF Attestation program to help commercial users of FreeBSD comply with new US Government procurement regulations. This program was informed by valuable feedback from NetApp, Metify, and NIST, and the genesis of the idea came thanks to my involvement with open source policy experts, in particular via the OSI’s Open Policy Alliance. The Open Container Initiative Technical Oversight Board voted in December to approve Doug Rabson’s proposal to create a Working Group to extend the OCI runtime specification to support FreeBSD. Huge thanks to all involved! An OCI runtime extension for FreeBSD is one of the most frequently requested capabilities and I was happy to play a small role in helping to coordinate this effort so far. The Vendor Summit in November was a great event. Huge props to John Baldwin and Anne Dickison for all the work to organize and orchestrate. I got a lot out of the event. Personal highlights were conversations with a diversity of users, the CHERI talk, the end user panel, and Allan’s talk on being an upstream first company. For a full recap on our efforts to strengthen partnerships and increase funding in 2023, check out: https://freebsdfoundation.org/blog/2023-in-review-partnerships-and-research/. Advocacy >From organizing and attending events, to creating technical content that educates, and expanding the coverage of FreeBSD in the media, here is a sample of what we did last quarter to support FreeBSD. • Helped organize and sponsor the November 2023 Vendor Summit held at NetApp in San Jose. Many consider this one of the best summits to date. Be sure to check out the videos. • Introduced FreeBSD to new and returning folks at All Things Open in North Carolina. • Provided an overview of FreeBSD 14: Security, Performance, and Interoperability; Introducing FreeBSD 14 • In collaboration with the Core team, released the 2024 FreeBSD Community Survey • Participated in an interview about FreeBSD: What the Dev Podcast: The Evolution of the FreeBSD Project • Release the September/October 2023 issue of the FreeBSD Journal now with HTML versions of the articles. For a full recap of what we did to advocate for FreeBSD in 2023, please check out the Advocacy Year in Review: https://freebsdfoundation.org/blog/2023-in-review-advocacy/ or the monthly newsletters: https://freebsdfoundation.org/our-work/latest-updates/?filter=newsletter. Fundraising Thank you to everyone who gave us a financial contribution last quarter to help fund our work to support the Project. You brought us even closer to our goal and we are grateful for your investment in FreeBSD! We are still receiving donations in the mail and will post the final number in mid-February. Please consider supporting our efforts in 2024 by making a donation here: https://freebsdfoundation.org/donate/. Or, check out our Partnership opportunities here: https://freebsdfoundation.org/our-donors/freebsd-foundation-partnership-program/. Legal/FreeBSD IP The Foundation owns the FreeBSD trademarks, and it is our responsibility to protect them. We also provide legal support for the core team to investigate questions that arise. Go to https://freebsdfoundation.org to find more about how we support FreeBSD and how we can help you! ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ FreeBSD Release Engineering Team Links: FreeBSD 13.3-RELEASE schedule URL: https://www.freebsd.org/releases/13.3R/schedule/ FreeBSD releases URL: https://download.freebsd.org/releases/ISO-IMAGES/ FreeBSD development snapshots URL: https://download.freebsd.org/snapshots/ISO-IMAGES/ Contact: FreeBSD Release Engineering Team, <r...@freebsd.org> The FreeBSD Release Engineering Team is responsible for setting and publishing release schedules for official project releases of FreeBSD, announcing code freezes and maintaining the respective branches, among other things. During the fourth quarter of the year, the Team continued work on 14.0-RELEASE, leading to the final RELEASE build and announcement in November. Planning has started for the upcoming 13.3-RELEASE and 14.1-RELEASE cycles. The Release Engineering Team continued providing weekly development snapshot builds for the main and stable/13 branches, and (after 14.0-RELEASE) started weekly builds for stable/14. After over a decade as Release Engineering Lead, Glen Barber has retired from the role; his Deputy, Colin Percival, has moved into the Lead role, while Mike Karels has assumed the position of Deputy Release Engineer. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Cluster Administration Team Links: Cluster Administration Team members URL: https://www.freebsd.org/administration/#t-clusteradm Contact: Cluster Administration Team <cluster...@freebsd.org> FreeBSD Cluster Administration Team members are responsible for managing the machines the Project relies on to synchronize its distributed work and communications. In this quarter, the team has worked on the following: • Regular support for FreeBSD.org user accounts. • Regular disk and parts support (and replacement) for all physical hosts and mirrors. • Enable mirroring of https://www.FreeBSD.org and https://docs.FreeBSD.org in the FreeBSD project-managed mirrors. • Cluster refresh, upgrading all hosts and jails to the most recent versions of 15-CURRENT, 14-STABLE, 13-STABLE, and 12-STABLE. • Begin sunsetting 12-STABLE infrastructure as the branch approaches its end of life. In addition to these projects, with Modirum generously sponsoring Philip’s time for most of October, we were able to bring pkgbase into "preview" production in time for 14.0-RELEASE in November. We also installed a new European mirror site in Sjöbo, Sweden, sponsored by Teleservice Skåne AB. Traffic in Europe is now directed roughly equally between our existing mirror in Frankfurt (sponsored by Equinix) and the new mirror in Sweden. After well over ten years in service, we plan to decommission our mirror site in the UK during first quarter of 2024. We would like to thank Bytemark Hosting for supporting this mirror for all this time. Next quarter, supported by the FreeBSD Foundation, we plan to bring up a new primary cluster site in Chicago. FreeBSD Official Mirrors Overview Current locations are Australia, Brazil, Germany, Japan (two full mirror sites), Malaysia, South Africa, Sweden, Taiwan, United Kingdom (full mirror site), United States of America — California, New Jersey (primary site), and Washington. The hardware and network connection have been generously provided by: • Bytemark Hosting (decommissioned during 2024Q1) • Cloud and SDN Laboratory at BroadBand Tower, Inc • Department of Computer Science, National Yang Ming Chiao Tung University • Equinix • Internet Association of Australia • Internet Systems Consortium • INX-ZA • KDDI Web Communications Inc • Malaysian Research & Education Network • Metapeer • NIC.br • Your.Org • 365 Data Centers • Teleservice Skåne AB (new since 2023Q4) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Continuous Integration Links: FreeBSD Jenkins Instance URL: https://ci.FreeBSD.org FreeBSD CI Tinderbox view URL: https://https://tinderbox.freebsd.org FreeBSD CI artifact archive URL: https://artifact.ci.FreeBSD.org Hosted CI wiki URL: https://wiki.FreeBSD.org/HostedCI 3rd Party Software CI URL: https://wiki.FreeBSD.org/3rdPartySoftwareCI Tickets related to freebsd-testing@ URL: https://bugs.freebsd.org/bugzilla/buglist.cgi?bug_status=open&email1=testing%40FreeBSD.org&emailassigned_to1=1&emailcc1=1&emailtype1=equals FreeBSD CI Repository URL: https://github.com/freebsd/freebsd-ci dev-ci Mailing List URL: https://lists.FreeBSD.org/subscription/dev-ci Contact: Jenkins Admin <jenkins-ad...@freebsd.org> Contact: Li-Wen Hsu <lw...@freebsd.org> Contact: freebsd-testing Mailing List Contact: IRC #freebsd-ci channel on EFNet In the fourth quarter of 2023, we worked with the project contributors and developers to address their testing requirements. Concurrently, we collaborated with external projects and companies to enhance their products by testing more on FreeBSD. Important completed tasks: • Adding job to build amd64 architecture with GCC 13. (Thanks jhb@) • Adding powerpc64le jobs config for stable-14 (Thanks alfredo@) • Updating the build env of jobs of main and stable/14 branches to 14.0-RELEASE Work in progress tasks: • Designing and implementing pre-commit CI building and testing and pull/ merged-request based system (to support the workflow working group) • Proof of concept system is in progress. • Designing and implementing use of CI cluster to build release artifacts as release engineering does, starting with snapshot builds • Simplifying CI/test environment setting up for contributors and developers • Setting up the CI stage environment and putting the experimental jobs on it • Redesigning the hardware test lab and adding more hardware for testing • Merge https://reviews.freebsd.org/D38815 • Merge https://reviews.freebsd.org/D36257 Open or queued tasks: • Collecting and sorting CI tasks and ideas • Setting up public network access for the VM guest running tests • Implementing use of bare-metal hardware to run test suites • Adding drm ports building tests against -CURRENT • Planning to run ztest tests • Helping more software get FreeBSD support in its CI pipeline (Wiki pages: 3rdPartySoftwareCI, HostedCI) • Working with hosted CI providers to have better FreeBSD support Please see freebsd-testing@ related tickets for more WIP information, and do not hesitate to join the effort! Sponsor: The FreeBSD Foundation ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Ports Collection Links: About FreeBSD Ports URL: https://www.FreeBSD.org/ports/ Contributing to Ports URL: https://docs.freebsd.org/en/articles/contributing/#ports-contributing + Ports Management Team URL: https://www.freebsd.org/portmgr/ Ports Tarball URL: http://ftp.freebsd.org/pub/FreeBSD/ports/ports/ Contact: Tobias C. Berner <portmgr-secret...@freebsd.org> Contact: FreeBSD Ports Management Team <port...@freebsd.org> The Ports Management Team is responsible for overseeing the overall direction of the Ports Tree, building packages, and personnel matters. Below is what happened in the last quarter. • According to INDEX, there are currently 31,942 ports in the Ports Collection. There are currently ~3,100 open ports PRs. The last quarter saw 9,424 commits by 157 committers on the main branch and 781 commits by 71 committers on the 2023Q4 branch. Compared to last quarter, this means a hefty decrease in the number of commits on the main branch (down from 11,454) and slightly fewer backports to the quarterly branch (down from 828). The number of ports also fell a bit (down from 34,600). In Q4 there were around 9424 commits to main. The most active committers where: sunpoet 2946 yuri 861 bofh 793 jbeich 419 fuz 324 eduardo 168 fernape 160 jhale 153 thierry 146 diizzy 123 During Q4 we welcomed Michael Osipov (michaelo) and Timothy Beyer (beyert) as new committers, but sadly also had to say goodbye to bland, sbruno, hselasky and gjb. We invited arrowd, flo and riggs to be part of portmgr-lurkers for the next months. Support for FreeBSD 12.x was removed at the end of the quarter. The end of Q4 also saw the introduction of subpackages to the ports tree. Similar to when flavors were introduced, new subpackages will require an approval by portmgr before being pushed to the tree. With subpackages it is possible to create multiple packages from a single build of a port. The following happened on the infrastructure side: * Packages for 14.0-RELEASE were built * Poudriere was updated to release-3.4 • Support for FreeBSD 12.x was removed. • The no-longer maintained www/qt5-webkit was removed. • postgresql11, php80, mysql57, percona57, ghostscript9 were removed. • The following default versions changed: • perl to 5.36 • ghostcript to 10 • corosync to 3 • Updates to major ports that happened were: • ports-mgmt/pkg to 1.20.9 • ports-mgmt/poudriere to 3.4.0 (subpackage support) • KDE-bits to plasma-5.27.10, frameworks-5.112, gear-23.08.4, and beta-2 • www/chromium to 120.0.6099.129 • www/firefox to 121.0 (rc1) • lang/rust to 1.74.1 • … and many more … During the last quarter, pkgmgr@ ran 26 exp-runs to test various ports upgrades, updates to default versions of ports, subpackage support and base system changes. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Bugmeister Team and Bugzilla Links: Bugmeister team URL: https://www.freebsd.org/administration/#t-bugmeister FreeBSD Bugzilla URL: https://bugs.freebsd.org/bugzilla/ Contact: Bugmeister <bugmeis...@freebsd.org> Some recent maintenance has been done on our Bugzilla instance: • the weekly reminder emails now include the correct values for mfc-* Flags queries; • the Dashboard page has had an obsolete query removed. (We no longer use the 'patch-ready' Keyword; it was too much paperwork. Thus, the query on that field was useless.); • the limit that capped the maximum number of reported PRs at 10000 has been raised to 12500. In addition, the Wiki documentation on our Bugzilla has been updated: • the page https://wiki.freebsd.org/Bugzilla/SearchQueries has been substantially reworked: □ In particular, documentation about how to search on Flag values has been added. (This may not have been done before.) Example: search for PRs with Flag 'mfc-stable14' set; □ This page may be of interest to all committers and contributors; • the page https://wiki.freebsd.org/Bugmeister/BugmeisterQA has also been updated; While similar to the above, it is of more specific interest to bugmeister and triagers. As well, PRs that are specific to FreeBSD 12 are being culled, as 12 has gone out of support as of 20231231. A further effort is being made to document our setup of Bugzilla itself, especially with respect to our customizations. This is needed to bring our own repository up to date with what is running on production. The number of PRs over the past quarter (and year) has remained consistent. However, we do seem to be closing incoming PRs more quickly these days. For reference: https://bugs.freebsd.org/bugzilla/page.cgi?id=dashboard.html&days=90 . The overall number of PRs remains around 11,400. Bugmeister is also working towards restarting the Bugathons. See the updated page https://wiki.freebsd.org/Bugathons. Bugmeister would like to thank a number of people who have assisted with bugbusting, including Mina Galić, Graham Perrin, Lorenzo Salvadore, and Fernando Apesteguìa, among others. In addition, bugmeister would like to thank all the FreeBSD committers who help process the PRs as they come in. Over the last few months we seem to be much closer to steady-state. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Userland Changes affecting the base system and programs in it. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Service jails — Automatic jailing of rc.d services Links: D40370: Infrastructure for automatic jailing of rc.d-services URL: https://reviews.freebsd.org/D40370 D40371: automatic service jails: some setup for full functionality of the services in automatic service jails URL: https://reviews.freebsd.org/D40371 D42779: Handbook / rc-article update for Service Jails URL: https://reviews.freebsd.org/D42779 Contact: Alexander Leidinger <netch...@freebsd.org> Service jails extend the rc(8) system to allow automatic jailing of rc.d services. A service jail inherits the filesystem of the parent host or jail, but uses all other limits of the jail (process visibility, restricted network access, filesystem mounting permissions, sysvipc, …) by default. Additional configuration allows inheritance of the IPs of the parent, sysvipc, memory page locking, and use of the bhyve virtual machine monitor (vmm(4)). If you want to put e.g. local_unbound into a service jail and allow IPv4 and IPv6 access, simply change rc.conf(5) to have: local_unbound_svcj_options=net_basic local_unbound_svcj=YES Note: all base system services are covered in the patches with either name_svcj_options or a hard-coded disabling of the service jails feature where it does not make sense (e.g. pure services which change the runtime configuration but do not start daemons, or where things are run which can not be run in a sensible way inside a jail). As such the local_unbound_svcj_options line above is superfluous and serves just as an example about the amount of configuration needed in total. While this does not have the same security benefits as a manual jail setup with a separate filesystem and IP/VNET, it is much easier to set up, while providing some of the security benefits of a jail like hiding other processes of the same user. Since the previous service jails status report, the following were added: • support for NFS inside jails in the service jails framework (untested), • the possibility of jailing other service commands than start and stop, • service jails options / config for all base system services in the patch in D40371, • a first step at documenting the service jails in the Handbook. Not all services are tested, but all services are covered with a config. Any testing and feedback (even as simple as "service X works in a service jail") is welcome. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Kernel Updates to kernel subsystems/features, driver support, filesystems, and more. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Packrat - NFS client caching on non-volatile storage Contact: Rick Macklem <rmack...@freebsd.org> NFSv4.1/4.2 provides support for a feature called delegations. When a NFSv4.1/ 4.2 client holds a delegation, the client has certain rights to a file, including a guarantee that no other client will make changes to the file unless the delegation is recalled. As such, when a client holds a delegation for a file, it can aggressively cache the file’s data, knowing that it will not be modified by other clients until it returns the delegation. This project is intended to allow the NFSv4.1/4.2 client to aggressively cache file data on client local non-volatile storage, when the client holds a delegation for the file. I created a patch long ago to try and do this for NFSv4.0, but it was never at a stage where it was worth using. This project is a complete rewrite of the patch, done in part because NFSv4.1/4.2 plus other recent NFSv4-related changes make doing this more feasible. I now have code running fairly well and hope to have a patch ready for others to test this winter. Early testing shows promise. For a test run of "make buildkernel", the test with and without packrat enabled performed as follows: Table 1. NFS operation counts NFS operation counts Getattr Lookup Read Write Total RPCs with packrats 433506 99254 0 0 371736 without packrats 2359913 97954 10748 0 2318810 Table 2. Elapsed Run Time Elapsed Run Time (sec) with packrat without packrat 5561 6203 As you can see, the packrat case ran a little faster and with fewer RPCs. Although this test was run on my little LAN, it is hoped that a NFSv4.1/4.2 mount over a WAN would show a larger difference in performance. I will note that the packrat cache was primed by unrolling a tarball of FreeBSD’s /usr/src into the NFSv4.1/4.2 mount. This will be very much an experimental feature, but it is hoped it will allow NFS mounts to be used more effectively, particularly in WAN situations, such as a mobile laptop. There is still work to be done, particularly with respect to recovery of delegations after a NFSv4.1/4.2 client restart. Hopefully, the next status report will include a URL that allows downloading of a patch for user testing. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Architectures Updating platform-specific features and bringing in support for new hardware platforms. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ armv7 Ports Quality Assurance Contact: Robert Clausecker <f...@freebsd.org> As part of a long term project to improve the quality of the FreeBSD ports collection for the armv7 architecture, a number of issues in the base system and in various ports have been fixed. Through this action, the number of binary packages that could be successfully built from the 2023Q4 branch of the ports collection was increased from 30018 (as of 2023-10-04) to 31118 (as of 2023-11-24). Two kernel bugs affecting package builds (PR 267788 and PR 274705) were identified and addressed, with these two alone being responsible for around 900 failed packages. The most common other causes for build failures include • lack of FreeBSD-specific armv7 support code • data alignment issues (armv7 being one of the few architectures for which we do not support unaligned memory accesses) • address space exhaustion during the build processes (usually LTO related; PR 274705 addressed many cases) • lack of OpenMP support on armv7 FreeBSD If you are a user of the FreeBSD ports collection on armv7, do not hesitate to file a bug report on our bug tracker should there be any issues. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SIMD enhancements for amd64 Links: FreeBSD Foundation blog post URL: https://freebsdfoundation.org/blog/a-sneak-peek-simd-enhanced-string-functions-for-amd64/ simd(7) URL: https://man.freebsd.org/cgi/man.cgi?query=simd&sektion=7&manpath=FreeBSD+15.0-CURRENT Work currently under acceptance testing URL: https://github.com/clausecker/freebsd-src/commits/acceptance-testing Contact: Robert Clausecker <f...@freebsd.org> The project to enhance the libc with SIMD implementations of string functions for amd64 has now concluded. In total, SIMD implementations for 17 libc functions have been written, complemented by scalar implementations where needed. Through this rewrite, performance of these functions on strings with an average length of 64 characters was improved by an average factor of 5.54. In addition, 9 other library functions were rewritten to call into the SIMD-enhanced routines, conveying benefits without requiring additional assembly implementations. Please see the FreeBSD Foundation blog post linked above for more details. Parts of the SIMD work are already found in the CURRENT branch. The rest is currently undergoing acceptance testing and will be merged if no problems emerge. It is planned to back port all improvements to 14-STABLE for inclusion into FreeBSD 14.1. Sponsor: The FreeBSD Foundation ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Cloud Updating cloud-specific features and bringing in support for new cloud platforms. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ OpenStack on FreeBSD Links: OpenStack URL: https://www.openstack.org/ OpenStack on FreeBSD URL: https://github.com/openstack-on-freebsd Contact: Chih-Hsin Chang <starb...@hey.com> Contact: Li-Wen Hsu <lw...@freebsd.org> In the fourth quarter, we successfully migrated the originally virtualized OpenStack platform to physical machines running FreeBSD 14.0-STABLE. The ported OpenStack components include Keystone, Glance, Placement, Neutron, and Nova. As part of this process, we took the opportunity to update the installation documentation and the list of dependencies. Moving forward, we encourage users and developers interested in this project to effortlessly recreate the OpenStack platform in their FreeBSD environments following this documentation. Any issues or difficulties encountered are welcome to be reported on the GitHub project page. Your contributions will contribute to the refinement of our installation documentation and the overall porting efforts. In the upcoming quarter, our focus will shift towards incorporating various patches and workarounds generated during the migration process into the project in a more structured code form. Additionally, we plan to develop FreeBSD ports for each OpenStack component, further streamlining the installation process. Sponsor: The FreeBSD Foundation ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ FreeBSD on Microsoft HyperV and Azure Links: Microsoft Azure article on FreeBSD wiki URL: https://wiki.freebsd.org/ MicrosoftAzure Microsoft HyperV article on FreeBSD wiki URL: https://wiki.freebsd.org/HyperV Contact: Microsoft FreeBSD Integration Services Team <bs...@microsoft.com> Contact: freebsd-cloud Mailing List Contact: The FreeBSD Azure Release Engineering Team <releng-az...@freebsd.org> Contact: Wei Hu <w...@freebsd.org> Contact: Souradeep Chakrabarti <schakraba...@microsoft.com> Contact: Li-Wen Hsu <lw...@freebsd.org> In this quarter, we have solved all the blocking issues and published the 14.0-RELEASE on Azure Marketplace, with complete architecture (amd64, arm64) and VM generation (gen1, gen2) support, available in both UFS and ZFS as the root file system. Work in progress tasks: • Automating the image building and publishing process and merging to src/ release/. • Building and publishing snapshot builds to Azure community gallery. The above tasks are sponsored by The FreeBSD Foundation, with resources provided by Microsoft. Open tasks: • Update FreeBSD related doc at Microsoft Learn • Support FreeBSD in Azure Pipelines • Update Azure agent port to the latest version • Upstream local modifications of Azure agent • Port Linux Virtual Machine Extensions for Azure Sponsor: Microsoft for people in Microsoft, and for resources for the rest Sponsor: The FreeBSD Foundation for everything else ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ FreeBSD on EC2 Links: FreeBSD/EC2 Patreon URL: https://www.patreon.com/cperciva Contact: Colin Percival <cperc...@freebsd.org> FreeBSD is available on both amd64 (Intel and AMD) and arm64 (Graviton) EC2 instances. Work continues to ensure that upcoming instance types will be supported; most recently, changes were needed to support "7th generation" Intel and AMD instances. FreeBSD 14.0-RELEASE shipped with experimental ZFS-root AMIs and "cloud-init" AMIs. Additional "flavored" FreeBSD AMIs are planned, including "AMI Builder" and "minimal" (no debug symbols). A bug in the release-building process which resulted in 14.0-RELEASE AMIs shipping with duplicate lines in /etc/rc.conf has been corrected and future releases should not be affected. A bug in the ec2-aws-imdsv2-get utility which resulted in 14.0-RELEASE AMIs not supporting binary user-data files has been corrected and future releases should not be affected. This work is supported by Colin’s FreeBSD/EC2 Patreon. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Documentation Noteworthy changes in the documentation tree, manual pages, or new external books/documents. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Documentation Engineering Team Link: FreeBSD Documentation Project URL: https://www.freebsd.org/docproj/ Link: FreeBSD Documentation Project Primer for New Contributors URL: https://docs.freebsd.org/en/books/fdp-primer/ Link: Documentation Engineering Team URL: https://www.freebsd.org/administration/#t-doceng Contact: FreeBSD Doceng Team <doc...@freebsd.org> The doceng@ team is a body to handle some of the meta-project issues associated with the FreeBSD Documentation Project; for more information, see FreeBSD Doceng Team Charter. During the last quarter: Glen Barber stepped down from doceng. doceng would like to thank gjb@ for his service. Ceri Davies' commit bit was taken for safekeeping as per his request. doceng would like to thank ceri@ for his contributions. mhorne@ to be mentored by carlavilla@ to obtain a documentation commit bit. FreeBSD Handbook: The Handbook was updated to show that FreeBSD 14.0 is the latest release. FreeBSD Translations on Weblate Link: Translate FreeBSD on Weblate URL: https://wiki.freebsd.org/Doc/Translation/Weblate Link: FreeBSD Weblate Instance URL: https://translate-dev.freebsd.org/ Q4 2023 Status • 17 team languages • 203 registered users Languages • Chinese (Simplified) (zh-cn) (progress: 7%) • Chinese (Traditional) (zh-tw) (progress: 3%) • Dutch (nl) (progress: 1%) • French (fr) (progress: 1%) • German (de) (progress: 1%) • Indonesian (id) (progress: 1%) • Italian (it) (progress: 5%) • Korean (ko) (progress: 33%) • Norwegian (nb-no) (progress: 1%) • Persian (fa-ir) (progress: 2%) • Polish (progress: 1%) • Portuguese (progress: 0%) • Portuguese (pt-br) (progress: 22%) • Spanish (es) (progress: 35%) • Turkish (tr) (progress: 2%) We want to thank everyone that contributed, translating or reviewing documents. And please, help promote this effort on your local user group, we always need more volunteers. FreeBSD Handbook working group Contact: Sergio Carlavilla <carlavi...@freebsd.org> • The Network chapter has been rewritten • The Jails chapter has been rewritten • The next section to work on will be the file systems part: UFS, ZFS, Other File Systems FAQ Working Group Contact: Sergio Carlavilla <carlavi...@freebsd.org> A new FAQ was released alongside FreeBSD 14.0. FreeBSD Website Revamp - WebApps working group Contact: Sergio Carlavilla <carlavi...@freebsd.org> Working group in charge of creating the new FreeBSD Documentation Portal and redesigning the FreeBSD main website and its components. FreeBSD developers can follow and join the working group on the FreeBSD Slack channel #wg-www21. The work will be divided into three phases: 1. Redesign of the Manual Pages on web Scripts to generate the HTML pages using mandoc. (Complete, Approved by Doceng, Deploy Date Not Decided Yet) Public instance on https://man-dev.FreeBSD.org 2. Redesign of the FreeBSD main website New design, responsive and dark theme. (Almost Complete, Presented at EuroBSDCon) 3. Redesign of the Ports page on web Ports scripts to create an applications portal. (Work in progress) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ FreeBSD Online Editor and Man Page Editor Links: FreeBSD Online Document Editor URL: https://github.com/Wang-Yan-Hao/FreeBSD-Online-Document-Editor FreeBSD Online Man Page Editor URL: https://github.com/Wang-Yan-Hao/man_page_editor Contact: Yan-Hao Wang <bses30...@gmail.com> Contact: Li-Wen Hsu <lw...@freebsd.org> This report provides a continued overview of the FreeBSD online editor and man page editor project, outlining recent efforts to enhance the documentation and manual page editing processes. In order to optimize the project’s structural integrity, we enlisted the expertise of a professional front-end programmer. We plan to release the editor soon and currently have some tasks that require additional support. 1. We are actively seeking a qualified individual to conduct a comprehensive front-end security review of the project. 2. A meticulous inspection of the JavaScript code is imperative to ensure its robustness and efficiency. We are looking for someone with expertise to thoroughly examine the codebase, identify any issues, and propose enhancements for optimal performance. 3. Since there is currently no existing JavaScript library for rendering mandoc, I had to create my own. However, there are still some hidden errors that emerge during the editing process. We are seeking assistance to fix these rendering issues. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ FreeBSD Wiki Links: Wiki URL: https://wiki.freebsd.org Contact: Wiki administration <wiki-ad...@freebsd.org> Plans are underway to familiarize our audience on Discord with the wiki (there are too many "silos" in our FreeBSD community). Contact Setesh on the FreeBSD Discord for more information. Preliminary work is being done on updating the wiki software itself. Continuing to run MoinMoin requires a jail with a downrev version of Python. The MoinMoin project itself seems to have stalled in the middle of a redesign; at a minimum, a complete upgrade of the backend database would be needed. Alternatives that are under consideration include MediaWiki and DocuWiki; see https://wiki.freebsd.org/Wiki/NextGeneration. Most of the discussion is occurring on Matrix; please contact wiki-ad...@freebsd.org if you would like to participate. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Ports Changes affecting the Ports Collection, whether sweeping changes that touch most of the tree, or individual ports themselves. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ KDE on FreeBSD Links: KDE/FreeBSD initiative URL: https://freebsd.kde.org/ FreeBSD — KDE Community Wiki URL: https://community.kde.org/FreeBSD Contact: Adriaan de Groot <k...@freebsd.org> The KDE on FreeBSD project packages CMake, Qt, and software from the KDE Community, for the FreeBSD ports tree. The software includes a full desktop environment called KDE Plasma (for both X11 and Wayland) and hundreds of applications that can be used on any FreeBSD machine. The KDE team is part of desktop@ and x11@, building the software stack to make FreeBSD beautiful and usable as a daily-driver graphical desktop workstation. The notes below describe mostly ports for KDE, but also include items that are important for the entire desktop stack. Infrastructure CMake was updated several times and is now version 3.28.1, the latest upstream release. FreeBSD ports are once again fully up-to-date. Qt5 is now on long-term support and updates only rarely. The KDE patch collection is a community-supported branch of Qt which pulls in upstream patches and fixes from the KDE community, and updated to 5.15.12. There were several deprecations (see below) in the Qt5 ports. Qt6 and KDE’s upcoming megarelease of KDE Plasma 6 (scheduled for 2024q1) are the next major milestone for the KDE team. Qt6 was updated to version 6.6.1 along with the Python bindings for Qt, PySide. An alpha-release of KDE Frameworks 6 was added to the ports tree. KDE Stack KDE Gear releases happen every quarter, KDE Plasma updates once a month, and KDE Frameworks have a new release every month as well. These (large) updates land shortly after their upstream release and are not listed separately. • KDE Frameworks reached version 5.112. The KDE Frameworks 5 series is winding down, although it will a few months still until it enters long-term support upstream. • KDE Plasma Desktop was updated to version KDE Plasma 5.27.10. • KDE Gear updated to 23.08.4. • KDE Frameworks 6 (alpha) 5.247 was updated in the ports tree. • KDE Plasma Desktop 6 (beta 2) 5.91.0 was updated in the ports tree. Related Ports The KDE ecosystem includes a wide range of ports — most maintained by kde@, all building on a shared base of Qt and KDE Frameworks. The KDE team updates them all as needed. This quarter the KDE team would like to thank Tobias C. Berner, Gleb Popov and Jason E. Hale again for keeping things up-to-date. Many ports have been "flavorized" to support a Qt5 and a Qt6 flavor in the ports tree. Special mention to: • New port x11/xwaylandvideobridge. By design, X11 applications can’t access window or screen contents for Wayland clients. The video bridge improves Wayland support for screen sharing tools like Discord, MS Teams, Skype, and more. Screen sharing is fully under the control of the Wayland user. • Update for multimedia/mlt7 which was updated to 7.20.0. • Update for sysutils/bsdisks which was updated to 0.33. • Bugfix for devel/llvm15 to make devel/kdevelop work again. • Security fixes for www/qt5-webengine and www/qt6-webengine. Deprecations Web browsers are huge, and have a considerable security surface. The venerable www/qt5-webkit WebKit port was removed on the last day of 2023. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ State of GNOME 44 Links: GNOME URL: https://www.gnome.org/ Development repository URL: https://codeberg.org/olivierd/freebsd-ports-gnome Contact: FreeBSD GNOME Team <gn...@freebsd.org> Contact: Olivier Duchateau < duchateau.oliv...@gmail.com> GNOME is a full desktop environment which is mainly based on GLib, GTK3/GTK4, and libadwaita. It provides two window managers or compositors: x11-wm/mutter and x11-wm/metacity. Currently in the ports collection, x11/gnome-shell is not supported by upstream anymore. As it is a lot of work, in order to have GNOME 44 available for users, I decided to split this update, because it impacts several ports. As a maintainer of x11/budgie and Pantheon desktop (a window manager based on x11-wm/mutter, developed for elementary OS) I need more recent versions of some GNOME libraries. Firstly I worked on WebKitGTK. The 4.0 "legacy" API is almost not used by GNOME’s libraries. The bare minimum is the 4.1 API. I created webkit.mk for the Mk/Uses framework, in order to flavorize www/webkit2-gtk3. There is an ongoing effort, but currently it is too unstable. Often applications such as Epiphany, mail clients (Geary, Evolution), or the online accounts panel in package:sysutils/gnome-control-center dump core. Nonetheless, remainder of desktop is usable and the latest release (44.7) of GNOME Shell is functional. I have begun sending my first patches for review (as well as those in Bugzilla). • D43183 • D43230 • D43244 • D40489 I have also ported the GNOME Flashback session module. It depends on x11-wm/ metacity and x11-toolkits/libwnck3. I also maintain a documentation, and we can see various desktops available. GNOME 45 is almost finished, except for GNOME Shell extensions. For this release I will focus on Wayland support (bug #258042 and bug #271836). Tests and patches are welcomed, especially for WebKitGTK. Next months I plan to work on: • Allowing selecting a session in display manager (gdm), it is regression with our patches. • Fixing sharing network (VNC, SSH) panel in gnome-control-center and backport for bug #275900. • Continuing to update applications and libraries for GNOME 45. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ GCC on FreeBSD Links: GCC Project URL: https://gcc.gnu.org/ GCC 10 release series URL: https://gcc.gnu.org/gcc-10/ GCC 11 release series URL: https://gcc.gnu.org/gcc-11/ GCC 12 release series URL: https://gcc.gnu.org/gcc-12/ GCC 13 release series URL: https://gcc.gnu.org/gcc-13/ Contact: Lorenzo Salvadore <salvad...@freebsd.org> Updating GCC default version to 13 is moving ahead. Thanks to Antoine Brodin who ran the exp-runs and to all other developers and ports maintainers involved. As you might remember from last quarter, additional patches were tested together with the default version updates. Some of them have already been merged: • lang/gcc11 has switched back to STANDARD_BOOTSTRAP and has been updated to 11.4.0; • lang/gcc13 has been updated to version 13.2.0. About half of the open bugs have been fixed, but another half remains. If you maintain any of the affected ports, please try to fix your port(s) and/or get your port buildable with the compiler in base. This quarter many bug reports have also been opened about GCC. As soon as the default GCC version update is finished, all of those bugs will be addressed. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Third Party Projects Many projects build upon FreeBSD or incorporate components of FreeBSD into their project. As these projects may be of interest to the broader FreeBSD community, we sometimes include brief updates submitted by these projects in our quarterly report. The FreeBSD project makes no representation as to the accuracy or veracity of any claims in these submissions. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Containers and FreeBSD: Pot, Potluck and Potman Links: Pot organization on GitHub URL: https://github.com/bsdpot Contact: Luca Pizzamiglio (Pot) <pizza...@freebsd.org> Contact: Bretton Vine (Potluck) <b...@honeyguide.eu> Contact: Michael Gmelin (Potman) <gre...@freebsd.org> Pot is a jail management tool that also supports orchestration through Nomad. During this quarter, Pot 0.16.0 was released containing a number of features and fixes, including a new setting to prevent direct traffic between VNET pots and new attributes to configure pot stop behavior. There were also maintenance/ stability releases to potnet (0.5.0) and a nomad-pot-driver (0.10.0). Potluck aims to be to FreeBSD and Pot what Dockerhub is to Linux and Docker: a repository of Pot flavours and complete container images for usage with Pot and in many cases Nomad. One of the new container images that have been added during the last quarter is Zincsearch, a more light-weight alternative to Elasticsearch written in Go. The Mastodon container is meanwhile powering the public mastodon.africa instance. Also, we got some more publicity: BSD Now Episode 536 is titled "Pot-flavored Jails". As always, feedback and patches are welcome. Sponsors: Nikulipe UAB, Honeyguide Group
