[arch-dev-public] Congrats to the Arch Conf Team
A big shout out to the Arch Conf Team! This weekend's conference was nothing short of awesome. It pulled together very well, particularly given the short organisation period and it being the first one run online. The recorded talks followed by live Q&A worked very well, and I enjoyed the live commentary by the community either on IRC or twitch. Looking forward to next year! :) Allan
Re: [arch-dev-public] Use detached package signatures by default
On 9/7/20 1:05 pm, Anatol Pomozov wrote: > Given this information I would like to propose to stop using embedded > signatures and move to detached signatures by default. This will > require pacman 6.x or as alternative backport the fix(es) to 5.x > branch. It will help to make system updates even faster, something > that me and many other Arch users really love. There are several steps we need to complete: 1) backport the patch (or wait for pacman-6.0, which may be a while yet). I'll leave that to the distro packagers to decide! 2) adjust repo-add to optionally add signatures. 3) make a time line that all users need to have the patched/released pacman installed - we usually require at least 6 months. 4) turn off signature inclusion in repo dbs. Allan
[arch-dev-public] Reproducible builds progress report #3
Hi all, A quick updated on the progress of reproducible builds. You may have noticed a couple of large rebuilds that occurred recently. These fixed issues of non-reproducible file ordering with old versions of makepkg. This and other hard work by the team improving our tooling and fixing packaging issues has resulted in 96% of [core] being reproducible, and 90% of [extra]. You can see the status of which packages are reproducible here [1]. The remaining packages to fix in [core] are dnssec-anchors, linux, linux-lts, nss and perl. With the possible exception of perl, these are in the "hard" basket. There is plans on how to fix the kernel packages, but that will require some time to sort out. We would be happy for more people to help out so we can get [core] to 100% reproducible. We have investigated some of the packages in [extra] that fail to reproduce here [2]. Note that there are quite a few packages that currently "Failed to build from source" (FTBFS) - it would be very helpful for the reproducible builds team if their maintainers can help fix the packages. You can also use the CI of Arch packages run by Debian to get an overview what the issue is with these packages and see many other packages that are currently failing to build [3]. We also need help to investigate and fix the packages that fail to reproduce that we have not investigated as of yet. There are two easy to use tools to attempt to reproduce a package - "makerepropkg" from devtools and "repro" from the archlinux-repro package. Once these have rebuilt a package, you can use the "diffoscope" tool to look at the differences. Jump in the #archlinux-reproducible IRC channel if you want help interpreting the output, or you could just link to a copy of it in the wiki. [1] https://reproducible.archlinux.org/ [2] https://wiki.archlinux.org/index.php/Reproducible_Builds/Status [3] https://tests.reproducible-builds.org/archlinux/extra.html
Re: [arch-dev-public] Discussion - Increasing our CPU requirements
On 30/3/20 12:39 am, Filipe Laíns wrote: > On Sun, 2020-03-29 at 23:37 +1000, Allan McRae via arch-dev-public wrote: >> On 29/3/20 11:17 pm, Filipe Laíns wrote: >>> I would also like to note that rebuilding everything with forced >>> support for AVX2 or whatever won't have much effect. Most packages do >>> not have workloads where it would make use sense to use these CPU >>> extensions, and as such, GCC would not use them. >> >> That assumes we just add AVX2. Whereas, requiring a CPU supporting AVX2 >> would bring other optimizations that would be used. > > No, it should be true for all extensions. > >> As I replied earlier, AVX2 may be going too far. But is a good starting >> point for discussion. If that is too far, what could we accept? >> SSE4.2? AVX? Surely we can do better than pure x86_64. > > No, SSE4.2 is too far. For me, the minimum should be AVX. SSE4.2 is 2008 for Intel, 2011 for AMD. Though I guess some processors were released without it for some time after that. AVX was released by both in 2011. So why is one too far and the other not? >> To have a separate architecture would require automated builds, which >> requires being able to sign packages automatically. And we have not >> achieved database signing in 9 years I'm looking for a boost that >> could be achieved now. > > No, it would not. Where is this coming from? I already build split > packages with SIMD instructions, I make the PKGBUILD build for 2 > architectures instead with a minimal patch. > > If pacman is not able to handle parallel architectures, we should fix > that. I think it's a valid use case. No need for pacman support. Just add higher instruction set to a new repo and set that repo with higher priority. But that involves developers choosing which packages to build with higher instruction sets, which requires extra developer time. Ideally, we would just autobuild for more optimized architectures, but this requires auto-signing packages, which has not happened in the last decade (but may in this one...). Picking an instruction set that is ~10 years old and making it the default for the distro seems a reasonable approach to me. A
Re: [arch-dev-public] Discussion - Increasing our CPU requirements
On 29/3/20 11:17 pm, Filipe Laíns wrote: > I would also like to note that rebuilding everything with forced > support for AVX2 or whatever won't have much effect. Most packages do > not have workloads where it would make use sense to use these CPU > extensions, and as such, GCC would not use them. That assumes we just add AVX2. Whereas, requiring a CPU supporting AVX2 would bring other optimizations that would be used. As I replied earlier, AVX2 may be going too far. But is a good starting point for discussion. If that is too far, what could we accept? SSE4.2? AVX? Surely we can do better than pure x86_64. To have a separate architecture would require automated builds, which requires being able to sign packages automatically. And we have not achieved database signing in 9 years I'm looking for a boost that could be achieved now. Allan
Re: [arch-dev-public] Discussion - Increasing our CPU requirements
On 29/3/20 8:52 pm, Evangelos Foutras wrote: > If I see a SIGILL on my AMD Phenom II X6 1090T then Arch will have failed me. > 😜 > > I believe your proposal should only be discussed as co-existing > optimized port(s) and even then I'm not sure it's worth the trouble. > Performance-critical applications can and frequently are optimized for > the running processor (I'm thinking of stuff like glibc and ffmpeg > here). AVX2 was a bold choice, and really a place to get a discussion started. We are currently supporting processors from 2003. We can be better than that. A
Re: [arch-dev-public] Discussion - Increasing our CPU requirements
On 29/3/20 9:27 pm, Amish wrote: > Also if I am not wrong Arch philosophy talks only about latest software > and no where there is mention of latest hardware being a compulsion. It used to. One of the original selling points was i686 optimization. Then we got lazy, and stopped innovating. A
[arch-dev-public] Discussion - Increasing our CPU requirements
Remember when Arch Linux was optimized out of the box. We have the blazingly fast i686 port while other distros hung out in i386 land. Those were the days where the idea of Arch being fast started. Now it has degraded to stuff of legend. Now, x86_64 is old. We should continue to push forward and add further optimization. Reasonable optimizations to consider: AVX2 FMA SSE4.2 AVX2 is Intel Haswell and newer or AMD Ryzen and newer. This CPUs released 2013 to 2015. So 5 - 7 years old. Discuss.
Re: [arch-dev-public] Restricting ability to post news items
On 6/1/20 9:12 am, Giancarlo Razzolini wrote: > Em janeiro 5, 2020 19:25 Allan McRae via arch-dev-public escreveu: >> >> Read the original message and not the partial quote that you made. I >> explicitly said there was an exception for --overwrite type posts. >> >> But any restriction being made on posting due to not posting drafts to >> the list would be complete. >> > > Hi Allan, > > I think you're overreacting (again) for something that's not properly > coded anywhere. > And, the last time this was discussed, I have talked about cases of news > that couldn't > be brought on a-d-p for discussion. We have other places to discuss > those (staff@), but > still, until we have *clear* guidelines about this, let's not rush to > implement anything. My apologies. I believed this had been covered on the mailing list and I am told it was only on IRC, and never passed on to the TU channel, which I will accept as an excuse despite everyone(?) involved but the poster being on both channels... > I have proposed a news entry regarding zstd, Robin did the statistics > and we've discussed this > for 2 days on #archlinux-tu. If you're looking for somebody to blame, > look no further. Take away > my news posting rights (you'll have to also take away my archweb admin > rights in this case). > > And let's properly codify this ok? Because it's not codified anywhere. I > could've asked Robin > to send an email to a-d-p, but most of the work we've done on the draft > was on a pad. This was > reviewed by quite a few people. I asked Robin to do it, because since > they pushed zstd, they > should've also be the ones doing the news entry. But I would do it > myself this week, if they > didn't. Do we really need to write down everything? Have we reached a point in the distro where common sense has stopped? Why would an announcement that affects the whole distro not be run past all team members by default? Allan
Re: [arch-dev-public] Restricting ability to post news items
On 6/1/20 8:04 am, Morten Linderud via arch-dev-public wrote: > On Mon, Jan 06, 2020 at 07:53:21AM +1000, Allan McRae via arch-dev-public > wrote: >> Following the roll out of the base metapackage, and its poorly written >> news post, we agreed that all new posts should have a draft posted to >> arch-dev-public. > > Wait, where was this agreed? I heard something about all drafts should be > headed > for arch-dev, but this hasn't been announced nor discussed anywhere. > > Are the TUs missing from the loop here? > After the base metapackage pile of crap that was posted, I (and others) reminded everyone of the process that had been held for years. The response was complaints that this process was not followed recently, to which we pointed at the arch-dev-public mailing list post for multiple non-trivial news entries. Only simple "--overwrite" type posts have not been posted on arch-dev-public. This is not new - it has been the standard for many, many years - spanning from before I was around the distro. Allan
Re: [arch-dev-public] Restricting ability to post news items
On 6/1/20 8:17 am, Morten Linderud via arch-dev-public wrote: > On Sun, Jan 05, 2020 at 11:10:17PM +0100, Bartłomiej Piotrowski via > arch-dev-public wrote: >> On 05/01/2020 23.04, Morten Linderud via arch-dev-public wrote: >>> On Mon, Jan 06, 2020 at 07:53:21AM +1000, Allan McRae via arch-dev-public >>> wrote: >>>> Following the roll out of the base metapackage, and its poorly written >>>> news post, we agreed that all new posts should have a draft posted to >>>> arch-dev-public. >>> >>> Wait, where was this agreed? I heard something about all drafts should be >>> headed >>> for arch-dev, but this hasn't been announced nor discussed anywhere. >>> >>> Are the TUs missing from the loop here? >>> >> >> If you look at the non-trivial news items, you can easily correlate them >> with drafts posted to arch-dev-public. > > You are writing about non-trivial news items, but Allan is writing explicitly > about *all* news items. There is a disconnect here, I'm not sure what has been > decided. Read the original message and not the partial quote that you made. I explicitly said there was an exception for --overwrite type posts. But any restriction being made on posting due to not posting drafts to the list would be complete. Allan
[arch-dev-public] Restricting ability to post news items
Hi all, Following the roll out of the base metapackage, and its poorly written news post, we agreed that all new posts should have a draft posted to arch-dev-public. There was a potential exclusion for trivial --overwrite posts [1]. This was followed for the Xorg update. It was not followed for the zstd update. Given people have shown an inability to follow simple instructions, I am proposing a restriction on the ability to make news posts. This can either be a bulk restriction now, or just removing anybody who does not follow the rule from ever making a news post again. Allan [1] and why have we had so many missing library symlinks lately... surely checkpkg detects all the missing symlinks compared to previous packages.
Re: [arch-dev-public] Adding a "posix" metapackage
On 4/1/20 12:47 pm, Sébastien Luttringer via arch-dev-public wrote: > One unfortunate consequence could be to have packages rely on it to make > dependencies shorter, and make us pull cups or cronie. What?! That is like saying one unfortunate consequence of pamcan hooks is that packages can have no files and just download and compile the source in a hook. It is a ridiculous consideration. A
Re: [arch-dev-public] libx11/xorgproto dependency
On 21/12/19 7:31 pm, Evangelos Foutras via arch-dev-public wrote: > Downstream consumers of libx11 shouldn't have to know and account for > libx11's headers/pkg-config files referencing xorgproto. A > libx11-devel package would depend on xorgproto. Since there's no > separate -devel package, the dependency stays with the regular libx11 > package. This matches my opinion on the matter. A
Re: [arch-dev-public] cleanup dead Xorg packages | news draft
On 20/12/19 8:35 am, Andreas Radke via arch-dev-public wrote: > Packages have been rebuilt and prepared to remove obsolete libdmx and > libxxf86dga. Xorgproto legacy support has been removed and wherever it > was added to be a runtime dependency it is now a build time > dependency. Some packages will need additional xorgproto makedependency > added when missing some header now that the libraries won't pull it in > anymore. That behavior is desired. I'm still in the process of fixing > the move from legacy proto headers to xorgproto headers. I'm going to > commit the changes to trunk for future builds. > > There's no package replacing libdmx and libxxf86dga so manual > intervention will be needed. So here's a small news draft: > > "Xorg cleanup requires manual intervention > > "In the process of Xorg cleanup [1] the update requires manual > intervention when you hit this message: > > :: installing xorgproto (2019.2-2) breaks dependency 'inputproto' required > by lib32-libxi > :: installing xorgproto (2019.2-2) breaks dependency 'dmxproto' required by > libdmx > :: installing xorgproto (2019.2-2) breaks dependency 'xf86dgaproto' > required by libxxf86dga > > when updating, use: > `pacman -Rdd libdmx libxxf86dga && pacman -Syu` > > to perform the upgrade. After the update it will be safe to also remove > the "xorgproto" package. This why does xorgproto not provides=('inputproto' )? Then all we need to do is announce, update and remove. Allan
Re: [arch-dev-public] Proposal: Build a ruleset for new packages and package quality
On 12/12/19 10:21 pm, Christian Rebischke via arch-dev-public wrote: > 3. revive the discussion around better PKGBUILD quality (see also Eli's > thread about PKGBUILD quality on arch-tu: > https://lists.archlinux.org/private/arch-tu/2019-November/83.html) Any chance that can be posted somewhere visible to those that aren't TUs? Allan
[arch-dev-public] Reproducible builds progress and the upcoming rebuild of [core]
Hi all, As you may know, we have had people busy looking at what it takes to make our packages reproducible. There has been a lot of progress there lately. Our reproducible builds team (along with the wider reproducible builds community) has been building our packages in different environments to test how stable the builds are [1]. The good news is that >80% of our packages could be built twice in varying environments and give the exact same result. However, that is only part of the picture. Ideally, we want people to be able to take one of our packages and rebuild it exactly. With the release of pacman-5.2, packages record a lot more information about their build environment. That means we can reconstruct a package's build chroot, and then rebuild it. There are two tools in the works to do this. One by Morton (Foxboron) [2] and one by Eli [3]. Note that both tools need more testing to be ready for a wider release and currently require some manual editing to run. The good news is, we have at least 10 packages that can be precisely reproduced using both these tools [4]! This means you can take one of these tools and rebuild a package from the repos, and get the exact same package out of it. This is an amazing effort - well done to the team! To keep this momentum going, it would be great to rebuild every package in [core] using makepkg from pacman-5.2+. That way we can test which packages are actually reproducible and work towards fixing those that are not. So be prepared for almost the entire repo to hit [testing] soon, and get your sign-off shoes on! Again, a huge congrats to our reproducible builds team. This has been a massive amount of work! Allan [1] https://tests.reproducible-builds.org/archlinux/archlinux.html [2] https://github.com/archlinux/archlinux-repro [3] https://github.com/eli-schwartz/devtools/blob/reproducible/makerepropkg.in [4] https://wiki.archlinux.org/index.php/DeveloperWiki:ReproduciblePackages
Re: [arch-dev-public] Using SPDX License list as identifiers
On 22/10/19 9:59 pm, Jerome Leclanche wrote: > It would also be a > little more accurate; eg. the SPDX allows for distinctions such as > "LGPL-3.0-or-later" vs. "LGPL-3.0-only". I thought we already managed that, but it seems in rather limited use these days looking at our -Si output. e.g. Licenses: LGPL-2.1 Licenses: LGPL-2.1+ A
Re: [arch-dev-public] New ideas for notifying users about (minor) changes
On 30/7/19 10:44 am, Christian Rebischke wrote: > One problem I see with the News section is that it's Dev only. I > wouldn't even know who I need to ask (and I am TU for several years > now). I mean I could just ask around in the IRC, but it would be nice to > have this at least documented somewhere. I guess the assumption is changes that affect a larger proportion of users are in packages maintained by devs. However, I'm happy opening the news to TUs if needed. Allan
Re: [arch-dev-public] New ideas for notifying users about (minor) changes
On 30/7/19 8:47 am, Christian Rebischke via arch-dev-public wrote: > Hello everybody, > I got assigned to this issue here: > > https://bugs.archlinux.org/task/62823 > > The users would like to have a notice in pre/post install/upgrade > notifications via pacman .install files. > > I am not a fan of spamming such news via pacman and I think the > installation/upgrade process should be clean of such messages, but I > don't have access to the news tool on our website as well. I think this is a clear case of user intervention needed. So a post_install message (restricted to the update where intervention was needed) is appropriate. If the package was more widely used, we have the News section of the website. I don't think anything else is needed. Allan
Re: [arch-dev-public] Proposal for a new organisation structure
On 3/6/19 2:39 pm, Christian Rebischke via arch-dev-public wrote: > 3. I don't like that devs and TUs live aside each other instead of > finally realizing themselves as community or group. The TUs are set up as an independent organisation with their own bylaws. Many of those bylaws were implemented to keep independence from the developer team. Saying that, almost all developers have been a Trusted User, and there is a decent overlap currently. I mostly see this as a responsibility divide. > I think the > community as itself would work much better if we would have user-access > based package repositories and just 'maintainers', instead of this > dev/TU split. I agree that the distinction between [extra] and [community] is rather limited. I think we need a better definition of what [extra] is, and move packages that don't fit that definition out of [extra] and into [community] where both devs and TUs can maintain them. Or even merge those two repos. >> As Gaetan already mentioned, the process is clear: somebody suggests a >> new developer and we discuss until a consensus is reached. Feel free to >> document that somewhere in our wiki if you think it needs to be >> documented. If you have specific concerns with that process, feel free >> to share them. However, I do not think anybody in the dev team ever had >> any objections against that procedure. > > What interests me is why is this whole process not transparent and > public? I mean we discuss adding new TUs publicly. Of course this > dicussion comes with all its good and bad parts, but atleast it's > transparent and people get a reason why they are not elected. You are very much overthinking what goes on when deciding on a new developer. "Electing" a developer is very different than electing a TU. People given developer positions have probably been around for years and are already doing the job before they get formally offered it. So it is a case of someone saying "this person should be a developer" and the rest going "OK". There is usually no discussion, because the candidate is well known already, and has obviously been doing a good job to even be nominated. Having TU style discussions on the suitability of a candidate makes no sense in that situation. Allan
Re: [arch-dev-public] Proposal for a new organisation structure
On 2/6/19 9:08 am, Christian Rebischke via arch-dev-public wrote: > 1. Simplified repositories > > Currently we have [core], [extra] and [community]. These three > repositories are there, because they have grown to this structure over > time. At the moment I don't see any real meaning for the split of [core] > and [extra]. So one suggestion would be to either: > - merge [extra] and [core] I stopped reading here. If you don't understand the difference between [core] and [extra], you should ask. Particularly before proposing the removal of [core]. Allan
Re: [arch-dev-public] bringing vivaldi browser to community
On 2/6/19 1:53 am, Ike Devolder via arch-dev-public wrote: > On Sat, 2019-06-01 at 21:30 +1000, Allan McRae wrote: >> You don't seem to >> explain why you need to ask in your email. > > Because it is proprietary and I explain that now there is a valid > reason compared to 3 years ago where there was practically no > difference between vivaldi, chromium and opera. > Does the license allow us to have it in the repos? After a quick look, I'd say no. A
Re: [arch-dev-public] bringing vivaldi browser to community
On 1/6/19 8:12 pm, Ike Devolder via arch-dev-public wrote: > 3 years have passed since I first proposed to bring vivaldi into > community. Now there is a clear differentiation between what vivaldi > offers out-of-the box compared to other browsers. > > Vivaldi offers a ton of customisation features out of the box, and is > also able to just use the chrome/ium addons from the chrome webstore. > > Personally I'm using vivaldi as my main browser since somewhere in 2015 > (shortly after the first beta was released) and the key features no > other browser currently offers are: > - webpanels > - quick commands > - tabtiling > - tabstacking > - tabbar positioning > > I'll bring it in the same way as with opera, where you have the > webbrowser + separate packge with libffmpeg.so to allow the playback of > proprietary formats like mp4. > Does the license allow us to distribute it? And dozens of packages get added to [community] a week. Why are you asking here unless you think there will be an issue? You don't seem to explain why you need to ask in your email. Allan
Re: [arch-dev-public] Create guidelines regarding SIMD instructions/x86 extensions
On 25/5/19 9:34 pm, Bruno Pagani wrote: > Out of curiosity, what did you rebuild of [core] lead to? I had a potentially slightly faster system for a week... It was mainly a test to see if I spotted some build issues of test suite failures beyond what is seen for x86_64. All was good. A
Re: [arch-dev-public] Create guidelines regarding SIMD instructions/x86 extensions
On 25/5/19 9:19 pm, Bruno Pagani via arch-dev-public wrote: > Hi, > > Le 25/05/2019 à 02:17, Filipe Laíns via arch-dev-public a écrit : >> I would also like to explore the idea of adding an "high performance" >> architecture which would be able to make use of SSE{,2,3,4,4.1,4.2} and >> AVX, which seem to be the standard for newer processors (>=2013). This >> would only be available for packages that do high performance computing >> (ex. openblas, sdrangel, etc.). Any thoughts on this? > > As said on IRC, they have been discussions before on having multiple > targets and corresponding repos, but the starting point is that we need > automated build before going into such a direction, and this in turn has > several requirements. I’ve linked to you the pad where we put our ideas > together regarding this. > > In the meantime, we had the case before of whether we should package > e.g. $pkgname-{sse4,avx} in a case where it mattered a lot, but it > turned out the software in question (embree) is able to do runtime > detection of available ISA. Maybe some other packages are doing this > too, else we could discuss whether allowing such flavours as a temporary > measure would be acceptable for selected packages. glibc detects available instruction sets and uses the best for many functions. I'd be very, very, very much against providing multiple variants of a package in our repos. Using asp and makepkg are is a hard solution for those who really need a few packages rebuilt. PS - I rebuilt [core] with -march=haswell recently as a test. Automated building is not an issue. Unattended package/database signing is the major stumbling block. Allan
Re: [arch-dev-public] Create guidelines regarding SIMD instructions/x86 extensions
On 25/5/19 5:22 pm, Lukas Jirkovsky via arch-dev-public wrote: > On Sat, 25 May 2019 at 04:27, Filipe Laíns via arch-dev-public > wrote: >> Setting `-mtune` to generic won't add any additional instruction sets >> by itself, but it does not prevent instruction sets from being added. >> Looks like GCC enables MMX, SSE and SSE2 by default, it isn't related >> at all to `-march` like I stated in the email but it still presents the >> same issue. > > As far as I know, MMX, SSE and SSE2 are mandatory part of the AMD64 > instruction set, so they are not enabled randomly just because someone > felt like it, but because they are be present on every x86_64 cpu. > . Correct. Using the command I gave in my first reply: $ gcc -march=x86-64 -Q --help=target | grep sse -mfpmath= sse -mno-sse4 [enabled] -msse [enabled] -msse2[enabled] -msse2avx [disabled] -msse3[disabled] -msse4[disabled] ... $ gcc -march=x86-64 -Q --help=target | grep mmx -mmmx [enabled] -mtune just tunes instructions for a "representative" set of "current" CPUs that run as x86-64. Allan
Re: [arch-dev-public] Create guidelines regarding SIMD instructions/x86 extensions
On 25/5/19 10:17 am, Filipe Laíns via arch-dev-public wrote: > Hello, > > Currently there are no guidelines stating which x86 extensions (ex. > SSE2, SEE3, SSE4, AVX, etc.) we support. This is a bit problematic > since it lets compilers do what they want and possible generate code > that can't run on some systems. > > Even though this is an issue, it's not complete anarchy, at least yet! > Just kidding :p. The vast majority of our native packages are compiled > with GCC and we do default to `-mtune=generic` which is good but not > optimal. `-mtune=generic` tells GCC to compile for a generic processor > so it's up to GCC to decide which architecture extensions would compose > a generic processor. I haven't been able to find any documentation on > what x86 extensions are enabled for a "generic" processor but I was > able to track them down to MMX, SSE (or KNI) and SSE2. Being > undocumented they could change at any time so I don't think we should > rely on `-mtune=generic`. > I think you need to look at the difference between -march and -mtune. We use "-march=x86-64", which defines the instruction sets that can be used. Adding "-mtune=generic" does not allow the inclusion of additional instruction sets. Look at the output of: gcc -march=x86-64 -Q --help=target Allan
Re: [arch-dev-public] RFC: (devtools) Changing default compression method to zstd
Given the suggestion of using -18-, I decided to calculate how much bigger our packages would be with the numbers given: cuda-10.0.130-2-x86_64.pkg.tar 58.5M 104.40% gcc-8.2.1+20181127-1-x86_64.pkg.tar 2.8M108.30% go-2:1.12.1-1-x86_64.pkg.tar10.6M 108.70% linux-5.0.3.arch1-1-x86_64.pkg.tar -0.4M 99.40% linux-headers-5.0.3.arch1-1-x86_64.pkg.tar 1.9M111.20% tensorflow-1.13.1-2-x86_64.pkg.tar 6.2M111.20% intellij-idea-community-edition-2:2018.3.5-18.1M102.10% That is a decent increase. Are the times given for decompress just a decompress, not install by pacman? Were they done on a SSD or pointed at /dev/null? (I assume not a spinning disk given the times) Allan
Re: [arch-dev-public] RFC: (devtools) Changing default compression method to zstd
On 25/3/19 9:28 am, Robin Broda via arch-dev-public wrote: > On 3/25/19 12:22 AM, Andrew Gregory wrote: >> On 03/25/19 at 12:15am, Robin Broda via arch-dev-public wrote: >>> On 3/24/19 11:20 PM, Evangelos Foutras via arch-dev-public wrote: >>>> On Sun, 24 Mar 2019 at 23:45, Allan McRae via arch-dev-public >>>> wrote: >>>>> >>>>> We need to assume every system has a copy of pacman-5.2+ before we can >>>>> switch packages to zstd. >>>> >>>> Why is pacman support needed here? I can already install .zstd >>>> packages using pacman 5.1.3. >>>> >>>> The crucial part seems to be libarchive support, which was added in >>>> v3.3.3 (~ September 2018). >>>> >>> >>> Yes, installing zstd packages works - the pacman release is merely >>> required for makepkg. Unless that has already landed too, which >>> would be news to me :) >>> >>> Thus i don't think we need a hold-off period like this, Allan. >> >> We still need a hold-off period, we're just waiting to make sure >> people have libarchive v3.3.3 instead of pacman v5.2.0. >> > > That update happened half a year ago, i'm sure that most people with an > installation that old will already have to fetch other packages, like the > keyring, separately for it to go through. Fetching a keyring does not potentially bump sonames. > Plus, with libarchives' release cycle, i don't think that libarchive itself > is gonna be rebuilt immediately after the change is implemented - providing > extra time to upgrade libarchive without having to download a release packed > as xz separately. And if openssl gets and soname bump? A
Re: [arch-dev-public] RFC: (devtools) Changing default compression method to zstd
On 25/3/19 4:34 am, Robin Broda via arch-dev-public wrote: > This change requires a new pacman release, as as of writing this, zstd > support is in master but hasn't landed in a release yet. Which is a complete blocker for quite a period of time. We need to assume every system has a copy of pacman-5.2+ before we can switch packages to zstd. Otherwise updates can break systems (pacman and all dependencies will need to stay as .xz for a year at least, and we have to hope that partial update does not break anything until a full update is run). Experience switching from .gz to .xz showed system breakage was a fairly regular occurrence. I would not do this until at least one year, maybe two after pacman-5.2 is released. Allan
Re: [arch-dev-public] A contrib repository
On 3/14/19 8:46 AM, Morten Linderud via arch-dev-public wrote: > There is a *lot* of small tools people have written over the years that > resides > in bin/ directories which could be useful for more people. We also have > several > such tools on soyuz, where sogrep was added to devtools this week. > > I have been thinking it could be great to have a simple `contrib` repository > where every team member has commit access. This could work as a staging area > for > tools we would like to promote to `devtools` later on. > > We could maybe sort this into directories for its purpose: > * packaging > * security > * devops > * testing > * bugwrangler > * misc > > Tools that can be added here is the `ch` scripts from Bluewind, and the pkg-* > tools eli has created. I also have some tools to look for pkgname in archweb, > check them out from svn and check them against a nvchecker file. > > This would hopefully give us a space where we can experiment with new > maintainer > tools in a collaborative manner. I'd love to hear some feedback or thoughs on > this! > I was fairly sure any user can create a git repo on our server. Look at "Developer Projects" on https://git.archlinux.org/ . Or use github, where some of these scripts are already located. I don't see the need for another repository. A
Re: [arch-dev-public] Python 2 modules
On 17/2/19 10:37 am, Eli Schwartz via arch-dev-public wrote: > On 2/16/19 4:36 PM, Christian Hesse wrote: >> Allan McRae via arch-dev-public on Sat, >> 2019/02/16 11:19: >>> Below is a list of python2 modules that are a dependency for any other >>> package. I did not check makedepends and I did not check recursively to >>> build this list. >> >> The list contains optional dependencies. How to handle these? > > I see no conceptual difference between optional dependencies and hard > dependencies. Both are equally deserving of being in the repos for the > sake of providing dependent functionality to other packages. > Yes - this was a quick pass list. There is probably a bunch that should not be removed and a lot more that could be removed... A
[arch-dev-public] Python 2 modules
Hi all, Python 2 reaches End of Life on 2020-01-01. We currently have >950 python2 modules in the repos. A lot of these are not used by any other package in the repositories. I think we should work towards removing them. Leaving only python2 modules that are really required by other software, highlights what needs worked on to port to python3. Note Fedora is doing a similar removal for F30: https://fedoraproject.org/wiki/Changes/Mass_Python_2_Package_Removal What are opinions on this? Should I make a TODO list? Below is a list of python2 modules that are a dependency for any other package. I did not check makedepends and I did not check recursively to build this list. python2-acme python2-antlr2 python2-anyjson python2-anytree python2-apache-libcloud python2-apispec python2-argcomplete python2-argon2_cffi python2-argparse python2-args python2-arrow python2-aspectlib python2-astor python2-atspi python2-aubio python2-audit python2-augeas python2-autobahn python2-autopep8 python2-backports.lzma python2-basemap python2-betamax-matchers python2-betamax-serializers python2-binary-memcached python2-biopython python2-bitvector python2-blist python2-blosc python2-bluepy python2-bottle python2-bottleneck python2-braintree python2-breathe python2-bsddb python2-btchip python2-btrees python2-cached-property python2-caja python2-cchardet python2-celery python2-chai python2-chameleon python2-characteristic python2-cjkwrap python2-click-log python2-click-threading python2-cloudflare python2-cmarkgfm python2-colander python2-colorclass python2-configargparse python2-construct python2-couchdb python2-cram python2-crayons python2-cryptography-vectors python2-cson python2-cssselect2 python2-cssutils python2-cx_freeze python2-d2to1 python2-daemon python2-daemonize python2-datrie python2-ddt python2-digitalocean python2-discid python2-distutils-extra python2-django python2-dnslib python2-dockerpty python2-docopt python2-docs python2-doublex-expects python2-dpcontracts python2-dropbox python2-editdistance python2-egenix-mx-base python2-elasticsearch-curator python2-email-validator python2-envisage python2-eric python2-ethtool python2-evdev python2-exam python2-exiv2 python2-eyed3 python2-factory-boy python2-fastpbkdf2 python2-faulthandler python2-flake8-blind-except python2-flake8-debugger python2-flaky python2-flask-gravatar python2-flask-htmlmin python2-flask-jwt python2-flask-mail python2-flask-migrate python2-flask-paranoid python2-flask-restful python2-flask-security python2-flask-socketio python2-flask-talisman python2-flask-wtf python2-flexmock python2-flickrapi python2-flup python2-fonttools python2-foolscap python2-fpconst python2-freezegun python2-fs python2-funcy python2-furl python2-fxa python2-gasp python2-gcp-devrel-py-tools python2-gdal python2-gdata python2-genshi python2-genty python2-geoip python2-gevent-websocket python2-gflags python2-gitpython python2-gnupg python2-gnupginterface python2-gnuplot python2-gpgme python2-grequests python2-gtkspellcheck python2-gudev python2-h2 python2-h5py python2-h5py-openmpi python2-hacking python2-harparser python2-helper python2-hexdump python2-hglib python2-httpretty python2-hunter python2-hypothesis python2-i3-py python2-ibm-db-sa python2-icalendar python2-igraph python2-importlib_resources python2-inet_diag python2-invoke python2-iocapture python2-ipdb python2-irc python2-isomd5sum python2-iwlib python2-jieba python2-js2py python2-jsbeautifier python2-json-logger python2-jsonrpclib-pelix python2-kaitaistruct python2-kajiki python2-kaptan python2-keybinder2 python2-keyrings-alt python2-keyutils python2-kitchen python2-kivy python2-klein python2-langdetect python2-language-server python2-lark-parser python2-levenshtein python2-libappindicator python2-libarchive-c python2-libforensic1394 python2-librabbitmq python2-libtmux python2-linux-procfs python2-llfuse python2-logbook python2-logilab-common python2-lttngust python2-m2r python2-magic python2-mamba python2-manhole python2-manuel python2-marisa python2-marshmallow python2-memcached python2-mimerender python2-mockito python2-mongoengine python2-mongomock python2-mpd2 python2-munkres python2-musicbrainz2 python2-mygpoclient python2-mysql-connector python2-nbxmpp python2-ndg-httpsclient python2-neovim python2-netcdf4 python2-netcdf4-openmpi python2-nine python2-nltk python2-nose2 python2-nose-cover3 python2-nose-exclude python2-nose-fixes python2-nose-randomly python2-nose-show-skipped python2-nosexcover python2-oauth2client python2-objgraph python2-olefile python2-openapi-spec-validator python2-openpyxl python2-openstackclient python2-oslo-concurrency python2-oslo-log python2-oslosphinx python2-oslotest python2-ovirt-engine-sdk python2-owslib python2-pacparser python2-pam python2-pandas-datareader python2-pandocfilters python2-parse python2-parsedatetime python2-parsel python2-paste python2-pastedeploy python2-pbkdf2 python2-pdoc python2-peewee python2-perf python2-periphery python2-phonenumbers python2-piexif
Re: [arch-dev-public] Proposal: minimal base system
On 13/2/19 8:17 am, Levente Polyak via arch-dev-public wrote: > On 2/12/19 7:16 PM, Gaetan Bisson via arch-dev-public wrote: >> [2019-02-12 16:40:08 +0100] Bruno Pagani via arch-dev-public: >>> Just in case it wasn’t clear, my answer would have been mostly the same >>> as Eli’s. >>> >>> So, Gaetan, Allan and Bartłomiej (or anyone else for that matter), do >>> you have further comments/questions regarding this, does the existence >>> of the base group alongside the arch/minimal-system now makes sense or >>> would you still prefer to go without it? >> >> Allan and I have both stated that we don't want to introduce a new group >> since we believe it would be highly redundant with base. >> >> Nothing new has been said since our last messages except Eli's post >> which argues that the base group is largely inadequate in its current >> state. This further supports our proposal that base should be improved >> instead of introducing a new group. >> >> So I really don't see what arguments could have changed our minds... >> It's also strange to me how you can concur with Eli's post without >> agreeing with our conclusions. >> >> To go forward I suggest you propose a clear definition of the perfect >> "minimal system" group you'd want to have, along with a proposed list of >> packages. When consensus is reached, we adopt this list of packages for >> base and put this definition on the wiki. >> >> Cheers. >> > > To make it as short as possible, the idea is not just to strip down the > base group further but primarily to not use a group in the first place. > It should be replaced with something that is consistent within itself > over the whole lifetime of the system. > Groups are the wrong tool for such a set: you explicitly install all > those packages so they won't automatically be mark as not-required > anymore once removed from that group, as well as new additions are not > consistent during the lifetime of the system. We are clear about that. Call it a group or metapackage or whatever, the objection is having the current base and the new "group" at the same time. A
Re: [arch-dev-public] Proposal: minimal base system
On 5/2/19 11:38 pm, Bruno Pagani wrote: > Maybe. As I said in my answer to Bartłomiej, I don’t know if beginners > know enough things to install what they need beyond the minimum system, > or if they just read the wiki about doing this or that, which might > assume they have the current base group installed. If only the wiki wasn't a static document, and could be updated after any change was made... Even then I am fairly sure people wanting to set-up LVM can install the lvm package. But I see "competent Linux user" has been removed from the front page, so maybe we target idiots these days. >> If we remove the excess from base, then we are down to a very small >> difference between that and archlinux-system. Only e2fsprogs, man, and >> an editor different? >> >> So I see the proposed archlinux-system group being essentially what base >> should be. > > That is because you see base as the minimal system. So I’ll turn this > differently: do you have objections against having, outside of the > minimal meta-package described in our proposal, a packages group of > “relatively standard” tools, that is purposed at beginner wanting to > have only one simple pacstrap command to issue in order to get started? > > Or put it yet another way: outside of this base group, does our proposal > of a minimal metapackage suits you? If so, why does it matter to you > that there is also a base group, provided the name is not subject to > confusion, that has this metapackage plus other tools (that e.g. people > coming from random other distro would expect to have at hand from the > start), knowing that you would likely have almost no interactions with > this group? If not, then I’m even more importantly waiting for your > comments. I don't object to redefining the minimal set of packages we expect installed and making it a metapackage. Currently base is bloated, and a metapackage makes some sense. archlinux-system - minimal set of packages base-devel - collection of utilities most people expect installed So where does that leave base? base - a smaller collection of utilities most people expect installed I see redundancy being created. That is what I object to. Allan
Re: [arch-dev-public] Proposal: minimal base system
On 5/2/19 9:06 pm, Bruno Pagani wrote: > Le 22/01/2019 à 00:59, Allan McRae a écrit : >> On 22/1/19 9:41 am, Bruno Pagani wrote: >>> Le 22/01/2019 à 00:23, Allan McRae via arch-dev-public a écrit : >>>> On 22/1/19 8:03 am, Levente Polyak via arch-dev-public wrote: >>>>> Everything that won’t be part of base-system needs to be added as a >>>>> dependency to all requiring packages; alternatively don't omit any first >>>>> level runtime dependencies at all. >>>>> >>>>> This package should only depend on strictly required explicit packages >>>>> to get a working minimal Arch Linux system. >>>>> >>>>> The proposed end result is: >>>>> - base: convenient helper group for quickly getting a working system >>>>> when installing, must include the new base-system package >>>>> - base-system: package defining the minimum dependencies for a working >>>>> base runtime >>>> I think the proposal is OK. I'm not comfortable with our line about >>>> base group packages being required given how many of them I don't have >>>> installed. >>>> >>>> However... I don't like idea of the base group and base-system package >>>> existing together. You definition of what base-system should be is much >>>> the same as what the base group was defined to be. What package >>>> justifies itself in the base group, but would not be in base-system? It >>>> seems we would have two very similar things where one would do. >>>> >>>> Allan >>> In the proposal, base would really just be a convenient helper for e.g. >>> beginners installing their system, so they could get all tools that are >>> often used during install (e.g. cryptsetup, lvm2, various FS/network >>> tools, etc.) or (POSIX) tools people coming from other distros would >>> expect to be here by default (man pages, nano/vi…) but that are >>> interactive ones and thus not really required for operating. >>> >>> Anyone knowing their stuff could just install base-system + what they >>> actually need (e.g., I would install cryptsetup and vim, and not care >>> about netctl, xfsprogs or lvm2). >> "Anyone knowing their stuff" is the essentially the stated Arch target >> audience. > > So apparently we did not answer all concerns here. I don’t expect Arch > users to know thing so well that they know exactly what tools are in > which packages when they install Arch for the first time. I think we > should not mistake Arch Power Users, people that have a level of > knowledge above Arch Users, that are just generic Linux Power Users. > >> So, the definitions of the sets of packages are: >> >> base-system - essential packages we assume everyone has installed >> (previous definition of base...) > > To be clearer, the new proposition would be to call this arch-system to > avoid confusion with base. However, note that this “previous definition > of base” is definitively not that clear: when I installed Arch, I read > things as “base is a convenient helper to get almost every standard > tools you could need to do your install”. > >> base group - base-system plus other packages some people probably >> want/expect and support packages for filesystem types most people don't >> actually need. > For me, base will be what it has ever been: a fast way to get started as > an Arch beginner. >> Maybe slightly facetious on that last one, but I don't see a clear need >> for the base group once base-system exists. > > Because, as an Arch dev, you definitively qualify as an Arch Power > Users. I wouldn’t use base either for myself, but I firmly believe most > Arch beginners would. > > Does that make sense to you, or do you still think every new Arch User > should already know exactly what is required to get started? > If someone knows they want to set up logical volumes and drive encryption, then they know enough to install lvm and cryptsetup. Same with jfsutils, xfsutils. So I don't think they should be in the base group (e.g. I would not call jfsutils a standard tool). If we remove the excess from base, then we are down to a very small difference between that and archlinux-system. Only e2fsprogs, man, and an editor different? So I see the proposed archlinux-system group being essentially what base should be. A
Re: [arch-dev-public] Proposal: minimal base system
On 22/1/19 9:41 am, Bruno Pagani wrote: > Le 22/01/2019 à 00:23, Allan McRae via arch-dev-public a écrit : >> On 22/1/19 8:03 am, Levente Polyak via arch-dev-public wrote: >>> Everything that won’t be part of base-system needs to be added as a >>> dependency to all requiring packages; alternatively don't omit any first >>> level runtime dependencies at all. >>> >>> This package should only depend on strictly required explicit packages >>> to get a working minimal Arch Linux system. >>> >>> The proposed end result is: >>> - base: convenient helper group for quickly getting a working system >>> when installing, must include the new base-system package >>> - base-system: package defining the minimum dependencies for a working >>> base runtime >> I think the proposal is OK. I'm not comfortable with our line about >> base group packages being required given how many of them I don't have >> installed. >> >> However... I don't like idea of the base group and base-system package >> existing together. You definition of what base-system should be is much >> the same as what the base group was defined to be. What package >> justifies itself in the base group, but would not be in base-system? It >> seems we would have two very similar things where one would do. >> >> Allan > > In the proposal, base would really just be a convenient helper for e.g. > beginners installing their system, so they could get all tools that are > often used during install (e.g. cryptsetup, lvm2, various FS/network > tools, etc.) or (POSIX) tools people coming from other distros would > expect to be here by default (man pages, nano/vi…) but that are > interactive ones and thus not really required for operating. > > Anyone knowing their stuff could just install base-system + what they > actually need (e.g., I would install cryptsetup and vim, and not care > about netctl, xfsprogs or lvm2). "Anyone knowing their stuff" is the essentially the stated Arch target audience. So, the definitions of the sets of packages are: base-system - essential packages we assume everyone has installed (previous definition of base...) base group - base-system plus other packages some people probably want/expect and support packages for filesystem types most people don't actually need. Maybe slightly facetious on that last one, but I don't see a clear need for the base group once base-system exists. Allan
Re: [arch-dev-public] Proposal: minimal base system
On 22/1/19 8:03 am, Levente Polyak via arch-dev-public wrote: > Everything that won’t be part of base-system needs to be added as a > dependency to all requiring packages; alternatively don't omit any first > level runtime dependencies at all. > > This package should only depend on strictly required explicit packages > to get a working minimal Arch Linux system. > > The proposed end result is: > - base: convenient helper group for quickly getting a working system > when installing, must include the new base-system package > - base-system: package defining the minimum dependencies for a working > base runtime I think the proposal is OK. I'm not comfortable with our line about base group packages being required given how many of them I don't have installed. However... I don't like idea of the base group and base-system package existing together. You definition of what base-system should be is much the same as what the base group was defined to be. What package justifies itself in the base group, but would not be in base-system? It seems we would have two very similar things where one would do. Allan
Re: [arch-dev-public] Mongodb and SSPL
On 17/1/19 12:02 am, Morten Linderud via arch-dev-public wrote: > Yo! > > As probably some of you have realized, there is a discussion regarding mongodb > and the relicense from AGPLv3 to SSPLv1. > Under section "5. Conveying Modified Source Versions" the most relevant part > for > us is section "a)". > > a) The work must carry prominent notices stating that you modified it, > and giving a relevant date. > > Which means we need to give some heads-up that the source is changed. > I looked at what changes are made: # Broken tls13 support, removing to fix build sed -i '/counts.tls13/d' src/mongo/util/net/ssl_manager_openssl.cpp So I'd agree that is modified enough that the rest of the conditions apply. My conclusion is, having this package in the repos would require too much interpretation of a non-standard license to ensure compliance. Drop the package. Allan
Re: [arch-dev-public] TU application process
On 6/11/18 8:15 pm, Bruno Pagani wrote: > Le 06/11/2018 à 11:12, Christian Rebischke via arch-dev-public a écrit : >> On Tue, Nov 06, 2018 at 08:09:03PM +1000, Allan McRae wrote: >>> On 6/11/18 7:54 pm, Christian Rebischke via arch-dev-public wrote: >>>> Hello everybody, >>>> >>>> First of all, the following mail has nothing to do with the last two TU >>>> applications, it's a general view on the current TU application process. >>>> >>>> I would like to propose a new process for TU applications due to several >>>> reasons: >>>> >>> Read the TU bylaws. It has specific instructions of where proposals >>> must be posted (hint: not here...). >>> >>> A >> Hi Allan, >> >> This mail wasn't meant as proposal. It's just a general discussion about >> this topic and people said in the TU IRC channel yesterday, that >> arch-dev-public would be the >> right mailinglist for such discussion. >> >> chris > Specifically, we are also interested in the input of devs, not just TUs. Strange, given TUs are set-up to be an independently governed group from developers... But because you asked my opinion, I think a TU council is a really, really, really bad idea. No need to set some TUs above others. We have never had a formal hierarchy in the developers (apart from our glorious leader), and are instead run by those who step up to lead what needs done. I believe that this is what makes Arch work, and governance would be detrimental to the distribution as a whole. Personally, I'd get rid of all quorum for electing a TU, and make inactive TUs be measured purely on the basis of package updating. Most TU application discussions are inane beyond the customary package review. And when someone applies and their packages are very bad, their sponsor should be held in shame. Finally, I don't want to hear what the minions are up to! Get back to your own mailing list. :) A
Re: [arch-dev-public] TU application process
On 6/11/18 7:54 pm, Christian Rebischke via arch-dev-public wrote: > Hello everybody, > > First of all, the following mail has nothing to do with the last two TU > applications, it's a general view on the current TU application process. > > I would like to propose a new process for TU applications due to several > reasons: > Read the TU bylaws. It has specific instructions of where proposals must be posted (hint: not here...). A
Re: [arch-dev-public] .gnuog owned by root when using devtools in first place
On 13/10/18 7:23 pm, NicoHood wrote: > I ran extra-x86_64-build on a new system with an uninitialized gnupg > keyring. I think it was responsible for creating a .~/gnupg directory, > but with root privilegs. > > I chowned it to my user, it is all good now. Maybe the script could be > improved to check if the gnupg keyring was initialzed or not, to prevent > such issues. I was confused in the first place. If only there was a place to report bugs. Somewhere where they could be tracked... A
Re: [arch-dev-public] Moving gcc7 into [community] for CUDA
On 29/05/18 11:00, Sven-Hendrik Haase wrote: > Hey, > > I would like to move gcc7 (based on the latest version 7 commit of gcc [0]) > into [community] because of cuda 9.2 and in return drop gcc54. > > I tried to make it work with current gcc but to no avail. In earlier > releases of cuda the incompatibilities could be patched with header hacks > but not this time. I tested the whole setup with cuda 9.2 locally and it > seems to work fine. > > I just want to get somebody's ok on this as apparently not everyone is > stoked about having yet another old version of gcc in there for some time. > > Sven > > [0] > https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/gcc&id=44e0a03db1b5cabf6135f194e540d513cf959245 > . > OK to add gcc7 given it will drop gcc54. A
Re: [arch-dev-public] RFC: Dropping -DCMAKE_BUILD_TYPE from packages using cmake
On 14/03/18 04:19, Bartłomiej Piotrowski via arch-dev-public wrote: > On 2018-03-13 14:40, Allan McRae via arch-dev-public wrote: >> On 13/03/18 21:22, Jan Alexander Steffens via arch-dev-public wrote: >>> For that matter, I'm all for putting an arch-configure helper into our >>> autoconf package. >> >> Don't muck around with vanilla packages. Put it in another package >> (devtools). >> >> A >> > > Makes no sense unless devtools become part of base-devel. It's hardly > any meddling as it doesn't hinder possibility to use default configure > script or meson binary. > It is Arch specific, so should be in an Arch specific package. A
Re: [arch-dev-public] RFC: Dropping -DCMAKE_BUILD_TYPE from packages using cmake
On 13/03/18 21:22, Jan Alexander Steffens via arch-dev-public wrote: > For that matter, I'm all for putting an arch-configure helper into our > autoconf package. Don't muck around with vanilla packages. Put it in another package (devtools). A
Re: [arch-dev-public] Automating package conflict resolution whenever possible (xorgproto/libxfont dependency conflict)
On 14/02/18 22:28, Florian Pritz via arch-dev-public wrote: > How about having this feature in pacman, maybe with an indicator if the > package is still in a repository? pacman -Qtd
Re: [arch-dev-public] Automating package conflict resolution whenever possible (xorgproto/libxfont dependency conflict)
Just because I had to look up the details of this - xorgproto replaces a lot of packages, including fontsproto: :: Replace fontsproto with extra/xorgproto? [Y/n] - libxfont requires fontsproto, so this causes the following: :: libxfont: removing fontsproto breaks dependency 'fontsproto>=2.1.3' - people with systems older than 2016-11-16, may have libxfont as xorg-server depended on it until that time. Now xorg-server depends on libxfont2. - libxfont2 does not replace libxfont, as it is a completely different API. - libxfont was removed from the Arch repos somewhere in April or May 2017. So nothing official depends on libxfont. I don't think it is unreasonable to expect people to run "pacman -Qi libxfont", see it was installed as a dependency and no package depends on it and remove it. There also does not seem to be a correct way of us to handle this - joys of rolling release! Funny thing... people using yaourt probably removed this package as I believe it highlights dependencies that are no longer needed after an upgrade! A
Re: [arch-dev-public] Package group between repositories
On 04/01/18 22:02, Balló György via arch-dev-public wrote: > Currently the 'xfce4-goodies' package group[1] is in split between [extra] > and [community]. Most of its packages are in [extra], but 'ristretto' and > 'xfce4-whiskermenu-plugin' are in [community]. > > My question is: is it okay, or should we avoid it by removing these two > packages from the 'xfce4-goodies' group or moving them to [extra]? > > [1] https://www.archlinux.org/groups/x86_64/xfce4-goodies/ > If a TU is maintaining the packages in [community], there is little point moving them. A
Re: [arch-dev-public] Merging multilib toolchain with [core]
On 22/11/17 21:19, Bartłomiej Piotrowski wrote: > Hi, > > For a long time we have been maintaining multilib-enabled gcc package > separately under [multilib] repo. Compilation takes a lot and usually it > falls behind because I forget to keep it in sync with [core]. > > I propose to merge it with core/gcc instead and split lib32-gcc-libs. > The latter would be moved to [core] due to dbscripts inability to > maintain split packages in multiple repositories. On the same note, the > same would happen to lib32-glibc. > I was planning this for once i686 was dropped. So good to see it happening! A
Re: [arch-dev-public] Switching the bugtracker to Bugzilla
On 15/11/17 07:34, Bartłomiej Piotrowski wrote: >> There are several options for migrating the bug history to Bugzilla and a >> few options are under >> debate. (input welcome) > As I said multiple times on IRC, I'm for starting from scratch. There > are way too many inactive or/and incorrect bugs open, and honestly any > effort to review that list is a waste of time. With no bugs open we can > 1) pretend everything works fine 2) hopefully avoid zombie-bugs > apocalypse that we have now. Flyspray could be mirrored with wget for > read-only version. If you don't migrate pacman bugs, don't bother creating a pacman component. A
Re: [arch-dev-public] [draft] The end of i686 support
On 06/11/17 21:16, Eli Schwartz wrote: > On 11/06/2017 05:36 AM, Alad Wenter via arch-dev-public wrote: >>> Bartłomiej Piotrowski hat am 6. >>> November 2017 um 11:21 geschrieben: >>> >>> Slightly changing the topic... We have plenty of space on our >>> PIA-sponsored mirrors. Given that said fork pretty strictly follows >>> our PKGBUILDs (much alike to ARM team), I'd like to host arch32 >>> mirrors there as well. What do you think? >>> >> I don't mind, but in the end it's up to those who pay for the >> mirrors. >> >> It does bring up the topic again on how the Arch community will >> support arch32. Does hosting arch32 mirrors give the impression that >> we support the fork through our channels, or is that unrelated? How >> will we otherwise react on support requests for or from arch32? IMO, >> the announcement is vague on that. >> >> (Personally I would support the idea of having both projects under a >> common umbrella. But by now arch32 has their own support >> infrastructure, including forums). > > Well, I doubt they wanted to be caught by surprise and have nothing > ready if we decided not to allow support requests for arch32... > > But if we are willing to allow arch32 to be hosted under our umbrella, > the presence of separate infrastructure should not IMHO cause us to go > back on that and therefore cause additional fragmentation that we were > initially okay with avoiding. > In all my time here, I can remember one i686 bug that did not also affect x86_64. That suggests a common infrastructure is warranted. A
Re: [arch-dev-public] switching to systemd-stable
On 06/07/17 17:44, NicoHood wrote: > ArchLinux At least spell the name of the distro correctly. It is the simple addition of one space character between "Arch" and "Linux". But I see it is easy for people to forget about silly one character issues. A
Re: [arch-dev-public] Changing compilation flags
On 02/07/17 06:51, Bartłomiej Piotrowski wrote: > On 2017-06-30 23:44, Allan McRae wrote: >> On 30/06/17 19:07, Bartłomiej Piotrowski wrote: >>> On 2016-10-24 05:56, Allan McRae wrote: >>>> 1) building gcc to enable PIE by default >>> >>> I am in the middle of rebuilding gcc with --enable-default-pie. When it >>> finishes, I will start a todo for rebuilding packages with static libraries. >>> >>> I also enabled --enable-default-ssp, which means that >>> -fstack-protector-strong will be dropped from our CFLAGS (as it will be >>> enforced by gcc) on the next opportunity. >>> >> >> Are you adding full RELRO + no-plt at the same time? >> >> A >> > > Yes, and -fstack-check=specific too, although I might drop no-plt if it > will cause too many builders. > I thought the conclusion from the Stack Clash bugs was that the current -fstack-check was fundamentally flawed and was being completely rewritten for the next gcc. Is the "=specific" version OK?
Re: [arch-dev-public] Changing compilation flags
On 30/06/17 19:07, Bartłomiej Piotrowski wrote: > On 2016-10-24 05:56, Allan McRae wrote: >> 1) building gcc to enable PIE by default > > I am in the middle of rebuilding gcc with --enable-default-pie. When it > finishes, I will start a todo for rebuilding packages with static libraries. > > I also enabled --enable-default-ssp, which means that > -fstack-protector-strong will be dropped from our CFLAGS (as it will be > enforced by gcc) on the next opportunity. > Are you adding full RELRO + no-plt at the same time? A
Re: [arch-dev-public] libalpm hook and wrapper for detecting broken perl modules
On 11/06/17 08:03, Florian Pritz via arch-dev-public wrote: > One possibility to counter this would be to replace the perl executable > with a wrapper that performs the check and prints errors to stderr. > Optionally it could also exit with an error code rather than continuing > to run the script. I'm not sure if I want this or not, but I'm leaning > towards a yes since exiting will make the error more difficult to miss. I am very against this part (hook is completely fine). Having the perl executable be something other than upstream (and every other distro) provides, is not what Arch is about. I'm assuming the new location would be transparent to packagers (i.e. they would not need to alter their PKGBUILDs)? A
Re: [arch-dev-public] OpenSSL 1.1.0
On 23/04/17 08:07, Gaetan Bisson wrote: > [2017-04-22 18:05:27 +0200] Sébastien Luttringer: >> When do you plan to move openssl rebuild out of testing? > > Quoting arojas on IRC: > > 2017-04-20 09:11:27 arojas: current blocker for openssl if FS#53618 > 2017-04-20 09:11:47 arojas: someone needs to decide whether we care about it > or not, and if yes do something to fix it > Given there is a workaround, a news item should be posted and we should stop blocking the entire distribution with this rebuild. Allan
Re: [arch-dev-public] Resigning from package maintenance
On 12/04/17 17:28, Bartłomiej Piotrowski wrote: > On 2017-04-12 03:07, Allan McRae wrote: >> Dealing with packaging has become a chore, so I will not be doing that >> for Arch anymore. I will continue maintaining the pacman codebase. >> >> Here is a list of packages up for adoption: >> >> Core [15]: >> autoconf >> automake >> binutils >> bison >> flex >> gcc (gcc-ada, gcc-fortran, gcc-go, gcc-libs, gcc-objc) >> glibc >> gmp >> libmpc >> libtool >> linux-api-headers >> m4 >> make >> mpfr >> pkg-config >> >> Extra [3]: >> dejagnu >> expect >> fakechroot >> >> Allan >> > > > Adopted gcc, glibc, gmp, pkg-config and packages from extra. I'm happy > to see someone co-maintain these with me. > If you have gcc and glibc, you need to take linux-api-headers and binutils. They are a set. A
[arch-dev-public] Resigning from package maintenance
Dealing with packaging has become a chore, so I will not be doing that for Arch anymore. I will continue maintaining the pacman codebase. Here is a list of packages up for adoption: Core [15]: autoconf automake binutils bison flex gcc (gcc-ada, gcc-fortran, gcc-go, gcc-libs, gcc-objc) glibc gmp libmpc libtool linux-api-headers m4 make mpfr pkg-config Extra [3]: dejagnu expect fakechroot Allan signature.asc Description: OpenPGP digital signature
Re: [arch-dev-public] Remaining TODO lists - Part 1: Packages with missing sources
On 26/03/17 11:51, Allan McRae wrote: > We have a number of TODO lists that have not been completed. > > The "Packages with missing sources" list [1] was created more than six > months ago (2016-08-09). These packages are either genuinely no longer > have an upstream source, or are just sitting unmaintained in our repos. > Either way, the should not be in our repos. > > I will remove any package still on that list in a week. > A bit over a week later, and there are four packages remaining: mangler mashup multibit ozerocdoff Last chance!
[arch-dev-public] Remaining TODO lists - Part 1: Packages with missing sources
We have a number of TODO lists that have not been completed. The "Packages with missing sources" list [1] was created more than six months ago (2016-08-09). These packages are either genuinely no longer have an upstream source, or are just sitting unmaintained in our repos. Either way, the should not be in our repos. I will remove any package still on that list in a week. Allan [1] https://www.archlinux.org/todo/packages-with-missing-sources/
Re: [arch-dev-public] [RFC] Deprecate ABS?
On 25/03/17 21:24, Bartłomiej Piotrowski wrote: > Hi all, > > Allan complained yesterday that ABS is apparently broken. Turns out that > most likely it is broken at least since 2016-07-07. > > The script received no changes since 2012-09-07 and the main advantage > it brings is an easy way to pull all PKGBUILDs. But we also provide > public Subversion and Git repositories. Initial svn checkout for > community and packages takes less than 3 minutes on my rather slow > connection (for European standards); obviously subsequent updates are > faster. Full git clone is slow, but shallow one takes ca. 10 seconds > (for both). > > If someone needs to checkout just one package, svn has sparse checkouts > and for git, Dave wrote asp[1]. This brings the question, what is the > point of keeping it running? > Can our server handle the load of svn/git checkouts? That was the main concern when we moved to SVN many years ago and the reason ABS was kept. I have been working on and off to include source packages into a pacman database to allow pacman to directly replace these. Allan
Re: [arch-dev-public] AUR ToS (aka making AUR user names public)
On 05/03/17 23:35, Lukas Fleischer wrote: > Hi, > > I was recently contacted by a Polish researcher asking for a list of AUR > account names. Please share this data. It looks like this person is doing genuine research into networks within open-source software communities. Providing easy access to already public data should be encouraged in all fields of research. Allan
Re: [arch-dev-public] Changing compilation flags
On 02/02/17 05:06, Bartłomiej Piotrowski wrote: > On 2016-12-24 08:39, Allan McRae wrote: >> If default PIE is wanted, someone other than me will need to figure this >> out. Just build gcc with --enable-default-pie and then build binutils >> to see the issues in the testsuite. >> >> Allan >> > > Default PIE is not everything you planned to implement. What about the rest? > Can't be fucked doing this twice. A
Re: [arch-dev-public] OpenSSL 1.1.0
On 30/01/17 08:30, Giancarlo Razzolini wrote: > Em janeiro 29, 2017 20:04 Doug Newgard escreveu: >> >> I haven't heard all that much from/about LibreSSL since shortly after >> the fork. >> Care to share what advantages it would bring, and at what cost? >> > > The cost for rebuilding everything against OpenSSL 1.1 will probably be > a big one. > For LibreSSL, it would be even bigger. I think the main advantage, right > away, is > that LibreSSL has a considerably better security track, specially after > their huge > flensing. > > I can only dream about the bugs that might lurk on both OpenSSL 1.1 and > LibreSSL. > But the defensive approach OpenBSD takes on LibreSSL already has paid > off in terms > of CVE's that didn't affected it, but were high/critical issues on OpenSSL. > Please cite one example. Every CVE I have seen that is of at least high severity has affected both. There have been some low severity ones only affecting openssl. Even worse, the fix time for libressl in the couple of issues I monitored was worse than openssl. A
Re: [arch-dev-public] News draft for i686 deprecation
On 24/01/17 07:09, Bartłomiej Piotrowski wrote: > What I infer from my conversation with Allan is that > neither of us is interested in leading this personally. FYI, I will provide advise/mentorship to any team that forms to continue this, but definitely can not take on leading it. A
Re: [arch-dev-public] [RFC] Send signoff reports somewhere else (or not at all)
On 17/01/17 03:50, Giancarlo Razzolini wrote: > Em janeiro 3, 2017 6:45 Giancarlo Razzolini escreveu: >> Em janeiro 3, 2017 6:29 Allan McRae escreveu: >>>> >>> >>> I'm all for making declaration by lack of objection, but you should wait >>> more than 16 hours. Particularly at this time of year... >>> > > Can I now declare signoff reports dead, by lack of objection? > Yes
Re: [arch-dev-public] Shadowing i686, round 1
On 06/01/17 07:08, Bartłomiej Piotrowski wrote: > Friendly reminder what I'm going to push through if nobody opposes. Can we have a detailed discussion what EOL for i686 means? I have seen a couple of proposals being: 1) We just kill i686 (as in stop distributing packages), or 2) Reduce i686 to a secondary architecture. I want to see secondary architectures happen, so this would be a good start in that direction. We start by advertising for a team that is responsible for building and fixing i686 packages. At a certain point, the only change to our tooling is that developers upload x86_64 only, and the other team plays catchup (which should lead to autobuilding...) Allan
Re: [arch-dev-public] [RFC] Send signoff reports somewhere else (or not at all)
On 03/01/17 18:11, Giancarlo Razzolini wrote: > Em janeiro 3, 2017 0:33 Sébastien Luttringer escreveu: >>> >> Please drop them. >> >> Cheers, >> > > By the action of inaction, they will not be sent anymore. > I'm all for making declaration by lack of objection, but you should wait more than 16 hours. Particularly at this time of year... A
Re: [arch-dev-public] FrOSCon 2017
On 02/01/17 02:24, Bartłomiej Piotrowski wrote: > Bonn is about 8 hours away by train from me so I'm not really eager to > travel that long. You think that's bad...
Re: [arch-dev-public] Changing compilation flags
On 13/12/16 19:07, Allan McRae wrote: > On 24/10/16 13:56, Allan McRae wrote: >> That means we will add all of these to our default CFLAGS/LDFLAGS etc. >> The changes are: >> >> 1) building gcc to enable PIE by default >> 2) add -z,now to LDFLAGS >> 3) and -fno-plt and -fstack-check to our CFLAGS > > For those wondering what happened with this: > > binutils build with gcc with default PIE: > > # grep Error binutils-git-89080.bfbf34d-1-x86_64-check.log > make[5]: *** [Makefile:7161: incremental_test_2] Error 1 > make[5]: *** [Makefile:7185: incremental_test_5] Error 1 > make[5]: *** [Makefile:7200: incremental_copy_test] Error 1 > make[5]: *** [Makefile:7206: incremental_common_test_1] Error 1 > make[5]: *** [Makefile:7132: ehdr_start_test_4] Error 1 > readelf: Error: the PHDR segment is not covered by a LOAD segment > make[4]: *** [Makefile:5609: check-am] Error 2 > make[3]: *** [Makefile:5613: check] Error 2 > make[2]: *** [Makefile:941: check-recursive] Error 1 > make[1]: *** [Makefile:6134: check-gold] Error 2 > make[5]: *** [Makefile:3678: check-DEJAGNU] Error 1 > make[4]: *** [Makefile:1953: check-am] Error 2 > make[3]: *** [Makefile:1793: check-recursive] Error 1 > make[2]: *** [Makefile:1955: check] Error 2 > make[1]: *** [Makefile:7547: check-ld] Error 2 > make: *** [Makefile:2206: do-check] Error 2 > > > binutils build with current gcc: > > # grep Error binutils-git-89080.bfbf34d-1-x86_64-check.log > readelf: Error: the PHDR segment is not covered by a LOAD segment > > > I need time to fix this. It is probably just test suite assumptions > rather than errors. I'm putting these changes on a permanent hold due to a lack of time on my behalf. I'm not putting a binutils build with this many testsuite failures into the repos until I fully understand why this is happening. If default PIE is wanted, someone other than me will need to figure this out. Just build gcc with --enable-default-pie and then build binutils to see the issues in the testsuite. Allan
Re: [arch-dev-public] Shadowing i686, round 1
On 14/12/16 08:05, NicoHood wrote: > On 12/13/2016 08:04 PM, Balló György via arch-dev-public wrote: >> 2016. 12. 13, kedd keltezéssel 12.29-kor Doug Newgard ezt írta: >>> On Tue, 13 Dec 2016 19:16:53 +0100 >>> Balló György via arch-dev-public >>> wrote: >>> -1 for dropping i686 completely. +1 for introducing automated builds, even if it's less secure. I would like to keep i686 supported, and willing to do anything that is needed to setup and maintain an official automated build server for i686 packages (and possibly for other architectures). >>> >>> I've got to ask, why do you feel so strongly about it? It's been >>> pointed out >>> that i686 really doesn't fit in with the original goals of Arch >>> anymore, is less >>> and less supported upstream, and essentially untested. What is the >>> compelling >>> reason for keeping it around? >> >> Because I still use an i686-only system occasionally, and I prefer to >> keep old hardware working with my favourite distribution. I agree that >> building packages manually for a small percentage of users is >> pointless. But most of the packages can be built for i686 without any >> modifications. We just need an automated build server, which takes the >> job. >> >> -- >> György Balló >> Trusted User >> > > If we have reproduceable builds we could also use this buildserver to > build the x64 packages. I know that this is a huge task, but then we > could automate the package building better in a centralized place. And > instead of dropping i686 we could integrate arm as well. > > Those will not be officially supported, but we could give people access > to fix those arch specific problems. Normal maintainers can focus on x64 > development while some others have a place to distribute and maintain > other architectures of their favorite os. > If we had a build server, I'd prefer we concentrated on Arch's original goals. To provide a more optimised distribution than everyone else. All other distros have caught up to us these days. Some more optimised x86_64 variants would be good... A
Re: [arch-dev-public] Shadowing i686, round 1
On 13/12/16 23:15, Florian Pritz via arch-dev-public wrote: > I still have two media players that run i686, but I can switch those to > a different distro if necessary although I'd obviously prefer not having > to do that. Count that as a -0.25. I have one webserver running i686. My vote counts as -1 :) A
Re: [arch-dev-public] Changing compilation flags
On 13/12/16 19:23, Jan Alexander Steffens via arch-dev-public wrote: > On Mon, Oct 24, 2016 at 5:56 AM Allan McRae wrote: > >> Hi all, >> >> The results from the test-sec-flags [1] suite are in. Many thanks to >> those that wrote this and those that submitted results. I'm not going >> to list the summaries here, but the results show that at worst enabling >> a bunch of security flags in our packages will have a <1% impact on >> performance (and more likely a fraction of a percent). >> >> That means we will add all of these to our default CFLAGS/LDFLAGS etc. >> The changes are: >> >> 1) building gcc to enable PIE by default >> 2) add -z,now to LDFLAGS >> 3) and -fno-plt and -fstack-check to our CFLAGS >> >> Adding PIE means that programs get loaded at a random address, >> preventing an attacker from manipulating global data or hijacking >> control by reusing code. Without this, ASLR is ineffective. Enabling >> this by default in our compiler (there is a configure flag) means we >> will need to rebuild all packages with static libraries (which should be >> a fairly limited set). >> >> Adding -z,now to LDFLAGS disables lazy loading. This means all function >> symbols are loaded at startup (with minor performance hit), but that >> means our RELRO support will make everything ro instead of some of the >> things. Doing this enables us to use -fno-plt in our CFLAGS, which is a >> run-time optimisation allowing faster use of libraries. >> >> Adding -fstack-check to our CFLAGS guarantees stack overflows aren't >> exploitable. >> >> Note that any of these flags can be disabled in a PKGBUILD if really >> needed... But if that is the case, bug reports should be filed. >> >> >> Given an assumed lack of objection, I will enable the build flags in our >> pacman.conf and rebuild gcc to enable pie and put them in [staging] at >> the end of this week (what better way to celebrate Halloween). We will >> need a new devtools release then too. Then the packages with static >> libraries will need rebuilt. >> >> After that, I would like to see [core] completely rebuilt, and audited >> to ensure our CFLAGS/LDFLAGS are actually being used in the build. >> >> Cheers, >> Allan >> > > Will this affect i686 as well? According to this commit ( > https://github.com/zen-kernel/zen-kernel/commit/cc701dc61a7187e9dfc300ad6ec3b7a677dc4717) > at least Ubuntu seems to have skipped that for now. That commit shows they disabled it for one package. It will affect both. A
Re: [arch-dev-public] Shadowing i686, round 1
On 13/12/16 19:20, Bartłomiej Piotrowski wrote: > On 2016-12-12 22:02, Jelle van der Waa wrote: >> I'm in favor of an auto build solution, since this has multiple >> bonusses. We could extend the auto build solution for reproducible >> builds (yay!). Auto rebuilds and maybe later when vendors get their act >> togehter (aarch64 *cough*). > > Creating a new or adapting existing solutions is time consuming. We > struggle to finish and switch to VCS-agnostic dbscripts for a long time > and I wouldn't expect that any automated build server will take less > time, not to mention security concerns expressed in previous > discussions. On the opposite, dropping support is effortless. Given we can't get databases signed, I can't ever see an autobuild system working. A
Re: [arch-dev-public] Changing compilation flags
On 13/12/16 19:10, Daniel Micay via arch-dev-public wrote: >> I need time to fix this. It is probably just test suite assumptions >> rather than errors. > > Could we start with the changes other than PIE? Those won't need any > immediate rebuilds. I'm not dealing with this twice.
Re: [arch-dev-public] Changing compilation flags
On 24/10/16 13:56, Allan McRae wrote: > That means we will add all of these to our default CFLAGS/LDFLAGS etc. > The changes are: > > 1) building gcc to enable PIE by default > 2) add -z,now to LDFLAGS > 3) and -fno-plt and -fstack-check to our CFLAGS For those wondering what happened with this: binutils build with gcc with default PIE: # grep Error binutils-git-89080.bfbf34d-1-x86_64-check.log make[5]: *** [Makefile:7161: incremental_test_2] Error 1 make[5]: *** [Makefile:7185: incremental_test_5] Error 1 make[5]: *** [Makefile:7200: incremental_copy_test] Error 1 make[5]: *** [Makefile:7206: incremental_common_test_1] Error 1 make[5]: *** [Makefile:7132: ehdr_start_test_4] Error 1 readelf: Error: the PHDR segment is not covered by a LOAD segment make[4]: *** [Makefile:5609: check-am] Error 2 make[3]: *** [Makefile:5613: check] Error 2 make[2]: *** [Makefile:941: check-recursive] Error 1 make[1]: *** [Makefile:6134: check-gold] Error 2 make[5]: *** [Makefile:3678: check-DEJAGNU] Error 1 make[4]: *** [Makefile:1953: check-am] Error 2 make[3]: *** [Makefile:1793: check-recursive] Error 1 make[2]: *** [Makefile:1955: check] Error 2 make[1]: *** [Makefile:7547: check-ld] Error 2 make: *** [Makefile:2206: do-check] Error 2 binutils build with current gcc: # grep Error binutils-git-89080.bfbf34d-1-x86_64-check.log readelf: Error: the PHDR segment is not covered by a LOAD segment I need time to fix this. It is probably just test suite assumptions rather than errors. A
[arch-dev-public] Changing compilation flags
Hi all, The results from the test-sec-flags [1] suite are in. Many thanks to those that wrote this and those that submitted results. I'm not going to list the summaries here, but the results show that at worst enabling a bunch of security flags in our packages will have a <1% impact on performance (and more likely a fraction of a percent). That means we will add all of these to our default CFLAGS/LDFLAGS etc. The changes are: 1) building gcc to enable PIE by default 2) add -z,now to LDFLAGS 3) and -fno-plt and -fstack-check to our CFLAGS Adding PIE means that programs get loaded at a random address, preventing an attacker from manipulating global data or hijacking control by reusing code. Without this, ASLR is ineffective. Enabling this by default in our compiler (there is a configure flag) means we will need to rebuild all packages with static libraries (which should be a fairly limited set). Adding -z,now to LDFLAGS disables lazy loading. This means all function symbols are loaded at startup (with minor performance hit), but that means our RELRO support will make everything ro instead of some of the things. Doing this enables us to use -fno-plt in our CFLAGS, which is a run-time optimisation allowing faster use of libraries. Adding -fstack-check to our CFLAGS guarantees stack overflows aren't exploitable. Note that any of these flags can be disabled in a PKGBUILD if really needed... But if that is the case, bug reports should be filed. Given an assumed lack of objection, I will enable the build flags in our pacman.conf and rebuild gcc to enable pie and put them in [staging] at the end of this week (what better way to celebrate Halloween). We will need a new devtools release then too. Then the packages with static libraries will need rebuilt. After that, I would like to see [core] completely rebuilt, and audited to ensure our CFLAGS/LDFLAGS are actually being used in the build. Cheers, Allan [1] https://www.archlinux.org/news/test-sec-flags-call-for-assistance/
Re: [arch-dev-public] Long out of date packages
On 20/10/16 22:25, Florian Pritz via arch-dev-public wrote: > > allan: > extra/x86_64/fakechroot Updating this breaks the pacman testsuite (due to a fakechroot bug...).
Re: [arch-dev-public] i686 and SSE2
Just to summarise what I have taken from discussion surrounding this so it does not stagnate: 1) Changing i686 to either i686 + SSE or i786 is not seen to be a good idea 2) There are suggestions we look into providing something more optimised than x86_64, but it not clear what platform(s) people are looking to support. 3) Autobuilding more the optimised platforms would be great, but runs into the same issue we have with database signing and security of such a key. The most pressing issue is #1. So lets get back to the issue of the increasing amount of software requiring SSE2. How do we solve this? It would be "simple" to add an SSE2 detection hook into the filesystem package that aborts any pacman transaction attempting to install a list of packages on i686 systems without SSE2 support. Is that a viable solution? Any other suggestions? Allan
Re: [arch-dev-public] i686 and SSE2
On 20/09/16 04:30, Ike Devolder via arch-dev-public wrote: > Someone mentioned AVX2, but it seems there is still a patch needed for > glibc. Solus added it very recently https://dev.solus-project.com/T503 > and they have taken it from Clear Linux. Nothing needs done for glibc to use AVX. It already does when present. Allan
Re: [arch-dev-public] i686 and SSE2
On 19/09/16 23:14, Balló György via arch-dev-public wrote: > 2016-09-19 7:02 GMT+02:00 Allan McRae : > >> This goes beyond just adding SSE2 support. >> >> Years ago, Arch Linux was "optimised for modern processors". These were >> the days when every other distro was using i386 and we had a blazingly >> fast i686 port. Now every other distro uses i686 while we have sat >> still. Even major software developments are starting to require SSE2. >> It is time we moved forward. >> >> How can we achieve this? I see several options: >> >> 1) Do "nothing". Add a hook to the filesystem package that detects >> whether a system has SSE2 support and blocks installation of certain >> packages. >> >> 2) Add SSE2 to our optimisations and require "i686 + SSE2" >> >> 3) Move our minimum CPU to something less than 20 years old (even i786 >> would get us SSE2+3 instructions and is 15 years old) >> >> 4) We add more modern CPU builds (and set them automatically building >> once the base architecture is updated). >> >> >> I am in favour of #3 for our 32-bit support. And that would be end of >> line as far as 32 bit support in this distribution goes. >> >> >> (We may want to consider #4 for our x86_64, but that is another >> conversation). >> >> Allan >> > > > I would not be happy with #3, because I still have two 13 years old systems > with NetBurst-based CPUs without SSE3 support. But of course I don't use > them in everyday use. > If we limit our choice based on your CPU, then we need to limit based on the other CPU mentioned in this thread. That should not be a consideration at all. What we need to do is think about what make our distribution worthy of being a distribution. Original the selling points were rolling release, vanilla packages and optimised binaries. We have lost the latter. Do we want to get it back? Allan
Re: [arch-dev-public] i686 and SSE2
On 19/09/16 19:05, Christian Hesse wrote: > Bartłomiej Piotrowski on Fri, 2016/09/16 21:44: >> Actually, why don't raise the bar higher? SSE2 has been introduced in >> 2001 – that's 15 years to upgrade one's hardware and given my sad >> experiences with computers, I find it hard to believe anyone has that >> old PC that happens to run Arch. > > I do. Running Arch Linux on a bunch of informational room displays... > These are based on Geode CPUs and I am pretty sure no SSE2 is available. > (Taking a look at /proc/cpuinfo these devices do not even feature SSE... The > CPU identifies itself as i586.) > > That said I will be able to handle that myself. ;) > Possibly I will have to do local rebuilds of webkitgtk2 to make surf run on > these devices from time to time. If we adopt SSE2 in any form, you will need to rebuild your entire system to use it. It will be used in optimizations everywhere. A
Re: [arch-dev-public] i686 and SSE2
This goes beyond just adding SSE2 support. Years ago, Arch Linux was "optimised for modern processors". These were the days when every other distro was using i386 and we had a blazingly fast i686 port. Now every other distro uses i686 while we have sat still. Even major software developments are starting to require SSE2. It is time we moved forward. How can we achieve this? I see several options: 1) Do "nothing". Add a hook to the filesystem package that detects whether a system has SSE2 support and blocks installation of certain packages. 2) Add SSE2 to our optimisations and require "i686 + SSE2" 3) Move our minimum CPU to something less than 20 years old (even i786 would get us SSE2+3 instructions and is 15 years old) 4) We add more modern CPU builds (and set them automatically building once the base architecture is updated). I am in favour of #3 for our 32-bit support. And that would be end of line as far as 32 bit support in this distribution goes. (We may want to consider #4 for our x86_64, but that is another conversation). Allan
Re: [arch-dev-public] Long out of date packages
On 18/08/16 05:52, Christian Hesse wrote: > Unflag the package, then flag it yourself with a comment of the details. At > least devs can find the information there. As a "bonus", unflagged then reflagging makes the package look as if it has only been out-of-date for 1 day...
[arch-dev-public] Discussion about optional dependencies
Hi all, We need to have a discussion about optional dependencies. I am going to use virtualbox as an example, because of the 10 bugs and counting caused by people not seeing the change in dependency to run the gui. However, this is just the latest in the list of such issues. So lets look at the optional dependencies: Optional Deps : qt5-x11extras: GUI support vde2: Virtual Distributed Ethernet support virtualbox-guest-iso: Guest Additions CD image virtualbox-ext-vnc: VNC server support virtualbox-sdk: Developer kit net-tools: Host-only or bridged networking support The optional dependency of note here is "qt5-x11extras: GUI support"., which is needed to run /usr/bin/virtualbox. That is the primary "binary" of the virtualbox package. My opinion is the primary binary for a package should run out of the box. If you really need to reduce dependencies, then a split package should be considered. Comments/opinions? Thanks, Allan
Re: [arch-dev-public] signoffs are dead
On 02/07/16 06:18, Christian Hesse wrote: > Gaetan Bisson on Tue, 2016/06/28 08:09: >> For a while now packages in [testing] have gotten little to no signoffs >> and I've been moving mine to [core] after a week without feedback. I >> suspect many of you have been doing this too. > > Yes, probably everybody does. ;) > > I do not run a system with [testing] enabled by default. I need my main > system in production every day - stable and reliable. > Packages that I really do want and/or need end up in my personal repository > for testing, though. I do give spare signoffs, but that packages received > real testing then. > > Possibly we should modify the process a bit: Expect a package to be fine > when it received enough signoffs - as is. Additionally add a NACK feature that > expresses something is (possibly) borked. If the package did not receive a > NACK within 48 or 72 hours it is expected to be fine as well. > Our bug wrangler Doug could have an eye on bugs that look critical and set > NACKs for the packages. This sounds like the Fedora policy where packages have to surpass a certain karma level to move into the main repositories. I'm not sure who gets to vote for that though. A
Re: [arch-dev-public] test-sec-flags announcement
On 27/05/16 07:19, Lukas Jirkovsky wrote: > On 25 May 2016 at 03:34, Allan McRae wrote: >> Hi all, >> >> There has been a group of people doing some analysis on the performance >> impact of various security flags we can use in our packaging. They are >> wanting to gather performance benchmarks from a bunch of people so we >> can have an informed discussion about their inclusion. See below. > > I'm missing one important piece of information, and that is what to do > with the results. There are some results on the github wiki, but > there's no obvious point where to submit the results. > >From the README: "Please add your results to the wiki as well, preferably maintaining the same format as the results already there."
[arch-dev-public] test-sec-flags announcement
Hi all, There has been a group of people doing some analysis on the performance impact of various security flags we can use in our packaging. They are wanting to gather performance benchmarks from a bunch of people so we can have an informed discussion about their inclusion. See below. I'll wait and see how many responses they get from this post before I post this as a news item too. Cheers, Allan --ANNOUNCEMENT-- I am thrilled to announce that test-sec-flags has been completed and is ready for download and use. What is test-sec-flags? Inspired by discussions on arch-general, test-sec-flags was created to test the performance impact of various security-oriented compilation and linking flags. The goal is to determine if these flags can be the new default for all Arch Linux packages. Preliminary results suggest that the performance impact is almost nonexistent, but I would like to collect and compare more results before moving forwards. How do I use it? See the README for installation and usage instructions.The results subdirectory contains instructions on how to pull out the relevant statistics from the result files. Where do I obtain it? https://github.com/pid1/test-sec-flags My sincerest thanks to anthraxx, strcat, sangy, and rgacogne for their invaluable assistance. Patches welcome. Cheers, pid1
[arch-dev-public] Community Etiquette document
Hi all, There has been some talk about extending the Forum Etiquette document into a combined community one. There is a draft at: https://wiki.archlinux.org/index.php/User:Jasonwryan/Community_Etiquette Please give opinions (preferably on the Talk page). Allan
Re: [arch-dev-public] Linux 4.6 enters [testing]
On 17/05/16 04:58, Tobias Powalowski via arch-dev-public wrote: > Hi folks, > > New kernel is in [testing], > > Binary modules which are not building: > > -nvidia series (all of them) > > -tp_smapi > This package should have been pushed to [staging] if the module rebuild was not complete. A
[arch-dev-public] Changes to x86_64 triplet
Hi all, gcc-6 decided to enforce the vendor part of the target triplet to be not "unknown". This means our vendor string is changed from "unknown" to "pc", aligning with the i686 triplet. I have release a new pacman with the triplet changed in pacman.conf: CHOST="x86_64-pc-linux-gnu" This _should_ have very little effect (the vendor string is quite unimportant). Allan
Re: [arch-dev-public] Hooks rebuild #1
On 28/04/16 21:47, Antonio Rojas wrote: > Allan McRae wrote: >> >> You have obviously figured it out so who cares... > > Yeah but there could be more packages affected > That can be dealt with once the rebuild is done. Then it will be super easy to find missed packages in ABS.
Re: [arch-dev-public] Hooks rebuild #1
On 28/04/16 21:28, Antonio Rojas wrote: > Allan McRae wrote: > >> On 28/04/16 21:18, Antonio Rojas wrote: >>> Allan McRae wrote: >>> >>>> No idea... I generated the list via a quick grep, so there are false >>>> positives (more than I expected...). Just mark them as done. >>>> >>>> A >>> >>> It seems that it also didn't catch install files from split packages, eg. >>> kdepim or kdewebdev are missing from the todo list. >>> >> >> Is the base package there? > > No, neither the base package nor any of its subpackages. > Hrm... it was on the list I uploaded. $ grep kdepim rebuild.txt kdepim kdepimlibs kdepimlibs4 kdepim-runtime You have obviously figured it out so who cares... A
Re: [arch-dev-public] Hooks rebuild #1
On 28/04/16 21:18, Antonio Rojas wrote: > Allan McRae wrote: > >> No idea... I generated the list via a quick grep, so there are false >> positives (more than I expected...). Just mark them as done. >> >> A > > It seems that it also didn't catch install files from split packages, eg. > kdepim or kdewebdev are missing from the todo list. > Is the base package there?
Re: [arch-dev-public] Hooks rebuild #1
On 28/04/16 04:40, Anatol Pomozov wrote: > Hi > > And another related question: are we going to have hooks for > systemd-sysusers and systemd-tmpfiles? > > Quite a lot of packages packages use systemd-tmpfiles and it sounds > like a good idea to move it to hooks. > They are listed on the wiki page [1] and this rebuild was labelled as "#1", so you could concluded there will be another rebuild... [1] https://wiki.archlinux.org/index.php/DeveloperWiki:Pacman_Hooks
Re: [arch-dev-public] Hooks rebuild #1
On 28/04/16 02:59, Pierre Schmitz wrote: > On 27.04.2016 14:36, Allan McRae wrote: >> We are ready to start the first hooks rebuild. This rebuild covers >> packages using these hooks: >> >> update-desktop-database >> update-mime-database >> install-info >> glib-compile-schemes >> gtk-update-icon-cacne/xdg-icon-resource >> gconf >> gio-querymodules >> >> Each rebuild requires the install file updated to remove these commands. >> >> No need to use staging/testing for this rebuild (except [core] packages). >> >> GO! > > Why is lib32-openssl on that list? > No idea... I generated the list via a quick grep, so there are false positives (more than I expected...). Just mark them as done. A
[arch-dev-public] Hooks rebuild #1
We are ready to start the first hooks rebuild. This rebuild covers packages using these hooks: update-desktop-database update-mime-database install-info glib-compile-schemes gtk-update-icon-cacne/xdg-icon-resource gconf gio-querymodules Each rebuild requires the install file updated to remove these commands. No need to use staging/testing for this rebuild (except [core] packages). GO!
[arch-dev-public] Hooks!
According to the news announcement, today is the day we can start using hooks! (as long as you live in the future like me - those of you in the past will need to catch up a day). I have started a wiki page to discuss which hooks to add [1]. So far this includes "update-desktop-database", "update-mime-database", and "info-install" hooks to be added. I also included a list of potential other hooks. I have no idea about these or how to create hooks for them, so it is up to other people. I'd like to get as many hooks as possible into our packages in one go and then create rebuild lists for them. This way, if a package can benefit from multiple hooks, it will only need rebuilt once. Please take a look at that page and add any hook that you think will be useful to your packages. I will dump the first three into packages in the next couple of days and make rebuild lists. These will not got to [staging]/[testing] as there is no harm having the hook do some duplicated effort until all the packages are rebuilt. Cheers, Allan [1] https://wiki.archlinux.org/index.php/DeveloperWiki:Pacman_Hooks
Re: [arch-dev-public] Consensus: DKMS modules
On 14/04/16 19:23, Ike Devolder wrote: > - use separate repo [kernel-update{-testing,-staging}] Why? We have staging for rebuilds like these.