Re: [arch-dev-public] Autumn extra cleaning

[Anatol Pomozov via arch-dev-public]

Hi

On Sun, Oct 4, 2020 at 10:16 PM Sven-Hendrik Haase via arch-dev-public
 wrote:
>
> Hey everyone,
>
> It was suggested as part of this year's spring cleanup of [community]
> that we should be have a cleanup in [core]/[extra] and move packages
> downwards into [community].
>
> This round only concerns [extra] packages.
>
> Devs that want the packages in [extra], please adopt packages, and TUs
> can notify which packages they are interested to maintain in [community]
>
> Orphaned packages in [extra]:
>
> a2ps
> aalib
> abook
> adwaita-icon-theme (Jan can you just take this?)
> antlr2
> antlr4
> arch-install-scripts
> asp
> aspell-es
> bzflag
> c-ares
> chromaprint
> cln
> cmark
> convertlit
> cvs
> cvsps
> davfs2
> dcraw
> enca
> expac
> fakechroot
> farstream
> fastjar
> font-bh-ttf
> freetds
> fvwm
> geeqie
> giblib
> gifsicle
> gnugo
> gperftools
> graphene
> gsfonts
> gstreamer
> gtksourceview4
> gts
> gupnp-igd
> gv
> hddtemp
> hunspell-fr
> hunspell-it
> hunspell-ro
> hyphen-fr
> hyphen-it
> icewm
> id3lib
> ifplugd
> ispell
> java-activation-gnu
> java-bcel
> java-commons-net1
> java-gnumail
> java-hamcrest
> java-inetlib
> java-jdepend
> java-jline
> java-jsch
> java-resolver
> java-rhino
> joe
> jpegexiforient
> junit
> leveldb
> libatomic_ops
> libkate
> liblqr
> libmilter
> libmng
> libnet
> libnice
> libnma
> libofa
> libots
> libpano13
> libspiro
> libstroke
> libtiger
> libtommath
> libuninameslist
> libusb-compat
> lua51
> lua52
> m17n-db
> mercurial
> mtools
> munin
> munin-node
> mysql-python
> mythes-fr
> mythes-it
> mythes-ro
> nawk
> netpbm
> npapi-sdk
> nss-mdns
> ocamlbuild
> opencore-amr
> perl-http-daemon
> pkgfile (I suggest we just drop this in favor of pacman -F)
> potrace
> psiconv
> python2-cairo
> python2-numpy
> python2-ordered-set
> python2-setuptools
> rcs
> rhino
> rhino-javadoc
> screen
> shared-color-targets
> snappy
> snarf
> swt
> tcl
> telepathy-farstream
> telepathy-idle
> telepathy-salut
> time
> tinycdb
> tk
> ttf-junicode
> ttf-linux-libertine
> uim
> usbmuxd
> vcdimager
> vim-spell (and its 50 split packages)
> w3c-mathml2
> x11-ssh-askpass
> x11vnc
> xalan-java
> xerces2-java
> xine-lib
> xine-ui
> xmltoman
> xorg-font-util
> xorg-xfontsel
> xorg-xlsfonts
> xournalpp
> yajl


I'll adopt libatomic_ops and snappy - it used by some of my packages.

Re: [arch-dev-public] Autumn extra cleaning

[Anatol Pomozov via arch-dev-public]

Hi

On Sun, Oct 4, 2020 at 11:11 PM Daurnimator via arch-dev-public
 wrote:
>
> On Mon, 5 Oct 2020 at 16:16, Sven-Hendrik Haase via arch-dev-public
>  wrote:
> > TUs can notify which packages they are interested to maintain in [community]
>
> > lua51
> > lua52
>
> Sure (though no upstream updates are expected)

If you plan to take care of lua packages, would you mind adopting the
"lua" package as well? I do not do much lua development and I would
prefer if someone else handled the lua ecosystem.

[arch-dev-public] Anatol is inactive due to situation in Belarus

[Anatol Pomozov via arch-dev-public]

Hello folks

My name is Anatol Pomozov and I am an Arch Linux developer. I've been
quiet for a couple of weeks now and will probably stay away from my
Arch Linux/pacman (+bcc:pacman-dev@) duties for a bit more. Please
feel free to take care of my packages if it's needed.

The reason for this decision is a situation that happens in my home
country. I am a belarusian. My home country, my beautiful Belarus is
essentially taken hostage by a usurpator Lukashenko. He has been
"ruling" the country for 26 years already. Lukashenko cancelled
results of the last election (where he took ~10% of votes) and
declared that he is going to stay in power for another 5 years. All
his political opponents were put into a jail or hiding from him
abroad. Anyone who demands fair elections is terrorized by his
personal army. Many people are tortured in jail.

You can see what happens in Belarus in these clips
https://www.youtube.com/watch?v=XG3D6881AXY
https://www.youtube.com/watch?v=-6tCGnM0xiQ
https://www.youtube.com/watch?v=E0VLjQvCxcc

I live abroad and I am safe now. But my parents and relatives are in
Belarus and they were essentially taken hostage by the Lukashenko's
government. The last few weeks were stressful for me and I want to
focus on my extended family as much as possible.

I understand that Arch maillist might not be the place for political
statements but this is something personal and something important for
me.

There is no place for such violence and there is no place for
dictators in Europe of the 21st century. Please support Belarusian
people in their fight against the usurpator. Talk to your
governor/mayor/deputy/..., ask them to contact Belarusian government
to stop the violence. Talk to the press, explain to them what is going
on in Belarus right now. Anything that brings peace to Belarus will be
helpful. Thank you.

Жыве Беларусь!

Re: [arch-dev-public] Use detached package signatures by default

[Anatol Pomozov via arch-dev-public]

Hi Giancarlo

On Tue, Jul 28, 2020 at 12:35 PM Giancarlo Razzolini
 wrote:
> This could be maintained as a patch on the package, it doesn't necessarily 
> have to be
> on pacman's code itself. Just so we make this transition as painless as 
> possible to users.

Having a seamless transition to the new technology is definitely a top
priority here.

> Can't we go with a different option here? Instead of an option the user sets
> on their end, we make pacman fallback to embedded db sigs, if there are no 
> detached
> *or* if the signature check fails for some reason.

The detached signatures are generated by makepkg toolset since a long
time ago. *.sig files are already in the Arch standard repository. I
also looked through a dozen of random repos at
https://wiki.archlinux.org/index.php/Unofficial_user_repositories and
all of them have *.sig files for the packages.

At this point we are trying to enable the detached signatures handling
at the client side while having a backup option to disable it.

Let me know about a specific situation when detached signatures cause an issue.

Re: [arch-dev-public] Use detached package signatures by default

[Anatol Pomozov via arch-dev-public]

Hi

On Wed, Jul 8, 2020 at 8:22 PM Allan McRae via arch-dev-public
 wrote:
>
> On 9/7/20 1:05 pm, Anatol Pomozov wrote:
> > Given this information I would like to propose to stop using embedded
> > signatures and move to detached signatures by default. This will
> > require pacman 6.x or as alternative backport the fix(es) to 5.x
> > branch. It will help to make system updates even faster, something
> > that me and many other Arch users really love.
>
> There are several steps we need to complete:
>
> 1) backport the patch (or wait for pacman-6.0, which may be a while
> yet).  I'll leave that to the distro packagers to decide!
>
> 2) adjust repo-add to optionally add signatures.
>
> 3) make a time line that all users need to have the patched/released
> pacman installed - we usually require at least 6 months.
>
> 4) turn off signature inclusion in repo dbs.

It sounds great. If we go this route for pacman 6.0 then it will take
about 1 year to switch to the detached signatures.

As it is quite an important change I would love to see its codepath
tested as much as possible before we remove the embedded signatures
from pacman database files. It will help to catch issues like
https://bugs.archlinux.org/task/67232.

What do you think about starting to use detached signatures by default
*and* having embedded signatures as a backup option for time being?
i.e. pacman database will have the signatures (the same as now) but it
will be ignored. Instead pacman will use the detached *.sig files. And
in case if there is a major issue with this implementation then a user
would be able to switch back to embedded signatures using a
pacman.conf option (e.g. "UseEmbeddedSignatures"). If folks are fine
with it I can implement a patch for it.

Re: [arch-dev-public] Use detached package signatures by default

[Anatol Pomozov via arch-dev-public]

Hi Jelle

On Thu, Jul 9, 2020 at 2:00 AM Jelle van der Waa  wrote:
>
> On 09/07/2020 05:05, Anatol Pomozov via arch-dev-public wrote:
> > TLDR; let’s start using detached package signatures to make system
> > updates faster.
> >
> > Hi folks,
> >
> > Some time ago there was a discussion at IRC where someone (Allan
> > maybe?) proposed to stop using embedded PGP signatures in favor of
> > detached signature files. I would like to bring this idea here and
> > quantify it with some numbers.
>
> The downside of not having the package signatures in the database is
> that consumers can not easily obtain this information. For archweb
> that's showing who signed the package on the package details page.
>
> How would I implement an efficient alternative without fetching package
> files or all the sig files? A separate sig database? :P

The best option is to download and parse the signature file directly.
Its filename is going to be .sig where 
is available in a package description as %FILENAME% entry.

> As far now I'll have to adjust the code not to break because of a
> missing PGPSIG entry.
>
> > Here is a bit of technical details on this topic. Pacman has the
> > ability to verify authenticity of package files with PGP signatures.
> > PGP signatures add protection against undesired package modifications
> > by a third-party and it improves security aspects of the package
> > management. This feature can be configured per repository and the
> > official Arch Linux repos have it enabled. Package signatures have
> > been used by Arch Linux successfully for a couple of years now.
>
> 
>
> > An alternative to embedded signatures are detached signatures. These
> > are signatures stored in a separate file next to the package itself
> > (in a .sig file to be specific). Instead of downloading *all*
> > signatures every time a database is updated, detached signatures are
> > downloaded only when a specific package is installed/updated. If Arch
> > could switch to this model then database files become 3 times smaller
> > that saves users bandwidth and system update time.
>
> It would be insightful to provide the database numbers, because one
> could argue 30% of 1MB is nothing, as 30% of 100M is nice improvement.
>
> Our biggest database should be community (5M atm), and with all the
> savings that would now be ~ 2 MB? Would be nice to have an overview of
> the real life numbers :)

For compressed "community" database the savings are going to be
5.2M -> 1.73M (gzip) or 1.26M (zstd -19). With other dbs I would
say that for an average user we are looking at 7M->2.2M total savings
in the database size.

Keep in mind that database downloading/parsing is located at the critical
path. Every user downloads these db files pretty much
every time "pacman -Sy" is run. Detached signatures make this step
faster by reducing the workload and downloading signatures on-demand later.

[arch-dev-public] Use detached package signatures by default

[Anatol Pomozov via arch-dev-public]

TLDR; let’s start using detached package signatures to make system
updates faster.

Hi folks,

Some time ago there was a discussion at IRC where someone (Allan
maybe?) proposed to stop using embedded PGP signatures in favor of
detached signature files. I would like to bring this idea here and
quantify it with some numbers.

Here is a bit of technical details on this topic. Pacman has the
ability to verify authenticity of package files with PGP signatures.
PGP signatures add protection against undesired package modifications
by a third-party and it improves security aspects of the package
management. This feature can be configured per repository and the
official Arch Linux repos have it enabled. Package signatures have
been used by Arch Linux successfully for a couple of years now.

Package signatures are stored as a part of a pacman database file (it
is called “embedded signatures”). One issue with embedded signatures
is that they represent a quite large chunk of database file. What is
worse, a PGP signature is high-entropy data and does not compress
well. I was mildly shocked to learn how much of the *.db files
signatures consume.

I ran experiments and repackaged extra, community databases without
PGP data. For uncompressed “extra” repository size drops to 83% of its
original size (though uncompressed size is not that interesting). Arch
uses GZIP compressed database and in this case removing signatures
reduces the “extra” database to 36.8% of its original size. To
emphasize it one more time - removing PGP signatures makes this repo
only 1/3 of its original size. The change is even more dramatic in
case of “zstd -19” compression where the final database file is only
31% of its original size.

For community.db the numbers are: uncompressed file gets 79.8% of its
original size, “gzip -9” gets 33.4%, and with “zstd -19” it gets
27.51% of its original size.

A database gets modified with every package update. Users need to
re-download the databases where 2/3 of it are package signatures that
are used only when a specific package is installed.

An alternative to embedded signatures are detached signatures. These
are signatures stored in a separate file next to the package itself
(in a .sig file to be specific). Instead of downloading *all*
signatures every time a database is updated, detached signatures are
downloaded only when a specific package is installed/updated. If Arch
could switch to this model then database files become 3 times smaller
that saves users bandwidth and system update time.

I looked through pacman code and most components have detached
signatures support already. Most of the places have a logic like this:

   if(pkg->embedded_sig) {
use(pkg->embedded_sig)
   } else {
sig = load_detached_sig(pkg)
use(sig)
   }

I found only 2 places where pacman does not fallback to a detached signature:

1) Keyring key check. Pacman was using embedded signatures only. This
has been fixed in pacman’s commit b01bcc7d3d680 and it will be
available in pacman version 6.x

2) dump_pkg_full() that dump package information. If a package uses
detached signatures only then it prints “None”. I think this is fine
as this function displays database entries and it does not affect the
package verification process.

I disabled the embedded signatures at my testing machine to use
detached signatures only and things look great so far. ‘pacman
--debug’ confirms that detached signatures are correctly downloaded
and used to verify the package content.

Given this information I would like to propose to stop using embedded
signatures and move to detached signatures by default. This will
require pacman 6.x or as alternative backport the fix(es) to 5.x
branch. It will help to make system updates even faster, something
that me and many other Arch users really love.

Re: [arch-dev-public] RFC: go-pie removal in favour of GOFLAGS

[Anatol Pomozov via arch-dev-public]

Hi Morten

On Sun, Apr 19, 2020 at 3:32 PM Morten Linderud via arch-dev-public
 wrote:
>
> Yo!
>
> After being lazy for a few weeks, I got around to writing the new guidelines 
> for
> Go packages. Currently it's a draft and I'd love if people read through it and
> ack/nacked
>
> https://wiki.archlinux.org/index.php/User:Foxboron/Go_packaging_guidelines

Thank you for coming up with this document.

Go is a fantastic language and I would love to see more people using
it. Having things documented will help our users to get more
comfortable with the golang packaging.

> The current changes is dropping anything pre-1.11 completely. I don't see the
> need to detail the GOPATH hacks as new packages should be using go modules, 
> and
> existing packages is moving to go modules.

+1. The current "go modules" implementation is so much more superior
to what Golang had before. "go modules" is the default option
nowadays.

> The other changes is explicitly listing up the needed CGO_* variables needed.
> Levente pointed out CGO_CFLAGS is still needed for FORTIFY_SOURCE, so for the
> sake of completeness it's all listed.
>
> I have also removed `-mod=vendor` from the default listing in `prepare()` as
> Anatol wasn't fond of the idea. I'm still unsure if we really want this as we
> would be willynilly downloading sources in `build()` and `check()` during
> packaging.

My concern was related to use of vendorized sources. "-vendor" was not
created for the cache management. If one wants to warmup the gomodules
download cache then "go mod download" should be used. See more info
about this tool here https://github.com/golang/go/issues/26610

And if we plan to encourage adding the cache warmup to all golang
PKGBUILD (that's gonna be thousands of packages) then this extra
complexity need be justified. Anyone who wants to follow our advise
needs to have clear answers to following questions: Is this feature
solely to avoid the download during the build() step or there is
something else? Why downloading in build() is bad, does it hurt the
users? What use-cases does show an advantage of the cache warmup..
etc..

Re: [arch-dev-public] RFC: go-pie removal in favour of GOFLAGS

[Anatol Pomozov via arch-dev-public]

Hello

On Sun, Mar 15, 2020 at 12:24 PM Morten Linderud via arch-dev-public
 wrote:
>
> On Sun, Mar 15, 2020 at 12:09:07PM -0700, Anatol Pomozov via arch-dev-public 
> wrote:
> > > Notice that `-mod=vendor` is also added to `GOFLAGS`.
> >
> > Most of the Go projects do not vendorize their dependencies to avoid
> > polluting the source tree.
> >
> > And there is no point to force vendorizing in the PKGBUILD neither. go
> > modules are doing a great job with pinning dependencies to a specific
> > version thus eliminating the main reason for vendor'ization existence
> > (i.e. reproducible builds).
> >
> > Thus following YAGNI principle I propose to drop this "-mod=vendor" flag.
>
> `-mod=vendor` is implicit as of go 1.14.

-mod=vendor is neither implicit nor required. -mod=vendor flag is
enabled by default only if upstream project uses vendorized project
structure so the build scripts do not have to add this flag manually.

The decisions whether to use a vendor directory structure or not
should be up to upstream developers.

> We are forcing this to ensure we are
> not running into new implicit behaviour in the future, such as updating pinned
> versions or what not. They have changed this once before already.

I do not quite follow this argument. Could you please expand more on it?

>  We need to fetch the dependencies before over the wire before build() and
> check().

go modules will fetch the dependencies using the information from
go.sum. I have go packages with dependencies, and following build()

build() {
  cd $dir
  go build
}

works great, including chroot environment.

Re: [arch-dev-public] RFC: go-pie removal in favour of GOFLAGS

[Anatol Pomozov via arch-dev-public]

Hi

On Sun, Mar 15, 2020 at 12:16 PM Christian Rebischke via
arch-dev-public  wrote:
>
> On Sun, Mar 15, 2020 at 12:09:07PM -0700, Public mailing list for Arch Linux 
> development wrote:
> > Hello Morten
> >
> > On Sun, Mar 15, 2020 at 5:38 AM Morten Linderud via arch-dev-public
> >  wrote:
> > >
> > >
> > > # Introduction
> > >
> > > To enable PIE compilation, we have relied on a patched version of the go
> > > compiler which has been distributed as `go-pie` since around 2017. 
> > > However, full
> > > RELRO support for go binaries has been a bit back and forth the past 
> > > years. With
> > > some thing working, and other things don't.
> > >
> > > With the release of Go 1.11 there was support for a general `GOFLAGS` 
> > > variable
> > > that lets us pass flags directly to the compiler. This email details what 
> > > these
> > > flags should be going forward.
> >
> > Big +1 for removing go-pie in favor of using GOFLAGS. It makes the
> > go-based packages management more straightforward and cleaner.
>
> +1 from me to for this step.
>
> > > # Flags
> > >
> > > Expected environment variables in PKGBUILDs:
> > >
> > > export CGO_LDFLAGS="$LDFLAGS"
> > > export GOFLAGS="-buildmode=pie -trimpath -mod=vendor -modcacherw"
> > >
> > >
> > > Explanation:
> > >
> > > * `CGO_LDFLAGS` passes the proper `LDFLAGS` to the linker. This should 
> > > enable
> > >   full RELRO when used in conjunction with `GOFLAGS`.
> > >
> > > * `-buildmode=pie` is the proper way to enable PIE and replaces the 
> > > `go-pie`
> > >   patch.
> > >
> > > * `-trimpath` this is to achieve reproducible builds and remove PWD from 
> > > the
> > >   binary.
> > >
> > > * `-modcacherw` modules are downloaded to `$GOPATH/pkg/mod` and by 
> > > default have
> > >   the permissions 444 for god knows why. If we want to run `makepkg -c` 
> > > or `git
> > >   clean` we won't have the correct permissions. This is probably not a big
> > >   problem for repository packages, but it's a nice addition so they work 
> > > as
> > >   expected.
> > >
> > > Notice that `-mod=vendor` is also added to `GOFLAGS`.
> >
> > Most of the Go projects do not vendorize their dependencies to avoid
> > polluting the source tree.
> >
> > And there is no point to force vendorizing in the PKGBUILD neither. go
> > modules are doing a great job with pinning dependencies to a specific
> > version thus eliminating the main reason for vendor'ization existence
> > (i.e. reproducible builds).
> >
> > Thus following YAGNI principle I propose to drop this "-mod=vendor" flag.
> >
>
> Correct me if I am wrong, but wouldn't that mean that, if you have a
> package with vendor directory you would download the same content twice?

No, it will not download twice. If upstream provides top-level
"vendor" directory  then go 1.14 modules will use these sources
instead of the one from cache.

Quoting the Go 1.14 release notes: "When the main module contains a
top-level vendor directory and its go.mod file specifies go 1.14 or
higher, the go command now defaults to -mod=vendor for operations that
accept that flag."

https://golang.org/doc/go1.14#go-command

Re: [arch-dev-public] RFC: go-pie removal in favour of GOFLAGS

[Anatol Pomozov via arch-dev-public]

Hello Morten

On Sun, Mar 15, 2020 at 5:38 AM Morten Linderud via arch-dev-public
 wrote:
>
>
> # Introduction
>
> To enable PIE compilation, we have relied on a patched version of the go
> compiler which has been distributed as `go-pie` since around 2017. However, 
> full
> RELRO support for go binaries has been a bit back and forth the past years. 
> With
> some thing working, and other things don't.
>
> With the release of Go 1.11 there was support for a general `GOFLAGS` variable
> that lets us pass flags directly to the compiler. This email details what 
> these
> flags should be going forward.

Big +1 for removing go-pie in favor of using GOFLAGS. It makes the
go-based packages management more straightforward and cleaner.

> # Flags
>
> Expected environment variables in PKGBUILDs:
>
> export CGO_LDFLAGS="$LDFLAGS"
> export GOFLAGS="-buildmode=pie -trimpath -mod=vendor -modcacherw"
>
>
> Explanation:
>
> * `CGO_LDFLAGS` passes the proper `LDFLAGS` to the linker. This should enable
>   full RELRO when used in conjunction with `GOFLAGS`.
>
> * `-buildmode=pie` is the proper way to enable PIE and replaces the `go-pie`
>   patch.
>
> * `-trimpath` this is to achieve reproducible builds and remove PWD from the
>   binary.
>
> * `-modcacherw` modules are downloaded to `$GOPATH/pkg/mod` and by default 
> have
>   the permissions 444 for god knows why. If we want to run `makepkg -c` or 
> `git
>   clean` we won't have the correct permissions. This is probably not a big
>   problem for repository packages, but it's a nice addition so they work as
>   expected.
>
> Notice that `-mod=vendor` is also added to `GOFLAGS`.

Most of the Go projects do not vendorize their dependencies to avoid
polluting the source tree.

And there is no point to force vendorizing in the PKGBUILD neither. go
modules are doing a great job with pinning dependencies to a specific
version thus eliminating the main reason for vendor'ization existence
(i.e. reproducible builds).

Thus following YAGNI principle I propose to drop this "-mod=vendor" flag.

> This will make sure we are
> using the vendored dependencies in the project. If they are not present, 
> please
> ensure they are downloaded in the `prepare` function:
>
> prepare(){
> cd $pkgname-$pkgver
> go mod vendor
> }
>
> If the project is *not* using Go 1.11 modules, missing `go.mod` and/or 
> `go.sum`
> in the project root, then disable it with `export GO111MODULE=off` and 
> continue
> with symlink hacks.
>
> Some upstreams override these values for strange reasons in their `Makefile` 
> and
> build systems. You *need* to read over them and ensure this does not happen!
>
>
> # Pacman
>
> Clearly we shouldn't have to specify this in every PKGBUILD, so I have been
> playing with a `pacman` patch that passes all of the variables. However I have
> been struggling with debug support and figuring out that part of the flags, so
> nothing have been upstreamed yet.
>
> However this is only applicable to around 126 packages, so I guess it's fine? 
>  ¯\_(ツ)_/¯
>
>
> # In conclusion...
>
> If there are no objections to the New Way Of Doing Things™, I'll be updating 
> the
> package guidelines within the next week or two and drop the `go-pie` package
> containing the patch. For the sake of compatibility, the `go` package will 
> contain a
> `replaces=('go-pie')`. I also expect people packaging go packages to follow 
> the
> guidelines!
>
> --
> Morten Linderud
> PGP: 9C02FF419FECBE16

Re: [arch-dev-public] Todos for language specific rebuilds

[Anatol Pomozov via arch-dev-public]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Hi

On 2020-01-12 at 00:04, arch-dev-public@archlinux.org wrote:
> On 1/10/20 4:42 PM, Christian Rebischke via arch-dev-public wrote:
> > Hi everybody,
> >
> > I would like to propose that we create todos for rebuilds of language
> > specific packages.
> >
> > We had two major rebuilds in the last months: python3.8 and ruby2.7.
> >
> > Can we agree that we create a todo before such rebuilds?
> > The advantages outweigh the disadvantages. We would gain:
> >
> > * More people help rebuilding the packages.'
>
> What help is needed? If this is just about having more people sed the
> pkgrel variable with "$pkgrel + 1", then try to build it, more people
> doesn't actually help. We have automated rebuilders which are very
> capable in this regard.
>
> > * Every maintainer gets informed about the rebuild.
>
> I agree with you that this is indeed a problem, and I would like to
> propose a pretty simple solution. Let's post on arch-dev-public to give
> people a heads-up.
>
> This means even if your package failed to be detected for rebuilding and
> would never appear on any TODO, you as a maintainer know that it
> happened and can manually rebuild your package.
>
> > * Maintainers have the possibility to test the packages.
>
> At least for the python rebuilds, the process of rebuilding the
> ecosystem is long and painfully drawn out, *because* packages with
> failing testsuites cannot be rebuilt automatically and go onto a TODO
> list of broken packages.
>
> Given this thread started because we just rebuilt ruby, can I assume
> that PKGBUILDs for ruby packages are in the general habit of not
> containing check() functions for running unittests? Either because
> upstream does not have unittests or because they are not being run?
>
> If packages have upstream unittests but don't run them, then the
> maintainer of the package has been derelict in his or her duty.
>
> If packages do NOT have upstream unittests, then this is unfortunate,
> and I don't currently have an answer for what we should do. :(
>
> > If tools exist for creating todos, I would like to ask the persons with
> > such tools to make them available for everybody (if not already
> > happened).
>
> It's a website submission form that expects you to write some
> explanatory message, then fill in a newline-separated list of pkgnames.
> Any rebuilder must by definition have the latter, even if that rebuilder
> is "I scrolled through archweb and did it all manually by flipping back
> and forth between my terminal and my browser".
>
> No "tool for creating todos" need exist. Ask instead about tools for
> enumerating language dependencies.
>
> ...
>
> For python, it's pretty simple.
>
> pkgfile -d '/usr/lib/python3.8/'
>
> For ruby, it's also pretty simple:
>
> pkgfile -d '/usr/lib/ruby/'
Here are the instructions that I used to find the ruby packages
https://gist.github.com/anatol/670a2768a97789d6a96b33010b7fedca
-BEGIN PGP SIGNATURE-
Version: FlowCrypt 7.4.2 Gmail Encryption
Comment: Seamlessly send and receive encrypted email

wsFcBAEBCgAGBQJeGmgKAAoJELAoVO11Pg8fwTsQAJtOyNvia5wXkkTsdjrQ
LLkLRw2EF79mMbJa+zN6/H7m/lJAIdZKXFjWmhar3lt+52d0J+7+WHPrqHH0
151hpuJ6SJpceGDkzbfIST9/i+++HcUB6a9vDcJPEx8RqYQFMPH/4JxGndCR
Dg7bi2f9PzmYsv9KkZ6JsJ13Z/jRfiVozGf0J9b+xi79Z5Y5RMM91Q2hp8gR
6HFRw+S8+0j5P+1MlDlSSjPmflvC3wsYDRqcDZuaXtArmqb1BiGAK6s6d4aQ
L11LQSdP8clgYfGPJpk4U37SDSyrkTaBM5yWQLPaKVB7DY2Z81OGd3Z34tMj
QWsh/LZjgqtEOl+WkGcF2ykSa+mjpdyxrttotDh6Mwwa+7YFNxWMQiSO8Hmd
Z1iI3SCzIhQ33t/3raE0MDSJM8nEQwwrkyI2nr0C2BFUEIwMkiRkcLr3KDwf
MQI3Q9X3A0Fo5xwzWcEEjbJjwjGz9fzPcDh0JeOUs/e9Ck1TL/n4qwqhPeUr
kJFrrzEAD6YcXCTdbudHFnbmrdDd0wdTpb2SAYZuKWhYusEMWn1oxDK6TnLe
5ufVTXP9uQyG5xoEUX95L4S0GAGqJ6jU9dY+B1TTfRZg+bG6nq+FdpverzHB
WVN9opo7WqL8sE1fifViBbWo20KV0BCv3vUyFFBYkCLVcoIWgeeLHFZUegmr
p/nl
=4pkk
-END PGP SIGNATURE-

Re: [arch-dev-public] Todos for language specific rebuilds

[Anatol Pomozov via arch-dev-public]

Hello

On Sat, Jan 11, 2020 at 6:58 AM Dave Reisner  wrote:
>
> On Fri, Jan 10, 2020, 16:43 Christian Rebischke via arch-dev-public <
> arch-dev-public@archlinux.org> wrote:
>
> > Hi everybody,
> >
> > I would like to propose that we create todos for rebuilds of language
> > specific packages.
> >
> > We had two major rebuilds in the last months: python3.8 and ruby2.7.
> >
> > Can we agree that we create a todo before such rebuilds?
> > The advantages outweigh the disadvantages. We would gain:
> >
>
> Hi,
>
> I'm not sure I understand. Can you clearly state the problems we've
> encountered due to not doing this? What downsides do you see to your
> proposal? Can you think of any alternative solutions?
>
> * More people help rebuilding the packages.
> >
>
> Solving the wrong problem, IMO. This is largely toil and should be
> automated away.

I agree with every statement that Dave made. Especially with this one.

We need to automate as much our daily routine as possible. Scripting/automation
is the only way to keep the increasing complexity of the system under control.

Adding more bureaucracy and putting more dumb manual work to the developers
will certainly slow down the development process.

>
> Foutrelis already has such a system that we've used for
> rebuilds in the past. We could/should instead work towards making this more
> generally available on the build machines.

I am a huge fan of this approach. I would really love to see the Foutrelis'
rebuild machine available as a tool at my workstation.

> * Every maintainer gets informed about the rebuild.
> >
>
> As a maintainer, I don't care that you're rebuilding my package to keep up
> with library changes. Rather, I'm thankful to whomever did this for me.
>
> Why would a language rebuild differ from any other soname bump?
>
> * Maintainers have the possibility to test the packages.

Did you have any problems with testing the recent language rebuilds?
Could you please add more information how a todo list will fix it?

Re: [arch-dev-public] Semi-away till 2019

[Anatol Pomozov via arch-dev-public]

Hi

On Tue, Sep 3, 2019 at 9:36 AM Bartłomiej Piotrowski via
arch-dev-public  wrote:
>
> Hi all,
>
> As I have free time shortage lately, I think it's fair to officially say
> I will be in semi-away mode till 2019. I will try my best to keep up
> with uncomplicated pkgver bumps for packages I maintain on my own and
> make sure all keys are properly signed, but that's all I can promise.
>
> It also means I'm unable to give toolchain attention it deserves. Please
> poke me somewhere if you happen to be interested in co-maintaining it.

I can try to handle it for the next couple releases and see how it works for me.
Maybe there is a chance that I will learn to love the GCC build system.

If you are OK with it please add me as a co-maintainer. Any advises with
maintaining GCC toolchain? Any tribal knowledge that worth learning before
moving forward?

>
> My responses on IRC or via e-mail may be also delayed so take that into
> account before kicking me out.
>
> Bartłomiej

Re: [arch-dev-public] Moving gcc7 into [community] for CUDA

[Anatol Pomozov via arch-dev-public]

Hi

> Agreed, we're moving in a net positive direction. We still have two
> versions of gcc, but at least the old version is a *newer* old version.
>
> (We could name it gcc-cuda if that makes people happier?)

gcc-cuda will probably introduce a lot of confusion. Let's use
standard naming practice for the old versioned packages (i.e. gcc7 or
gcc-7).

[arch-dev-public] LLVM 6.0 rebuild

[Anatol Pomozov via arch-dev-public]

On Fri, Mar 16, 2018 at 6:20 AM, Arch Website Notification
 wrote:
> The todo list "LLVM 6.0" has had the following packages added to it for which 
> you are a maintainer:
>
>
> * community/crystal (x86_64) - 
> https://www.archlinux.org/packages/community/x86_64/crystal/
>
> Todo list information:
> Name: LLVM 6.0
> URL: https://www.archlinux.org/todo/llvm-60/
> Creator: Evangelos Foutras
> Description:
> libLLVM-5.0.so -> libLLVM-6.0.so
> libclang.so.5 -> libclang.so.6
>
> In addition to the above soname changes, our clang package no longer ships
> static libraries. Instead, those are built as individual shared libraries
> (using CMake's BUILD_SHARED_LIBS flag).
>
> ** I'll add LLVM 5 packages once it's clear they are needed for software that
> hasn't been ported to LLVM 6 yet. Feel free to ping me on IRC for this. **


Hello Evangelos

I tried to rebuilt Crystal package with LLVM and hit a compilation issue

./bin/crystal build --release --no-debug -D i_know_what_im_doing -o
.build/crystal src/compiler/crystal.cr -D without_openssl -D
without_zlib
_main.o: In function
`*Crystal::Compiler#pass_manager_builder:LLVM::PassManagerBuilder':
main_module:(.text+0x52dc08): undefined reference to `LLVMInitializeInstCombine'
collect2: error: ld returned 1 exit status


That described here
https://github.com/crystal-lang/crystal/issues/5556 and it turns out
to be a LLVM bug https://bugs.llvm.org/show_bug.cgi?id=35947.

This LLVM bug is fixed in HEAD (https://reviews.llvm.org/D44140) and
should be released in 6.0.1. Or maybe we can pull it to the Arch
package?

Re: [arch-dev-public] PSA: third-party gems have been split from 'ruby' package

[Anatol Pomozov via arch-dev-public]

Hello

There is actually another big third_party component that is currently
shipped together with ruby package - rubygems.

Rubygems is developed as a project [1] separately from ruby. Once in a
while ruby developers check-in rubygems into their source tree [2].
And up until now we used ruby's version of rubygems.

The issue is that rubygems keeps getting releases that are never
integrated into ruby releases. For example rubygems 2.7.6 has a number
of security bugfixes and it was not merged into ruby 2.5 branch.

I am going to split 'rubygems' package from 'ruby' and bring
'rubygems' up-to-date. 'ruby' will have a dependency to 'rubygems'
thus update does *not* require installing 'rubygems' separately.


[1] https://github.com/rubygems/rubygems
[2] https://github.com/ruby/ruby/tree/trunk/lib/rubygems
[3] 
https://github.com/rubygems/rubygems/commit/5971b486d4dbb2bad5d3445b3801c456eb0ce183

Re: [arch-dev-public] PSA: third-party gems have been split from 'ruby' package

[Anatol Pomozov via arch-dev-public]

Hi Christian

On Mon, Jan 29, 2018 at 9:36 AM, Christian Rebischke
 wrote:
> On Fri, Jan 26, 2018 at 04:36:32PM -0800, Public mailing list for Arch Linux 
> development wrote:
>> Hello folks
>>
>> There been a packaging issue with 'ruby' package that annoyed me for a
>> while. The problem comes from the fact that ruby-lang.org source
>> tarballs contain ruby sources itself *and* some third party packages
>> from rubygems.org. The third-party gems shipped by 'ruby' tarball are:
>> minitest, net-telnet, did_you_mean, power_assert, rake, test-unit,
>> xmlrpc, rdoc. Currently we repack these gems and ship it as a part of
>> ruby Arch package.
>>
>> Because gems are bundled with ruby package we have no way to update
>> gems separately. All we can do is to wait until ruby developers update
>> their bundle and release it.
>>
>> The plan is to split these gems from 'ruby' package and install it
>> independently from what ruby developers bundle. Upcoming ruby changes
>> do that - ruby-2.5.0-4 will not include any of the third-party gems
>> mentioned above.
>>
>> Two popular packages - rake and rdoc will get Arch packages (ruby-rake
>> and ruby-rdoc respectively). Other gems need to be installed either
>> from AUR or from gem.
>
>
> Hello Anatol,
> Good decision! I have rebuild asciidoctor with ruby-rdoc as
> makedependency. Shall I move it to testing and you move all packages in
> one turn over to the stable repositories? Will you create a todo for it?

As ruby-rdoc is a makedependency then you don't need to rebuild your
package. rdoc functionality did not change, it just split into
separate package.

>
> Chris

[arch-dev-public] PSA: third-party gems have been split from 'ruby' package

[Anatol Pomozov via arch-dev-public]

Hello folks

There been a packaging issue with 'ruby' package that annoyed me for a
while. The problem comes from the fact that ruby-lang.org source
tarballs contain ruby sources itself *and* some third party packages
from rubygems.org. The third-party gems shipped by 'ruby' tarball are:
minitest, net-telnet, did_you_mean, power_assert, rake, test-unit,
xmlrpc, rdoc. Currently we repack these gems and ship it as a part of
ruby Arch package.

Because gems are bundled with ruby package we have no way to update
gems separately. All we can do is to wait until ruby developers update
their bundle and release it.

The plan is to split these gems from 'ruby' package and install it
independently from what ruby developers bundle. Upcoming ruby changes
do that - ruby-2.5.0-4 will not include any of the third-party gems
mentioned above.

Two popular packages - rake and rdoc will get Arch packages (ruby-rake
and ruby-rdoc respectively). Other gems need to be installed either
from AUR or from gem.

Re: [arch-dev-public] Stepping back as Arch Linux developer

[Anatol Pomozov via arch-dev-public]

Hi Daniel

Thank for all the work you've done for Arch! It was please for me to
work with you.

Have fun with your new interests and offline activities.

On Mon, Oct 16, 2017 at 10:17 AM, Daniel Isenmann  wrote:
> Hi there,
>
> that wasn't an easy decision, but after months of inactivity and only
> sporadic checks of the mailing lists and emails, I have decided to step
> back. My interests moved and my free time is limited and Arch Linux
> deserves more time than I can actually offer.
>
> It was a fantastic and excited journey over all these years. It was great
> to be part of this community. I have met interesting people along the way,
> had interesting discussions and learned a lot.
>
> I wish Arch Linux and each of you all the best. Nevertheless I will be
> around as a user.
>
> Keep Arch Linux rolling
>
> Cheers
> Daniel

[arch-dev-public] fuse packages reorganization

[Anatol Pomozov via arch-dev-public]

Hi folks

I want to give you heads up about fuse packages reorganization.

fuse project had a major release recently - fuse v3 is officially out [1].
Following recommendations from the upstream project [1] I renamed package
'fuse' to 'fuse2' and added 'fuse3' package. Common files from these
packages are moved to 'fuse-common'.

Fuse libraries v2 and v3 can co-exist at the same system. If you own a
package that depends on fuse please specify what exactly version it uses -
either 'fuse2' or 'fuse3'. 'fuse' is an alias for 'fuse2' now but some day
in the future this alias will be removed.

[1] https://github.com/libfuse/libfuse/releases/tag/fuse-3.0.0