Re: Exploded source tree
On Tue, Jun 6, 2017 at 4:47 AM, Michal Sekletar wrote: > On Tue, Jun 6, 2017 at 7:14 AM, Vít Ondruch wrote: > >> So what would be your workflow here? You would start update of the >> package by "fedpkg upload" to upload the new tarball and let dist-git to >> explode it? And what would be next? > > My ideal workflow would be to use pagure (repo with exploded source) > just like I use github/gitlab/bitbucket these days. Hence if I am > owner I commit directly or perhaps I file pull-request against the > package source. Then once commit is merged, package maintainer tells > koji to build the package. Note that Koji *has* support to build > directly from git repo referenced in your spec. We just don't use this > feature in Fedora, because up until pagure came along there was no > suitable code hosting/collaboration platform available to Fedora > package maintainers. > >> >> I thought that the exploded tree would be more passive, i.e. it would >> mirror just the basic tarball, possibly each Koji build would result in >> some release branch containing also the applied patches, but it seems >> you want to use it more actively > > Not only more actively, I would like to use it almost exclusively. > > In my opinion move into this direction would be hands down the best > change for package maintainers in years. Also I personally think such > possibility is also necessary if Fedora wants to attract new > contributors in era of Github. Because let's be honest, plain dist-git > (spec+patches) is just terrible experience compared to what people are > used to upstream. > I'd really rather not see this become a thing. There are five big reasons why I don't want this to become a thing with Fedora packages: 1. It encourages monkey patching and desynchronization from upstream. Given our philosophy, I'd rather it be more explicit and obvious that something is patched and altered, because the mere *burden* of patching should encourage us to track upstreams in general and fix things there. If you *are* the upstream, then you should not be making downstream forks of your own project. That's just dumb and can create all kinds of weird issues down the line. I have this problem with my software packaged by Debian people and it's very aggravating. They do fixes and don't bother to talk to me until they literally break everything, and by that point I don't feel particularly charitable anymore. They made their mess. I don't want us to become like that. 2. We actually have decent tooling for tracking more than one VCS repo in parallel now. Tools like tito and rdopkg (which sadly isn't in Fedora and doesn't support Python 3, boo!) can concurrently track and enable better management of the code and packaging when they are in separate repositories. 3. Pagure is already a huge disfavor for people who prefer non-Git VCSes (I personally prefer Mercurial whenever I can use it, and I've heard of people jumping onto VCSes like pijul, which has an interesting and different model of source control that doesn't map to Git at all). 4. I don't particularly want to have the same kind of terrible squawking that goes on in Debian about "how to structure code + packaging" for a package that often leads to nothing getting done for months. The fact we use Git for packaging is irrelevant and immaterial to upstreams, and they can use whatever they want. If we want the "user experience" to look like an exploded tree (like what "dpkg-source -x " does), we could make tooling to support that, but I would not want us to go down the road of actually maintaining that merged tree ourselves in Pagure. 5. It makes things worse when we have to scrub code. Debian has the lovely option of fancifully ignoring software patents because they make no money and there's no point suing them. We don't have that. Red Hat essentially owns Fedora for all legal purposes and they inherit the packages we have for Red Hat Enterprise Linux. That means we scrub code *before* uploading (freetype, chromium/qtwebengine, gst plugins, etc.). With the current model, this is easy to do and maintain. Merged trees make this much harder. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Exploded source tree
On Tue, Jun 6, 2017 at 6:25 AM, Miroslav Suchý wrote:
> Dne 5.6.2017 v 18:59 Richard W.M. Jones napsal(a):
>> It would be nice if this was more automated.
>
> Tito [1] can do it.
Also a shout out to https://github.com/softwarefactory-project/rdopkg
. This tool can between a "upstream" repo, a downstream fork
("patches") repo, and a dist-git repo. We use this in the Ceph and
OpenStack projects to manage many downstream patches in dist-git.
- Ken
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Exploded source tree
Dne 5.6.2017 v 18:59 Richard W.M. Jones napsal(a): > It would be nice if this was more automated. Tito [1] can do it. You can create branch (or fork from upstream) change in .tito/tito.props tagger to ReleaseTagger and builder to tito.builder.UpstreamBuilder which will change every commit since last tag to PatchX in spec or you can use tito.distributionbuilder.DistributionBuilder which will merge all commits since last tag to one huge Patch1. And there is KojiReleaser which can push the result directly into dist-git. However Tito assume few things (like having spec file in git-root). It needs to. Otherwise the automation is hard. Because otherwise every single project is using very unique work flow. And you cannot automate unique things. [1] https://github.com/dgoodwin/tito/ -- Miroslav Suchy, RHCA Red Hat, Senior Software Engineer, #brno, #devexp, #fedora-buildsys ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Exploded source tree
On Tue, Jun 06, 2017 at 07:14:09AM +0200, Vít Ondruch wrote: > So what would be your workflow here? You would start update of the > package by "fedpkg upload" to upload the new tarball and let dist-git to > explode it? And what would be next? First off, I don't think that the tarball should be stored in git. Debian does that (with pristine-tar etc) and it's very complex. So let's concentrate on projects where upstream already uses git and where their tarball URLs can be derived from git release tags. We fork the upstream git project to pagure. We push our downstream patches on top. The automation could then simply add Patch lines in the dist-git spec file with a list of the patches and do a build. I'm not sure how that would be triggered - perhaps just by the act of pushing to Pagure? We'd also need branches in Pagure corresponding to branches of Fedora. Getting the upstream tarball is trickier. Assuming upstream have a sensible naming scheme and stable location for tarballs, then ‘something something release tag something download the tarball’. This is not a fully formed idea, but I think we should keep it simple, unlike Debian's horribly complex https://wiki.debian.org/PackagingWithGit Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Exploded source tree
On Tue, Jun 6, 2017 at 7:14 AM, Vít Ondruch wrote: > So what would be your workflow here? You would start update of the > package by "fedpkg upload" to upload the new tarball and let dist-git to > explode it? And what would be next? My ideal workflow would be to use pagure (repo with exploded source) just like I use github/gitlab/bitbucket these days. Hence if I am owner I commit directly or perhaps I file pull-request against the package source. Then once commit is merged, package maintainer tells koji to build the package. Note that Koji *has* support to build directly from git repo referenced in your spec. We just don't use this feature in Fedora, because up until pagure came along there was no suitable code hosting/collaboration platform available to Fedora package maintainers. > > I thought that the exploded tree would be more passive, i.e. it would > mirror just the basic tarball, possibly each Koji build would result in > some release branch containing also the applied patches, but it seems > you want to use it more actively Not only more actively, I would like to use it almost exclusively. In my opinion move into this direction would be hands down the best change for package maintainers in years. Also I personally think such possibility is also necessary if Fedora wants to attract new contributors in era of Github. Because let's be honest, plain dist-git (spec+patches) is just terrible experience compared to what people are used to upstream. Just my 0.02... Michal ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Exploded source tree
Dne 5.6.2017 v 18:59 Richard W.M. Jones napsal(a): > On Mon, Jun 05, 2017 at 01:33:56PM +0200, Vít Ondruch wrote: >> >> Dne 4.6.2017 v 15:22 Richard W.M. Jones napsal(a): >>> Will the second Pagure instance store exploded trees of the upstream >>> software? Will the dist-git RPM source + patches form be generated >>> from that? >>> >>> Rich. >>> >> Hi Rich, >> >> Could you elaborate about your use case? > For example: > > https://pagure.io/fedora-ocaml/commits/fedora-27-4.04.1 > > This is the exploded tree of OCaml which eventually gets built into > the Fedora OCaml package. It's upstream OCaml at some upstream > release + some downstream patches (and patches from other forks in the > case of this particular package). > > Currently I manually synchronize the Fedora dist-git package > (ie. using ‘git format-patch’ to generate the patches, add them to > dist-git, list the patches in the spec file, commit and rebuild). > > The Fedora package ends up looking like this: > > http://pkgs.fedoraproject.org/cgit/rpms/ocaml.git/tree/ocaml.spec#n44 > > It would be nice if this was more automated. > >> Just today, I discussed this with other people and approached Copr guys >> (since they are currently formalizing the dist-git [1]) and discussed >> the possibility of having this implemented in Copr (for the start), but >> I was not able to convince them (because "you can expand the tarballs >> and compare them" etc). So may be you have more convincing arguments for >> this feature > The upstream tarball is still involved here, and the patches are still > visible in the spec file and applied to the tarball, same as always. > > We essentially trust that the upstream tarball is identical (or close > enough) to the upstream git release that the downstream patches will > apply properly. Which if upstream are doing it right shouldn't be a > problem. > > Rich. > So what would be your workflow here? You would start update of the package by "fedpkg upload" to upload the new tarball and let dist-git to explode it? And what would be next? I thought that the exploded tree would be more passive, i.e. it would mirror just the basic tarball, possibly each Koji build would result in some release branch containing also the applied patches, but it seems you want to use it more actively Vít ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Exploded source tree
On Mon, Jun 05, 2017 at 01:33:56PM +0200, Vít Ondruch wrote: > > > Dne 4.6.2017 v 15:22 Richard W.M. Jones napsal(a): > > > > Will the second Pagure instance store exploded trees of the upstream > > software? Will the dist-git RPM source + patches form be generated > > from that? > > > > Rich. > > > > Hi Rich, > > Could you elaborate about your use case? For example: https://pagure.io/fedora-ocaml/commits/fedora-27-4.04.1 This is the exploded tree of OCaml which eventually gets built into the Fedora OCaml package. It's upstream OCaml at some upstream release + some downstream patches (and patches from other forks in the case of this particular package). Currently I manually synchronize the Fedora dist-git package (ie. using ‘git format-patch’ to generate the patches, add them to dist-git, list the patches in the spec file, commit and rebuild). The Fedora package ends up looking like this: http://pkgs.fedoraproject.org/cgit/rpms/ocaml.git/tree/ocaml.spec#n44 It would be nice if this was more automated. > Just today, I discussed this with other people and approached Copr guys > (since they are currently formalizing the dist-git [1]) and discussed > the possibility of having this implemented in Copr (for the start), but > I was not able to convince them (because "you can expand the tarballs > and compare them" etc). So may be you have more convincing arguments for > this feature The upstream tarball is still involved here, and the patches are still visible in the spec file and applied to the tarball, same as always. We essentially trust that the upstream tarball is identical (or close enough) to the upstream git release that the downstream patches will apply properly. Which if upstream are doing it right shouldn't be a problem. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
