Dist Git for Copr
Hello everyone, I would like to make a proposal and start a discussion about Dist Git for Copr. At the moment, Copr can build packages only from source rpms, which has been uploaded in advance somewhere in the internet - so if you want to build something, you need your own file server. As people often delete the sources from their server after the build process, the builds are no longer easily reproducible. And there is also no history of the sources. We would like to use the Dist Git project[1] to store the sources. This would make all builds reproducible, there would be no need of having a file server to build and we would have a history of changes to each package. At first, we would use the Dist Git server as a storage only. Users would not have direct access. Instead, they would either: a) upload their source rpms somewhere and use Copr as before (no change in work-flow) b) submit the source rpms directly into Copr In both cases, Copr would import ($ fedpkg import) the packages into the Dist Git and build them from there. The design of git repos would be: Repo name: 'user-name/copr-project/pkg-name' Branch name: 'epel-7', 'fedora-22' At that moment, we would need a discussion on how to open the Dist Git server to users. - What would be the best web frontend? - Could we use Pagure? (used to be progit) - Which tool would they use? - How would they authenticate? There has been some unfinished discussion on https://fedorahosted.org/fedora-infrastructure/ticket/4564 The main purpouse of this is having a storage for sources, reproducible builds and history of sources. Eventualy, we can extend it with bug tracking, easy forking and merging and the other "Git-Hub-like" stuff. Thanks! Adam Samalik [1] https://github.com/release-engineering/dist-git ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Dist Git for Copr
Dne 6.5.2015 v 12:31 Adam Samalik napsal(a): > Branch name: 'epel-7', 'fedora-22' > > Can we stick to Fedora's dist-git branch name conventions please? Thanks Vít ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Dist Git for Copr
Sorry, the message I wanted to say was "git repos would represent packages and branches would represent platforms". So yes, I agree, the branch names should be the same as in Fedora. Adam - Original Message - From: "Vít Ondruch" To: infrastructure@lists.fedoraproject.org Sent: Wednesday, May 6, 2015 1:25:04 PM Subject: Re: Dist Git for Copr Dne 6.5.2015 v 12:31 Adam Samalik napsal(a): > Branch name: 'epel-7', 'fedora-22' > > Can we stick to Fedora's dist-git branch name conventions please? Thanks Vít ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Dist Git for Copr
On Wed, 6 May 2015 06:31:03 -0400 (EDT) Adam Samalik wrote: > Hello everyone, > > I would like to make a proposal and start a discussion about Dist Git > for Copr. Sounds good. We had some discussion in tickets, but here is likely a better place anyhow. ;) > At the moment, Copr can build packages only from source rpms, which > has been uploaded in advance somewhere in the internet - so if you > want to build something, you need your own file server. As people > often delete the sources from their server after the build process, > the builds are no longer easily reproducible. And there is also no > history of the sources. Yeah. ...snip... > There has been some unfinished discussion on > https://fedorahosted.org/fedora-infrastructure/ticket/4564 > > The main purpouse of this is having a storage for sources, > reproducible builds and history of sources. Eventualy, we can extend > it with bug tracking, easy forking and merging and the other > "Git-Hub-like" stuff. How about a short term and a longer term plan? Short term: have copr download and store the src.rpm from build urls. This would at least make things reproducable and at least someone could download the src.rpm and send a patch. Along with this a easy way to mail the owner of a copr would be good. Longer term: use pagure. Have people setup their project there and work with pagure folks to integrate copr. We could have an easy 'build this in copr' type thing and on the copr side a 'visit this project on pagure'. I think trying to do a standalone dist-get will just cause problems when/if we want to move to something like pagure and has a lot of issues without building up a bunch of infrastructure around it. Just my thoughts on it... kevin pgpbraIpiHdgt.pgp Description: OpenPGP digital signature ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Dist Git for Copr
Dne 6.5.2015 v 21:08 Kevin Fenzi napsal(a): > How about a short term and a longer term plan? > > Short term: have copr download and store the src.rpm from build urls. > This would at least make things reproducable and at least someone could > download the src.rpm and send a patch. Along with this a easy way to > mail the owner of a copr would be good. Nope, short term is that user select SRPM and do "file upload" from form -- or provide url. And the result will be stored in dist-git. So they do not need to host src.rpm on some public site. > Longer term: use pagure. Have people setup their project there and work > with pagure folks to integrate copr. We could have an easy 'build this > in copr' type thing and on the copr side a 'visit this project on > pagure'. I really dislike to store project in pagure. Unless it have git-annex or similar and we store there just spec and SOURCEX as in dist-git. We could not force users to move their github project to pagure. > I think trying to do a standalone dist-get will just cause problems > when/if we want to move to something like pagure and has a lot of > issues without building up a bunch of infrastructure around it. Why it should cause trouble. The goal is to have literally copy of fedora-dist. Just have it on different HW/Storage. Then every change would be just carbon copy. -- Miroslav Suchy, RHCA Red Hat, Senior Software Engineer, #brno, #devexp, #fedora-buildsys ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Dist Git for Copr
Do we want to store the project, or packages? To explain what I mean by "project" and "packages", let's have a look at the Copr project. The "project" is hosted here: https://git.fedorahosted.org/cgit/copr.git This is the home of the project. It stores the source code and could also have some issue tracker etc. If we want to store "project", Pagure seems like a nice solution. The "packages" (for fedora) are here: http://pkgs.fedoraproject.org/cgit/copr This hosts the packages and is more focused on the spec file and the packaging side. The project needs to live somewhere else. If we want to store "packages", Dist Git might be great. What we want to have? Or do we want to have some hybrid of those two above? Adam - Original Message - From: "Miroslav Suchý" To: infrastructure@lists.fedoraproject.org Sent: Friday, 15 May, 2015 2:41:07 PM Subject: Re: Dist Git for Copr Dne 6.5.2015 v 21:08 Kevin Fenzi napsal(a): > How about a short term and a longer term plan? > > Short term: have copr download and store the src.rpm from build urls. > This would at least make things reproducable and at least someone could > download the src.rpm and send a patch. Along with this a easy way to > mail the owner of a copr would be good. Nope, short term is that user select SRPM and do "file upload" from form -- or provide url. And the result will be stored in dist-git. So they do not need to host src.rpm on some public site. > Longer term: use pagure. Have people setup their project there and work > with pagure folks to integrate copr. We could have an easy 'build this > in copr' type thing and on the copr side a 'visit this project on > pagure'. I really dislike to store project in pagure. Unless it have git-annex or similar and we store there just spec and SOURCEX as in dist-git. We could not force users to move their github project to pagure. > I think trying to do a standalone dist-get will just cause problems > when/if we want to move to something like pagure and has a lot of > issues without building up a bunch of infrastructure around it. Why it should cause trouble. The goal is to have literally copy of fedora-dist. Just have it on different HW/Storage. Then every change would be just carbon copy. -- Miroslav Suchy, RHCA Red Hat, Senior Software Engineer, #brno, #devexp, #fedora-buildsys ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Dist Git for Copr
On Fri, 2015-05-15 at 08:56 -0400, Adam Samalik wrote: > Do we want to store the project, or packages? > > To explain what I mean by "project" and "packages", let's have a look at the > Copr project. > > The "project" is hosted here: https://git.fedorahosted.org/cgit/copr.git > This is the home of the project. It stores the source code and could also > have some issue tracker etc. > If we want to store "project", Pagure seems like a nice solution. > > The "packages" (for fedora) are here: http://pkgs.fedoraproject.org/cgit/copr > This hosts the packages and is more focused on the spec file and the > packaging side. The project needs to live somewhere else. > If we want to store "packages", Dist Git might be great. Pagure could also be great for the second. It offers git hosting, the way dist-git needs it (to store specs and patches), and it (could if it doesn't already) offers some space storage to upload tarballs. Then, maintainers of packages in coprs would gain a place for users to report their issues with the copr packages, which doesn't sound like a terrible thing. -- Mathieu ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Dist Git for Copr
On 05/15/2015 02:41 PM, Miroslav Suchý wrote: > Dne 6.5.2015 v 21:08 Kevin Fenzi napsal(a): >> How about a short term and a longer term plan? >> >> Short term: have copr download and store the src.rpm from build urls. >> This would at least make things reproducable and at least someone could >> download the src.rpm and send a patch. Along with this a easy way to >> mail the owner of a copr would be good. > Nope, short term is that user select SRPM and do "file upload" from form -- > or provide url. > And the result will be stored in dist-git. > So they do not need to host src.rpm on some public site. > >> Longer term: use pagure. Have people setup their project there and work >> with pagure folks to integrate copr. We could have an easy 'build this >> in copr' type thing and on the copr side a 'visit this project on >> pagure'. > I really dislike to store project in pagure. Unless it have git-annex or > similar and we store there just spec and > SOURCEX as in dist-git. BTW I'm working on integrating git-annex to dist-git, see https://github.com/release-engineering/dist-git/issues/1 -- Richard irc: impure_hate ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Dist Git for Copr
But the second variant - to store "packages" - would mean that I have a git repo for each package. For example Developer Toolset would need about 70 git projets because it's got about 70 packages: https://copr.fedoraproject.org/coprs/rhscl/devtoolset-3-el7/monitor/ That's fine for dist git, but not as a project. Ideal could be some kind of hybrid where we can have one project. This project could host the upstream sources (pagure) or just have link to github, etc. Whatever we decide here. We can even have both options to be more friendly :) And this project would also include the packages. I can think about two variants here: a) include the packages in dist git or something like that b) have some easy description for Copr how to build it. The example description for b) could look like this: 1) have this project: https://git.fedorahosted.org/cgit/copr.git 2) build from these directories: backend, selinux, cli, frontend, keygen 3) get the sources by: tito build --srpm Adam - Original Message - From: "Mathieu Bridon" To: "Fedora Infrastructure" Sent: Friday, 15 May, 2015 2:58:52 PM Subject: Re: Dist Git for Copr On Fri, 2015-05-15 at 08:56 -0400, Adam Samalik wrote: > Do we want to store the project, or packages? > > To explain what I mean by "project" and "packages", let's have a look at the > Copr project. > > The "project" is hosted here: https://git.fedorahosted.org/cgit/copr.git > This is the home of the project. It stores the source code and could also > have some issue tracker etc. > If we want to store "project", Pagure seems like a nice solution. > > The "packages" (for fedora) are here: http://pkgs.fedoraproject.org/cgit/copr > This hosts the packages and is more focused on the spec file and the > packaging side. The project needs to live somewhere else. > If we want to store "packages", Dist Git might be great. Pagure could also be great for the second. It offers git hosting, the way dist-git needs it (to store specs and patches), and it (could if it doesn't already) offers some space storage to upload tarballs. Then, maintainers of packages in coprs would gain a place for users to report their issues with the copr packages, which doesn't sound like a terrible thing. -- Mathieu ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
