Re: [Rpm-maint] [rpm-software-management/rpm] RFE: BuildRequires generator (#104)
I agree that %prep is needed to do this. But there is another thing to think about: Right now BuildRequires are the Requires of the SRPM. So the procedure is to build the SRPM and then use its Requires to set up the build root. So you do not have any Requires for building the SRPM. And indeed the SRPM can be build without any additional tools and libraries. But for determining the dynamic BuildRequires (or even just running %prep) you will need additional tools. So the question is where do you get the Requires from for these. So you probably need to do: - Build SRPM - See that it contains dynamic BuildRequieres - Rebuild SRPM with BuildRequires(pre) installed running %prep and the dynamic Buildrequires scripts - Setup the build root with the SRPM and build the binary rpms -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/104#issuecomment-432966176___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: BuildRequires generator (#104)
> But for determining the dynamic BuildRequires (or even just running %prep) you will need additional tools. So the question is where do you get the Requires from for these. You have static `BuildRrequires` for this puprose. Those should be installed first, so the dynamic build requires can be calculated. If I remember correctly, somebody suggested to do "fix-point" calculation (install dynamic BR in loop, till something gets installed - but I doubt this is needed in real world). -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/104#issuecomment-432972680___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: BuildRequires generator (#104)
Of course, the SRPM format needs to be updated first; so we can store the dynamic build requires "unexpanded" there. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/104#issuecomment-432973624___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: BuildRequires generator (#104)
Well, the issue here is what is the procedure to build the package. There are two options: The one I sketched above with a two stage SRPM build. You could use BuildRequires(pre) or something similar to setup the second stage SRPM build. (You probably can replace the first stage by a pure parsing pass.) The other option is to spit out the additional requirements during the build of the binary packages by executing a %buildrequires script between %pre and %build and hope the build system is capable of installing those on the fly (or restarting the build afterwards). This seem much more fragile and dangerous as it requires root operations being triggered from a non root build. Overall the static Buildrequires are not quite the same as the Requires needed to calculate the dynamic BuildRequires and they should be kept separate. Especially for the cases where there is no script for dynamic build requires. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/104#issuecomment-432979163___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: BuildRequires generator (#104)
@ffesti I thought the same thing as you at first but you don't absolutely need the separate BuildRequires syntax. You can perfectly limit static BuildRequires to the part needed to compute additional BuildRequires before `%build` and just add any static BuildRequires needed for %build or %check to the output of this computation. (from a technical POW that's just putting those BuildRequires in an expand, and cat >> the result to the dynamic BuildRequires file) So, from a technical POW, sure the separate syntax would be nice to have, but it's not absolutely *required*. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/104#issuecomment-432987623___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: BuildRequires generator (#104)
@praiskup > If I remember correctly, somebody > suggested to do "fix-point" calculation (install dynamic BR in loop, till > something gets installed - but I doubt this is needed in real world). That was me and after playing with the concept for a year I agree you can live without it in the real world. A single install transaction before `%build` is sufficient. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/104#issuecomment-432988614___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: BuildRequires generator (#104)
On Thursday, October 25, 2018 11:23:58 AM CEST Florian Festi wrote: > There are two options: > [..second option..] This seem much more fragile and dangerous as it > requires root operations being triggered from a non root build. 1. You **wouldn't** trigger root actions (the fact that pm_request plugin does this is only a serious blocker bug which means that we can not ever use that in production), you'd only generate list of BuildRequires. From security POV, it is equivalent to specify static BuildRequires. 2. Well, it is as fragile as the first option where you need to run something like %prepforprep before %prep. And then "restart" the build. Or rely on the build system that it does a good thing. I can see that option 1 would be slightly more optimal from "build system POV", because you wouldn't necessarily install what you don't have to for that specific build stage; but it would much less optimal for package maintainers - because they'd have to think about the two stages, and perhaps duplicate the %prep section. So yes, the other thing is that we could make the BuildRequires more optimal, like to have BuildRequires(prep), BuildRequires(danamic_buildrequires), BuildRequires(check), etc. But this seems to be orthogonal "optimization" RFE to this PR. And IMO the good-old-static-BuildRequires can still stay equivalent to the *.src.rpm Requires. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/104#issuecomment-432996888___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Short option for SCM setting. (#564)
Okay so I do see some real-world uses for specifying temporary scm from the cli, such as not wanting backups by default but needing to rediff, so you might want to specify "gendiff" on an occasion. Or not wanting the overhead of git or other real scm for regular builds but wanting to use git for bisecting between patches. My biggest problems with this are 1) the apparent promise of scm interchangeability when they are not, I think this needs a disclaimer in the man page 2) use of a single letter option for what is quite a rare use If it's changed to --scm= and the use (and limitations) explained in the manual, I'd be okay merging it. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/564#issuecomment-433001037___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: BuildRequires generator (#104)
On Saturday, October 20, 2018 2:51:03 PM CEST nim-nim wrote: > So, with a year of hindsight, I've simplified the requirements to > > 1. run `%prep` > 2. run BuildRequires computation logic (either as part of prep or in a new >`%reqs` section between prep and build) I'd still prefer the separated section; (1) it would be clear what specfile/SRPM uses the dynamic BR computation, and (b) it would make the interface much more clear -> we could simply print the list of BuildRequires to stdout (newline separated list of BRs). Also - to keep things sane - the new section shouldn't change the %_builddir (mock _should_ execute that part with read-only %_builddir ideally). > 3. install the computed BuildRequires in the build root before `%build` if >you're running in one of those (ie mock). -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/104#issuecomment-433002932___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: BuildRequires generator (#104)
@praiskup
> -> we could simply print the list of BuildRequires to stdout (newline
> separated list of BRs).
>From a packager and macro writer POW a file like in %files -f is a bit simpler
>to manipulate – after a while one gets lost between the macro stdouts that are
>intended to write parts of the spec file, the ones intended to end up in
>%{echo:, the ones you want to end up in build logs, and so on. Plus taking
>over stdout prevents the use of good old echo/printf to debug problems.
But, I could live with stdout. I'd just prepare a detached file in `%prep` and
cat it in `%req`
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/104#issuecomment-433016835___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Short option for SCM setting. (#564)
@ascherer pushed 1 commit. 27409eb Add short rationale for shortcut '--scm=SCM'. -- You are receiving this because you are subscribed to this thread. View it on GitHub: https://github.com/rpm-software-management/rpm/pull/564/files/824ee78bafe094f73dbded3416a760b60177a3c4..27409eb3f8a943e3e2909a4ccdc450dbcb9ef268 ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: BuildRequires generator (#104)
OK, so: Add support a %buildrequires script that's executed after the %prep during the binary build. If it exists it is executed and its stdout read in as BuildDependencies. These are then checked. Those not found (or all of them) are printed out and written to a file and the build is terminated with a special exit code and the message: "Please install these additional build dependencies. They can be found in $BUILDROOT/__rpm_build_requires". Build systems would react with installing the new dependencies (may be caching the list somewhere) and restart the build afterwards - may be even with some short circuit param that uses the result of %prep and %buildrequires. The rpmbuild side of things can probably be done quite easily. No idea about the build system stuff. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/104#issuecomment-433024163___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: BuildRequires generator (#104)
@ffesti So, just to be sure we understand ourselves, we could have something like: ```specfile %prep %buildrequires cat buildrequires.lst %build ``` That works for me. Nice clean simple and effective. I assume the syntax of the lines outputted in `%buildrequires` would be the same as current `BuildRequires:` lines, without the `BuildRequires:` line header? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/104#issuecomment-433030417___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: BuildRequires generator (#104)
Yup, something along these lines. It would be encouraged to do the part right in the %buildrequires section and use scripts that write directly to stdout but this pattern would also work. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/104#issuecomment-433039439___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: BuildRequires generator (#104)
@ffesti As a packager, I'd do it that way just because having stdout taken over sucks from a debugging POW. That's one of the things I dislike most about autoprovs: no easy way to put debug statements to stdout when there is a problem (I've been known to write autodep code that echos fake "debug(anything I need to know)" provs to diagnose problems). -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/104#issuecomment-433047865___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: BuildRequires generator (#104)
Isn't that for stdout/stderr separation is for? All we need is to display the one that's not used for the reqs. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/104#issuecomment-433049137___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: BuildRequires generator (#104)
Well yes in theory. In practice you never have enough info to debug a problem quickly so a setup like `%prep`or `%build`where rpm echoes each executed line, and where you can insert debugging printfs as needed, is pretty much ideal. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/104#issuecomment-433050497___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: BuildRequires generator (#104)
One thing that concerns me is that now when srpms require certain packages, this information is visible from the source repo. I can run repoquery to check that nothing requires what I intent to orphan etc. If we generate those, we should make sure the srpms we put in the source repo have the info in them available. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/104#issuecomment-433050948___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: BuildRequires generator (#104)
That's interesting think for policy POV, thanks. That would certainly be an issue for FESCO before allowing us to use that in Fedora. But I don't think it is necessarily a blocker for the actual implementation in mock/rpm. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/104#issuecomment-433053248___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: BuildRequires generator (#104)
> One thing that concerns me is that now when srpms require certain packages, > this information is visible from the source repo. I can run repoquery to > check that nothing requires what I intent to orphan etc. If we generate > those, we should make sure the srpms we put in the source repo have the info > in them available. That's pretty much what I wrote in https://github.com/rpm-software-management/rpm/issues/104#issuecomment-431577981 if you want to do this kind of thing, you need either to redefine srpms as "current srpm + result of %buildrequires" or some other format between current srpms and rpms which is "current srpm + result of %buildrequires" . And then just output this format during builds, store it in repos the usual way, query it with repoquery, etc. Since the solution proposed by @ffesti does not require any special new rpm header, as long as you output this format, I don't think the rest of the tooling needs any specific adaptation. But, that's an enhancement. It's not strictly necessary to make dynamic buildrequires work. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/104#issuecomment-433054259___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: BuildRequires generator (#104)
> The rpmbuild side of things can probably be done quite easily. No idea about > the build system stuff. I actually like this idea and as maintainer of mock, I can promise implementation of this in Mock. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/104#issuecomment-433248114___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: BuildRequires generator (#104)
> I can run repoquery to check that nothing requires what I intent to retire > etc. If we generate those, we should make sure the srpms we put in the source > repo have the info in them available. I can imagine `rpmbuild -bs --try-really-hard` which would: 1) generate src.rpm the classic way 2) run %prep and %buildrequires 3) generate new src.rpm with injected requires from %buildrequires I can make support in mock for that and we can make this default in Koji, so all builds from Koji will have full set of buildrequires and you can still repoquery them. I am not afraid about Koji/Mock, but what about OBS @mlschroe ? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/104#issuecomment-433249318___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: BuildRequires generator (#104)
Well, the result of %buildrequires could be added if the srpm is build side by side with the binary packages. We could add some marker if it is build stand alone. So you could know in advance that there are dynamic buildrequirements still missing. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/104#issuecomment-433298639___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
