Re: [Rpm-maint] [rpm-software-management/rpm] spec file preprocessing on rpm level (#1472)
> A spec will no longer be a spec if such a feature was added. I can see how > templating can be useful, but I think this would need to be thought of as a > different file type - say, `.spec.in` rather than `.spec` to make it clear. I think it's mostly matter of preference and compatibility. Additional options (except differentiating file type) that come to mind: - introduce new spec version, make it possible to declare spec version in spec file by e.g. `# rpmspec:2` in the first line - bump rpm major version with introduction of this feature - maybe something else Having filetype separated seems quite clean to me although it depends on whether rpm would be able to process such templates/files by default and the extension would trigger preprocessing or if an extra switch would be needed (likely cumbersome). -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/1472#issuecomment-779600475___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
[Rpm-maint] [rpm-software-management/rpm] spec file preprocessing on rpm level (#1472)
Hello,
since this change https://pagure.io/fesco/issue/2532 will not likely be
accepted, I would like to ask if you were willing to accept a pull request to
enable spec file preprocessing directly on rpm level. It would add a third type
of macro (except `%{}` and `%()`), which would work similarly to `%()` except
that the values (after expansion) would be put verbatim into the spec file that
goes into an srpm generated by rpmbuild.
Basically, spec files would be always passed through `preproc` utility first
before they would be handed over for further rpm processing. Enabling/disabling
of this feature would be only system-wide, not spec-wide as you need to
preprocess a template first before parsing it.
This is a way to bring innovation to rpm ecosystem mainly by:
- enabling automatic version, release, changelog, VCS generation e.g. from git
metadata
- enabling dynamic source generation e.g. from a git submodule with mirrored
upstream sources
- providing a way for automatic generation of requires and build requires by
analyzing upstream sources
The only added dependency would the `preproc` utility, which I would just
rewrite from python to C, it is less than 100 lines of code currently so it
won't be a problem.
People basically didn't like that `rpmbuild -bs` would no longer always work on
a spec file within Fedora DistGit after accepting the aforementioned change.
Having the preprocessor directly in rpm would solve that problem.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/1472___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: read sources checksums from the SPEC file and verify them (#463)
Well, on the other hand, it is a minimalistic solution that doesn't introduce any new rpm constructs. Also, you don't need to know or read the RFC. You just need to read the rpm documentation where it is described that source checksums are attached to source URLs as `#()` suffixes. With an example, I think, people would get it immediately. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/463#issuecomment-605676707___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: read sources checksums from the SPEC file and verify them (#463)
But, yeah, maybe you guys have some other/better solution in mind. Idk. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/463#issuecomment-603244500___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: read sources checksums from the SPEC file and verify them (#463)
Cool, ok, I admit I am not sure how big deal is the macOS compatibility. I think if somebody complains that the feature doesn't work there, it would be a reason to instead use some c library for the checksum validation. Maybe that could be done immediately? I would still consider relying on coreutils a good start. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/463#issuecomment-603237373___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: read sources checksums from the SPEC file and verify them (#463)
> > We could use a bit of bash code `%([ "$(sha256sum
> > | cut -d " " -f 1)" = ])` to do the verification per downloaded
> > source
>
> Yep, something like this is where I was heading to.
>
> > but i think `` might be slightly tricky unless rpm
> > exposes enviroment variable like 'SOURCES'
>
> RPM exposes `%{SOURCE1}` right after it appears in the specfile.
>
> > All urls that are now pointing to upstream would need to change to point to
> > dist-git lookaside cache if the rpm mechanism for downloading should be
> > used instead of the fedpkg one.
>
> No, the `%{SOURCE1}` points to the rpmbuild directory, not to the URL. I
> don't think that fetching sources by RPM is really in question here, but it
> could be used.
Alright but I think we are maybe tracking different goals then :). I thought
the issue was about making `%_disable_source_fetch 0` reliable. It's what the
original post suggested to me.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/463#issuecomment-603225969___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: read sources checksums from the SPEC file and verify them (#463)
Hmm, on Fedora: ``` $ dnf repoquery --requires rpm Last metadata expiration check: 0:00:08 ago on Tue 24 Mar 2020 01:34:47 PM CET. /usr/bin/bash /usr/bin/db_stat /usr/bin/sh coreutils curl libarchive.so.13()(64bit) libc.so.6(GLIBC_2.4)(64bit) libpopt.so.0()(64bit) libpopt.so.0(LIBPOPT_0)(64bit) libpthread.so.0()(64bit) libpthread.so.0(GLIBC_2.2.5)(64bit) librpm.so.9()(64bit) librpmio.so.9()(64bit) popt(x86-64) >= 1.10.2.1 rtld(GNU_HASH) ``` I can see coreutils there. So I guess you mean on other distributions...if you can name those and where `sha512sum` and similar checksum checking utils can be found, that would be great! Generally speakingIs adding dependency to rpm for checksum checking utilities a big deal? It seems to me that it is something that rpm could support. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/463#issuecomment-603214751___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: read sources checksums from the SPEC file and verify them (#463)
Anyway, what about something like `%_verify_fetched_source_checksums` macro with values `0/1/2` where ``` 0: do not check source checksums even if present 1: check source checksums if present as `#()` url suffix by invoking `sum` command from coreutils to do the check 2: check source checksums and return false when some fetched source does not have a checksum attached ``` I imagine the verification would only apply to files that were fetched by rpmbuild. Those that were already present before the build started wouldn't be checked with assumption that user gave to rpmbuild valid sources. What do you think? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/463#issuecomment-602809707___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: read sources checksums from the SPEC file and verify them (#463)
> One problem with the `sources` file is that it is distro specific -- Fedora > uses `sources`, OpenMandriva uses a similar file (though with slightly > different syntax) called `.abf.yml`, probably other distributions have yet > other workarounds. > > Another problem is that it's not the spec file -- I don't think we want to > end up with a mess similar to what dpkg has in those debian directories. I think it is distro-specific because it is tight down to particular distribution-git that some higher-level tools work with (fedpkg, centpkg, rpkg, ...), i.e. where the packages are stored. But the SourceX: in rpm spec file typically does not reference a tarball in that particular dist-git but it instead references sources from upstream (at least in Fedora/CentOS and for OpenMandriva that seems to be true as well). But that's not from where the tarballs are downloaded when they are going to be built. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/463#issuecomment-602752407___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: read sources checksums from the SPEC file and verify them (#463)
...Actually the fragment part shouldn't even be sent to the server so that note should probably be fixed... -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/463#issuecomment-601923626___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: read sources checksums from the SPEC file and verify them (#463)
> For me, causal user of OS sometimes building small packages, current way of > building rpms is very confusing. There are various tools which duplicate > functionality: rpmbuild, mock, rpkg, fedpkg etc. It's much harder to > understand than PKGBUILDs on Arch Linux for example. For me having ability to > declare checksums in rpmspec in standardized way would make UX much better. I am not sure what it would actually bring. It is good that you can make `%_disable_source_fetch 0` trustworthy but one could also do e.g. ``` Source0: https://github.com/release-engineering/dist-git/archive/dist-git-1.12-1.tar.gz#sha256 ``` with the current rpm syntax and then have a macro that verifies the tarball checksum based on the `#sha256`. Is it too hacky? :) It might not even be a hack because https://tools.ietf.org/html/rfc2396#section-4.1 - the fragment identifier is in the realm of client. So imho, new rpm syntax is not even needed. I think we could extend rpm macros to support this and it would be nice e.g. for COPR. It's a pity that url fragment already seems to be used for something: https://fedoraproject.org/wiki/Packaging:SourceURL#Troublesome_URLs but that maybe it doesn't necessarily mean we couldn't extend its use...? Btw. there is a note: "Sometimes this does not work because the upstream cgi tries to parse the fragment " that would be a server-side error according to the RFC. But for Fedora this has still a limited use because we would like to be independent of any of github/gitlab/bitbucket stuff. We could use that information (the hash) to automatically fetch the sources from upstream the first time when they are not yet present in our (Fedora) lookaside cache and then further rely on the `sources` file, the checksum there and the lookaside cache but I think this is a problematic solution. It would mean build system would need to commit into dist-git, which would mess up automatic versioning (if there is any). I haven't thought about this too much, there might be other issues. I think the fedpkg upload way is quite fine and it can be automated too (outside of Fedora infra) if one trusts 100% to the tarball source/provider. It's hard to tell if such automation should be recommended. > > This is the reason of huge success of AUR and PKGBUILDs in Arch Linux world — > simplicity. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/463#issuecomment-601922444___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] RFE: read sources checksums from the SPEC file and verify them (#463)
Currently, fedpkg downloading and uploading functionality is independent of rpm which brings an advantage that we can use it to upload/download custom binary stuff for other "package" types (e.g. containers). So placing the source hashes into spec file itself would limit the lookaside-cache functionality to rpms only. There is another factor that currently when I call `fedpkg sources` I know I am safe because it doesn't parse spec file. If we put the hashes into spec file, it would mean `fedpkg sources` would suddenly become a different beast with different security requirements (which should be considered when doing this automatically on production systems). -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/463#issuecomment-601781568___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] version comparison with epoch does not work as expected (#450)
>Whatever scheme is implemented for RPM comparison with missing values, the >following pathological dependency with all values missing SHOULD behave >naturally: > Requires: foo >= :- That seems to be really an extreme case. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/450#issuecomment-400365640___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] version comparison with epoch does not work as expected (#450)
> No. And that should be the end of that, thank you. While I value your point of view a lot, you haven't provided any arguments. Also it's not very useful to cut something down immediately without discussing it first. You might be right but you also might not be right. I understand you went through this before but that's not really an argument if this is a beneficial change to be made, which I believe it is. So, please, let's give it some space and time. I don't think this needs to be immediately rejected or accepted. If you have links to the previous discussions, that would help as well to save some time. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/450#issuecomment-400224928___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] version comparison with epoch does not work as expected (#450)
Yes, you are right the {E,V,R} compare sequence is alright but the `0:`
auto-prepending isn't.
I think that for the next major RPM release this change could be discussed.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/450#issuecomment-400107061___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] version comparison with epoch does not work as expected (#450)
> Well, the fact that e.g. 2:1.26 satisfies tar >= 1.28 looks like a clear bug > to me. I should also say why. That's because `tar 2:1.26` contains a completely different code than `tar 1.28`. `Requires: tar >= 1.28` is there because the codebase up from `1.28` is required to run the package correctly. It should not be possible to satisfy that requirement with tar version less than `1.28`. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/450#issuecomment-399394937___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] version comparison with epoch does not work as expected (#450)
Well, the fact that e.g. `2:1.26` satisfies `tar >= 1.28` looks like a clear bug to me. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/450#issuecomment-399354721___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] version comparison with epoch does not work as expected (#450)
Hey Jeff, So there are two things, right? First, you can say that when Epoch tag is not used, then Epoch tag's value is zero for a package. If rpm needs to compare two packages, which one is newer, then it can do full-fleged comparison with epochs included. If however comparing a package version (that always contains an epoch number) against a custom version expresion, which does not include an epoch number in it, it can just skip comparing the epochs. So, yes, missing Epoch tag implicates Epoch = 0 for a package. But missing epoch in a version expression means: "do not compare epochs". Does it make sense to you? The thing is that i would like to get spec files to be used directly by upstream and if they really needed to care about an epoch being set somewhere e.g. in Fedora, that just makes the whole thing...well, not possible. It's like this epoch algo really separates the whole rpm ecosystem into individual clusters. Could we, please, consider it again whether the behavior really can't be tweaked? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/450#issuecomment-399163634___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] version comparison with epoch does not work as expected (#450)
`Requires: tar >= 2:1.28` would work for me right now because the tar package I built has version `2:1.30-3`. But that seems rather accidental. The problem here is that if tar packager resets the epoch to zero at some point the `Requires: tar >= 2:1.28` will suddenly become impossible to satisfy and my package will there become impossible to install. If I put `Requires: tar >= 2:1.28` into the specfile, I will need to carefully watch what's going on with the tar package so that I change the condition in my specfile at the right time. This is really not something I would willingly want to do. When I put `Requires: tar >= 1.28` into the specfile, I am talking about tar 1.28 that is provided through rpm because I depend on a certain feature introduced in that version. The epoch should not really matter and certainly the condition should not get satisfied by `tar 2:1.26`. Therefore, I am quite positive this is a bug. I understand your explanation about no epoch meaning zero but at the same time, I would like to challenge this at least in the context of version comparison because it does not produce a convenient behavior for a packager, nor for an upstream developer who maintains his/her own spec file. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/450#issuecomment-398787595___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Add optional Class: attribute into Preamble (#236)
I've made a prototype of what I had in mind: https://pagure.io/lamp and it is just a normal package, in the end. It probably will be able to pass the Fedora standard package review process so the Class attribute is probably not needed. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/236#issuecomment-309665749___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
[Rpm-maint] [rpm-software-management/rpm] Add optional Class: attribute into Preamble (#236)
Hello, right now, spec files do not have any 'type' identifier, which means, they all need to be treated the same way in a distribution. If they contained a class specifier, there could be two (or more) types of packages, each with a different set of requirements given to them by a distribution standard (e.g. Fedora Packaging Guidelines for Fedora distribution). This might be very useful because with containers, the rpm packages can be constrained less heavily in what they can do in the system. At the same time, it will be useful to mark those packages as having those extended capabilities so that user (and tooling like a package manager) knows what to expect from them. That's why this information should be ideally stored in rpm header to be easily accessible to everyone. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/236___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
