Re: [aur-general] 'provides' info in AUR
On 2013-12-12 01:39, Xyne wrote:
> On 2013-12-11 22:57 +
> Jerome Leclanche wrote:
>
> >These are all issues that can easily be fixed. Sources/checksums can
> >be added, extra architectures can be generated, and split packages can
> >be described in sections. (.conf aka .ini file format would be a
> >perfect fit here and would be forwards-compatible with the current
> >format but pretty much anything with sections would do.)
> >
> >Just to be clear, I'm not advocating to specifically use .PKGINFO;
> >just to use a compiled, parseable file (such as the .SRCINFO that was
> >talked about).. and it seems the consensus is that this is the best
> >solution. Am I wrong? What are the issues with .SRCINFO?
>
> My response is not directly at you in particular. I'm reply to the thread as a
> whole.
>
> The real problem is that we are attacking this from the wrong end. We should
> no be trying to excavate machine-parsable metadata from Bash. We should be
> thinking of ways to encapsulate the metadata in a flexible way that can be
> used
> to generate Bash.
>
> This would solve all of these problems and open the doors for numerous
> packaging tools (graph generators, metadata miners, fully automated repo
> generators, etc.).
>
> I've played around a bit with encoding package metadata in JSON. All of the
> packages on my site are autogenerated from JSON files with associated Bash
> stubs containing functions. My own approach is customized to my release
> scripts
> and not immediately generalizable, but the idea itself is workable.
>
> The key selling point of Bash is flexibility due to it being a full scripting
> language, but for the metadata itself we don't need the full flexibility. I
> think the following would suffice:
>
> 1) A convention for expressing per-package namespace for split packages, with
> a
>way to inherit and override from other namespaces to avoid redundancies.
>This can be done with sub-objects and a little syntax, e.g.
>{
> "python-foo" : {
> ...
> },
> "python2-foo < python-foo" : {
>"depends" : ["python2", ...]",
>...
> }
>
> 2) Variables for convenience. A top-level field to specify a delimiter would
>suffice, e.g. {"variable delimiter" : "$", ...} and then later {..., "url"
> :
>"http://example.com/$pkgname$-$version$.tar.gz";, ...}. The file could be
>safely interpolated. Interpolation requires some thought but it should be
>easy to do in all sane cases.
>
> 3) The build, package, prepare and version functions would obviously still be
>written in Bash, but they would be kept in a separate file and devoid of
>metadata.
>
> 4) If truly necessary, syntax for inserting command output could be added as
>well, again with a top-level limiter for the command. This would allow
>parsers to decide exactly which commands to run, if any. The use of such
>commands would be restricted to cases in which they are absolutely
> necessary
>(if there are any).
>
> Overall, it is not much more verbose than current PKGBUILDS:
>
> "powerpill": {
> "pkgdesc": "Pacman wrapper for parallel and segmented downloads.",
> "pkgrel": 1,
> "pkgver": "2013.5.13",
> "arch": "any",
> "backup": [
> "etc/powerpill/powerpill.json"
> ],
> "depends": [
> "python3",
> "pyalpm",
> "pm2ml>2012.12.12",
> "reflector",
> "aria2"
> ],
> "optdepends": {
> "python3-threaded_servers": "internal Pacserve support",
> "rsync": "Rsync download support"
> },
> "md5sums": [
> "0f0d4c0096bd60287ab923038c3bbc47",
> "f58432feaaeb339e5d6b3692d3cecf17"
> ],
> ...
>
>
> As soon as I have some time I will provide a proof-of-principle implementation
> and post a RFC.
>
> Regards,
> Xyne
I'm very much in favour of this (I'd even though of something slightly
similar at some point). We need a PKGBUILD2.0 format that can overcome
of our current limitations.
The downside is flexibility. What if the sources are complete different
for each architecture. At the moment, we can have an "if" in place. The
same applies for depends that vary according to architecture. Would we
just have a bunch of differently named-fields?
Before you post an RFC, a complete example of a really complex package
(with wierd sources, depends and stuff) would be great! :D
Anyway, if this becomes a reality, feel free to CC me; I'd be willing
to help the development of tools for this format! :D
--
Hugo Osvaldo Barrera
pgpxAn4sLSDKZ.pgp
Description: PGP signature
Re: [aur-general] 'provides' info in AUR
On 2013-12-12 01:39 + Xyne wrote: >My response is not directly at you in particular. I'm reply to the thread as a >whole. *directed *replying etc. Apparently I'm dysgraphic. :/
Re: [aur-general] 'provides' info in AUR
On 2013-12-11 22:57 +
Jerome Leclanche wrote:
>These are all issues that can easily be fixed. Sources/checksums can
>be added, extra architectures can be generated, and split packages can
>be described in sections. (.conf aka .ini file format would be a
>perfect fit here and would be forwards-compatible with the current
>format but pretty much anything with sections would do.)
>
>Just to be clear, I'm not advocating to specifically use .PKGINFO;
>just to use a compiled, parseable file (such as the .SRCINFO that was
>talked about).. and it seems the consensus is that this is the best
>solution. Am I wrong? What are the issues with .SRCINFO?
My response is not directly at you in particular. I'm reply to the thread as a
whole.
The real problem is that we are attacking this from the wrong end. We should
no be trying to excavate machine-parsable metadata from Bash. We should be
thinking of ways to encapsulate the metadata in a flexible way that can be used
to generate Bash.
This would solve all of these problems and open the doors for numerous
packaging tools (graph generators, metadata miners, fully automated repo
generators, etc.).
I've played around a bit with encoding package metadata in JSON. All of the
packages on my site are autogenerated from JSON files with associated Bash
stubs containing functions. My own approach is customized to my release scripts
and not immediately generalizable, but the idea itself is workable.
The key selling point of Bash is flexibility due to it being a full scripting
language, but for the metadata itself we don't need the full flexibility. I
think the following would suffice:
1) A convention for expressing per-package namespace for split packages, with a
way to inherit and override from other namespaces to avoid redundancies.
This can be done with sub-objects and a little syntax, e.g.
{
"python-foo" : {
...
},
"python2-foo < python-foo" : {
"depends" : ["python2", ...]",
...
}
2) Variables for convenience. A top-level field to specify a delimiter would
suffice, e.g. {"variable delimiter" : "$", ...} and then later {..., "url" :
"http://example.com/$pkgname$-$version$.tar.gz";, ...}. The file could be
safely interpolated. Interpolation requires some thought but it should be
easy to do in all sane cases.
3) The build, package, prepare and version functions would obviously still be
written in Bash, but they would be kept in a separate file and devoid of
metadata.
4) If truly necessary, syntax for inserting command output could be added as
well, again with a top-level limiter for the command. This would allow
parsers to decide exactly which commands to run, if any. The use of such
commands would be restricted to cases in which they are absolutely necessary
(if there are any).
Overall, it is not much more verbose than current PKGBUILDS:
"powerpill": {
"pkgdesc": "Pacman wrapper for parallel and segmented downloads.",
"pkgrel": 1,
"pkgver": "2013.5.13",
"arch": "any",
"backup": [
"etc/powerpill/powerpill.json"
],
"depends": [
"python3",
"pyalpm",
"pm2ml>2012.12.12",
"reflector",
"aria2"
],
"optdepends": {
"python3-threaded_servers": "internal Pacserve support",
"rsync": "Rsync download support"
},
"md5sums": [
"0f0d4c0096bd60287ab923038c3bbc47",
"f58432feaaeb339e5d6b3692d3cecf17"
],
...
As soon as I have some time I will provide a proof-of-principle implementation
and post a RFC.
Regards,
Xyne
Re: [aur-general] 'provides' info in AUR
On Wed, Dec 11, 2013 at 9:11 PM, Dave Reisner wrote: > On Wed, Dec 11, 2013 at 08:21:44PM +, Jerome Leclanche wrote: >> On Wed, Dec 11, 2013 at 6:42 PM, Dave Reisner wrote: >> > On Wed, Dec 11, 2013 at 06:12:59PM +, Jerome Leclanche wrote: >> >> On Wed, Dec 11, 2013 at 5:52 PM, Sam S. wrote: >> >> > On Tue, Dec 10, 2013 at 4:21 PM, Jerome Leclanche >> >> > wrote: >> >> > >> >> >> I don't understand why makepkg -S >> >> >> doesn't include the .PKGINFO file from makepkg (and subsequently the >> >> >> AUR would use that if present instead of the grep system which fails >> >> >> as soon as variables/expansions are involved which is every other >> >> >> package). All the implementation is there. >> >> >> >> >> > >> >> > It would probably be better then what we have now, but a perfect >> >> > solution >> >> > would also account for PKGBUILDs that use Bash conditionals to set >> >> > different variables on i386 systems than on x86_64 systems (which is >> >> > pretty >> >> > common among AUR packages that package upstream binaries rather than >> >> > compiling from source). >> >> > >> >> > Reading values from .PKGINFO would populate the AUR with the values for >> >> > whichever architecture the package uploader happened to use. (So if the >> >> > maintainer changes, or the same maintainer works on different >> >> > computers, a >> >> > simple re-upload of an AUR package could suddenly change the package's >> >> > meta-info, i.e. the AUR would become more "fragile".) >> >> > >> >> > Of course, the "perfect solution" would be pretty difficult to >> >> > implement. >> >> > Gentoo had a GSoC project last year [1], to implement an efficient and >> >> > safe >> >> > (side-effect free) limited Bash parser / pseudo-interpreter in C++, >> >> > sufficient to extract all necessary values from Genoo's equivalent of >> >> > PKGBUILDs. Surely, this could have been useful for the AUR as well. But >> >> > I >> >> > can find no evidence of continued project activity after the GSoC period >> >> > ended, so it appears they have given up... :( >> >> > >> >> > --- >> >> > [1] http://dev.gentoo.org/~qiaomuf/libbash.html >> >> >> >> I love the fact someone could be working on a bash parser but that >> >> solution is *insane*. >> > >> > It's designed to be incomplete, and the project appears very dead. >> > >> >> This is a solved problem: use a metafile compiled by whatever tools >> >> you use in your distro/domain that can be parsed safely and easily. >> >> For Arch, those are PKGINFOs. >> > >> > No, go see historical conversations about a mythical .SRCINFO -- this is >> > what .AURINFO is based on. .SRCINFO is vaporware right now, and I again >> > refer you to unresolved discussions about how it would handle split >> > packages and package-specific overrides. >> > >> >> Good point on the differences per arch. I guess the obvious solution >> >> that comes to mind here is to have makepkg -S generate the source >> >> files for each arch value (eg. PKGINFO.x86_64, etc) but that's not >> >> necessarily good and is the subject of another discussion imo. >> > >> > To be clear, .PKGINFO is not the solution here. This file describes a >> > built package (something the AUR explicitly does not support). >> > >> > d >> >> Care to explain the reasoning? I'm looking at a few example PKGINFOs >> and they contain nothing that can exclusively be generated at >> package() or build() time (other than pkgver but that's already the >> case currently). >> >> J. Leclanche > > Here's a few reasons that come to mind: > > 1) As you've already discovered, you'd need a .PKGINFO file for every > potential architecture, rather than just a .SRCINFO which might describe > what architectures are known available. A .SRCINFO could express all > package and architecture specific overrides. The fact that an override > exists might be a useful bit of information to convey. > > 2) .PKGINFO doesn't contain things that are useful for source packages, > like, say... the source URL? checksums? > > 3) You can't possibly express split packages well. Having pkgname fields > in a .SRCINFO would mean you could describe all packages created by a > PKGBUILD rather than having some loose association between a PKGBUILD > and some possible .PKGINFO files which it might have generated. > > This is about *source* packages. The metadata needs for source packages > are not the same as the needs for binary packages. > These are all issues that can easily be fixed. Sources/checksums can be added, extra architectures can be generated, and split packages can be described in sections. (.conf aka .ini file format would be a perfect fit here and would be forwards-compatible with the current format but pretty much anything with sections would do.) Just to be clear, I'm not advocating to specifically use .PKGINFO; just to use a compiled, parseable file (such as the .SRCINFO that was talked about).. and it seems the consensus is that this is the best solution. Am I wrong? What are
Re: [aur-general] 'provides' info in AUR
On Thu, Dec 12, 2013 at 08:47:49AM +1000, Justin Dray wrote: > What is to stop us using both? I don't think anyone has said we should stop > parsing the PKGBUILD for the AUR and exclusively parse a .PKGINFO instead. We should stop parsing the PKGBUILD for the AUR and exclusively parse a .SRCINFO instead.
Re: [aur-general] 'provides' info in AUR
On Thu, Dec 12, 2013 at 7:11 AM, Dave Reisner wrote: > On Wed, Dec 11, 2013 at 08:21:44PM +, Jerome Leclanche wrote: > > On Wed, Dec 11, 2013 at 6:42 PM, Dave Reisner wrote: > > > On Wed, Dec 11, 2013 at 06:12:59PM +, Jerome Leclanche wrote: > > >> On Wed, Dec 11, 2013 at 5:52 PM, Sam S. wrote: > > >> > On Tue, Dec 10, 2013 at 4:21 PM, Jerome Leclanche < > adys...@gmail.com> wrote: > > >> > > > >> >> I don't understand why makepkg -S > > >> >> doesn't include the .PKGINFO file from makepkg (and subsequently > the > > >> >> AUR would use that if present instead of the grep system which > fails > > >> >> as soon as variables/expansions are involved which is every other > > >> >> package). All the implementation is there. > > >> >> > > >> > > > >> > It would probably be better then what we have now, but a perfect > solution > > >> > would also account for PKGBUILDs that use Bash conditionals to set > > >> > different variables on i386 systems than on x86_64 systems (which > is pretty > > >> > common among AUR packages that package upstream binaries rather than > > >> > compiling from source). > > >> > > > >> > Reading values from .PKGINFO would populate the AUR with the values > for > > >> > whichever architecture the package uploader happened to use. (So if > the > > >> > maintainer changes, or the same maintainer works on different > computers, a > > >> > simple re-upload of an AUR package could suddenly change the > package's > > >> > meta-info, i.e. the AUR would become more "fragile".) > > >> > > > >> > Of course, the "perfect solution" would be pretty difficult to > implement. > > >> > Gentoo had a GSoC project last year [1], to implement an efficient > and safe > > >> > (side-effect free) limited Bash parser / pseudo-interpreter in C++, > > >> > sufficient to extract all necessary values from Genoo's equivalent > of > > >> > PKGBUILDs. Surely, this could have been useful for the AUR as well. > But I > > >> > can find no evidence of continued project activity after the GSoC > period > > >> > ended, so it appears they have given up... :( > > >> > > > >> > --- > > >> > [1] http://dev.gentoo.org/~qiaomuf/libbash.html > > >> > > >> I love the fact someone could be working on a bash parser but that > > >> solution is *insane*. > > > > > > It's designed to be incomplete, and the project appears very dead. > > > > > >> This is a solved problem: use a metafile compiled by whatever tools > > >> you use in your distro/domain that can be parsed safely and easily. > > >> For Arch, those are PKGINFOs. > > > > > > No, go see historical conversations about a mythical .SRCINFO -- this > is > > > what .AURINFO is based on. .SRCINFO is vaporware right now, and I again > > > refer you to unresolved discussions about how it would handle split > > > packages and package-specific overrides. > > > > > >> Good point on the differences per arch. I guess the obvious solution > > >> that comes to mind here is to have makepkg -S generate the source > > >> files for each arch value (eg. PKGINFO.x86_64, etc) but that's not > > >> necessarily good and is the subject of another discussion imo. > > > > > > To be clear, .PKGINFO is not the solution here. This file describes a > > > built package (something the AUR explicitly does not support). > > > > > > d > > > > Care to explain the reasoning? I'm looking at a few example PKGINFOs > > and they contain nothing that can exclusively be generated at > > package() or build() time (other than pkgver but that's already the > > case currently). > > > > J. Leclanche > > Here's a few reasons that come to mind: > > 1) As you've already discovered, you'd need a .PKGINFO file for every > potential architecture, rather than just a .SRCINFO which might describe > what architectures are known available. A .SRCINFO could express all > package and architecture specific overrides. The fact that an override > exists might be a useful bit of information to convey. > > 2) .PKGINFO doesn't contain things that are useful for source packages, > like, say... the source URL? checksums? > > 3) You can't possibly express split packages well. Having pkgname fields > in a .SRCINFO would mean you could describe all packages created by a > PKGBUILD rather than having some loose association between a PKGBUILD > and some possible .PKGINFO files which it might have generated. > > This is about *source* packages. The metadata needs for source packages > are not the same as the needs for binary packages. > > What is to stop us using both? I don't think anyone has said we should stop parsing the PKGBUILD for the AUR and exclusively parse a .PKGINFO instead. It would just provide some supplementary information that isn't currently available in an easy to parse method from the PKGBUILD without running arbitrary code on the server side. Reading both files would solve that, right? Regards, Justin Dray E: jus...@dray.be M: 0433348284
Re: [aur-general] 'provides' info in AUR
On Wed, Dec 11, 2013 at 08:21:44PM +, Jerome Leclanche wrote: > On Wed, Dec 11, 2013 at 6:42 PM, Dave Reisner wrote: > > On Wed, Dec 11, 2013 at 06:12:59PM +, Jerome Leclanche wrote: > >> On Wed, Dec 11, 2013 at 5:52 PM, Sam S. wrote: > >> > On Tue, Dec 10, 2013 at 4:21 PM, Jerome Leclanche > >> > wrote: > >> > > >> >> I don't understand why makepkg -S > >> >> doesn't include the .PKGINFO file from makepkg (and subsequently the > >> >> AUR would use that if present instead of the grep system which fails > >> >> as soon as variables/expansions are involved which is every other > >> >> package). All the implementation is there. > >> >> > >> > > >> > It would probably be better then what we have now, but a perfect solution > >> > would also account for PKGBUILDs that use Bash conditionals to set > >> > different variables on i386 systems than on x86_64 systems (which is > >> > pretty > >> > common among AUR packages that package upstream binaries rather than > >> > compiling from source). > >> > > >> > Reading values from .PKGINFO would populate the AUR with the values for > >> > whichever architecture the package uploader happened to use. (So if the > >> > maintainer changes, or the same maintainer works on different computers, > >> > a > >> > simple re-upload of an AUR package could suddenly change the package's > >> > meta-info, i.e. the AUR would become more "fragile".) > >> > > >> > Of course, the "perfect solution" would be pretty difficult to implement. > >> > Gentoo had a GSoC project last year [1], to implement an efficient and > >> > safe > >> > (side-effect free) limited Bash parser / pseudo-interpreter in C++, > >> > sufficient to extract all necessary values from Genoo's equivalent of > >> > PKGBUILDs. Surely, this could have been useful for the AUR as well. But I > >> > can find no evidence of continued project activity after the GSoC period > >> > ended, so it appears they have given up... :( > >> > > >> > --- > >> > [1] http://dev.gentoo.org/~qiaomuf/libbash.html > >> > >> I love the fact someone could be working on a bash parser but that > >> solution is *insane*. > > > > It's designed to be incomplete, and the project appears very dead. > > > >> This is a solved problem: use a metafile compiled by whatever tools > >> you use in your distro/domain that can be parsed safely and easily. > >> For Arch, those are PKGINFOs. > > > > No, go see historical conversations about a mythical .SRCINFO -- this is > > what .AURINFO is based on. .SRCINFO is vaporware right now, and I again > > refer you to unresolved discussions about how it would handle split > > packages and package-specific overrides. > > > >> Good point on the differences per arch. I guess the obvious solution > >> that comes to mind here is to have makepkg -S generate the source > >> files for each arch value (eg. PKGINFO.x86_64, etc) but that's not > >> necessarily good and is the subject of another discussion imo. > > > > To be clear, .PKGINFO is not the solution here. This file describes a > > built package (something the AUR explicitly does not support). > > > > d > > Care to explain the reasoning? I'm looking at a few example PKGINFOs > and they contain nothing that can exclusively be generated at > package() or build() time (other than pkgver but that's already the > case currently). > > J. Leclanche Here's a few reasons that come to mind: 1) As you've already discovered, you'd need a .PKGINFO file for every potential architecture, rather than just a .SRCINFO which might describe what architectures are known available. A .SRCINFO could express all package and architecture specific overrides. The fact that an override exists might be a useful bit of information to convey. 2) .PKGINFO doesn't contain things that are useful for source packages, like, say... the source URL? checksums? 3) You can't possibly express split packages well. Having pkgname fields in a .SRCINFO would mean you could describe all packages created by a PKGBUILD rather than having some loose association between a PKGBUILD and some possible .PKGINFO files which it might have generated. This is about *source* packages. The metadata needs for source packages are not the same as the needs for binary packages.
Re: [aur-general] 'provides' info in AUR
On Wed, Dec 11, 2013 at 6:42 PM, Dave Reisner wrote: > On Wed, Dec 11, 2013 at 06:12:59PM +, Jerome Leclanche wrote: >> On Wed, Dec 11, 2013 at 5:52 PM, Sam S. wrote: >> > On Tue, Dec 10, 2013 at 4:21 PM, Jerome Leclanche >> > wrote: >> > >> >> I don't understand why makepkg -S >> >> doesn't include the .PKGINFO file from makepkg (and subsequently the >> >> AUR would use that if present instead of the grep system which fails >> >> as soon as variables/expansions are involved which is every other >> >> package). All the implementation is there. >> >> >> > >> > It would probably be better then what we have now, but a perfect solution >> > would also account for PKGBUILDs that use Bash conditionals to set >> > different variables on i386 systems than on x86_64 systems (which is pretty >> > common among AUR packages that package upstream binaries rather than >> > compiling from source). >> > >> > Reading values from .PKGINFO would populate the AUR with the values for >> > whichever architecture the package uploader happened to use. (So if the >> > maintainer changes, or the same maintainer works on different computers, a >> > simple re-upload of an AUR package could suddenly change the package's >> > meta-info, i.e. the AUR would become more "fragile".) >> > >> > Of course, the "perfect solution" would be pretty difficult to implement. >> > Gentoo had a GSoC project last year [1], to implement an efficient and safe >> > (side-effect free) limited Bash parser / pseudo-interpreter in C++, >> > sufficient to extract all necessary values from Genoo's equivalent of >> > PKGBUILDs. Surely, this could have been useful for the AUR as well. But I >> > can find no evidence of continued project activity after the GSoC period >> > ended, so it appears they have given up... :( >> > >> > --- >> > [1] http://dev.gentoo.org/~qiaomuf/libbash.html >> >> I love the fact someone could be working on a bash parser but that >> solution is *insane*. > > It's designed to be incomplete, and the project appears very dead. > >> This is a solved problem: use a metafile compiled by whatever tools >> you use in your distro/domain that can be parsed safely and easily. >> For Arch, those are PKGINFOs. > > No, go see historical conversations about a mythical .SRCINFO -- this is > what .AURINFO is based on. .SRCINFO is vaporware right now, and I again > refer you to unresolved discussions about how it would handle split > packages and package-specific overrides. > >> Good point on the differences per arch. I guess the obvious solution >> that comes to mind here is to have makepkg -S generate the source >> files for each arch value (eg. PKGINFO.x86_64, etc) but that's not >> necessarily good and is the subject of another discussion imo. > > To be clear, .PKGINFO is not the solution here. This file describes a > built package (something the AUR explicitly does not support). > > d Care to explain the reasoning? I'm looking at a few example PKGINFOs and they contain nothing that can exclusively be generated at package() or build() time (other than pkgver but that's already the case currently). J. Leclanche
Re: [aur-general] 'provides' info in AUR
On Wed, Dec 11, 2013 at 06:12:59PM +, Jerome Leclanche wrote: > On Wed, Dec 11, 2013 at 5:52 PM, Sam S. wrote: > > On Tue, Dec 10, 2013 at 4:21 PM, Jerome Leclanche wrote: > > > >> I don't understand why makepkg -S > >> doesn't include the .PKGINFO file from makepkg (and subsequently the > >> AUR would use that if present instead of the grep system which fails > >> as soon as variables/expansions are involved which is every other > >> package). All the implementation is there. > >> > > > > It would probably be better then what we have now, but a perfect solution > > would also account for PKGBUILDs that use Bash conditionals to set > > different variables on i386 systems than on x86_64 systems (which is pretty > > common among AUR packages that package upstream binaries rather than > > compiling from source). > > > > Reading values from .PKGINFO would populate the AUR with the values for > > whichever architecture the package uploader happened to use. (So if the > > maintainer changes, or the same maintainer works on different computers, a > > simple re-upload of an AUR package could suddenly change the package's > > meta-info, i.e. the AUR would become more "fragile".) > > > > Of course, the "perfect solution" would be pretty difficult to implement. > > Gentoo had a GSoC project last year [1], to implement an efficient and safe > > (side-effect free) limited Bash parser / pseudo-interpreter in C++, > > sufficient to extract all necessary values from Genoo's equivalent of > > PKGBUILDs. Surely, this could have been useful for the AUR as well. But I > > can find no evidence of continued project activity after the GSoC period > > ended, so it appears they have given up... :( > > > > --- > > [1] http://dev.gentoo.org/~qiaomuf/libbash.html > > I love the fact someone could be working on a bash parser but that > solution is *insane*. It's designed to be incomplete, and the project appears very dead. > This is a solved problem: use a metafile compiled by whatever tools > you use in your distro/domain that can be parsed safely and easily. > For Arch, those are PKGINFOs. No, go see historical conversations about a mythical .SRCINFO -- this is what .AURINFO is based on. .SRCINFO is vaporware right now, and I again refer you to unresolved discussions about how it would handle split packages and package-specific overrides. > Good point on the differences per arch. I guess the obvious solution > that comes to mind here is to have makepkg -S generate the source > files for each arch value (eg. PKGINFO.x86_64, etc) but that's not > necessarily good and is the subject of another discussion imo. To be clear, .PKGINFO is not the solution here. This file describes a built package (something the AUR explicitly does not support). d
Re: [aur-general] 'provides' info in AUR
On Wed, Dec 11, 2013 at 5:52 PM, Sam S. wrote: > On Tue, Dec 10, 2013 at 4:21 PM, Jerome Leclanche wrote: > >> I don't understand why makepkg -S >> doesn't include the .PKGINFO file from makepkg (and subsequently the >> AUR would use that if present instead of the grep system which fails >> as soon as variables/expansions are involved which is every other >> package). All the implementation is there. >> > > It would probably be better then what we have now, but a perfect solution > would also account for PKGBUILDs that use Bash conditionals to set > different variables on i386 systems than on x86_64 systems (which is pretty > common among AUR packages that package upstream binaries rather than > compiling from source). > > Reading values from .PKGINFO would populate the AUR with the values for > whichever architecture the package uploader happened to use. (So if the > maintainer changes, or the same maintainer works on different computers, a > simple re-upload of an AUR package could suddenly change the package's > meta-info, i.e. the AUR would become more "fragile".) > > Of course, the "perfect solution" would be pretty difficult to implement. > Gentoo had a GSoC project last year [1], to implement an efficient and safe > (side-effect free) limited Bash parser / pseudo-interpreter in C++, > sufficient to extract all necessary values from Genoo's equivalent of > PKGBUILDs. Surely, this could have been useful for the AUR as well. But I > can find no evidence of continued project activity after the GSoC period > ended, so it appears they have given up... :( > > --- > [1] http://dev.gentoo.org/~qiaomuf/libbash.html I love the fact someone could be working on a bash parser but that solution is *insane*. This is a solved problem: use a metafile compiled by whatever tools you use in your distro/domain that can be parsed safely and easily. For Arch, those are PKGINFOs. Good point on the differences per arch. I guess the obvious solution that comes to mind here is to have makepkg -S generate the source files for each arch value (eg. PKGINFO.x86_64, etc) but that's not necessarily good and is the subject of another discussion imo.
Re: [aur-general] 'provides' info in AUR
On Tue, Dec 10, 2013 at 4:21 PM, Jerome Leclanche wrote: > I don't understand why makepkg -S > doesn't include the .PKGINFO file from makepkg (and subsequently the > AUR would use that if present instead of the grep system which fails > as soon as variables/expansions are involved which is every other > package). All the implementation is there. > It would probably be better then what we have now, but a perfect solution would also account for PKGBUILDs that use Bash conditionals to set different variables on i386 systems than on x86_64 systems (which is pretty common among AUR packages that package upstream binaries rather than compiling from source). Reading values from .PKGINFO would populate the AUR with the values for whichever architecture the package uploader happened to use. (So if the maintainer changes, or the same maintainer works on different computers, a simple re-upload of an AUR package could suddenly change the package's meta-info, i.e. the AUR would become more "fragile".) Of course, the "perfect solution" would be pretty difficult to implement. Gentoo had a GSoC project last year [1], to implement an efficient and safe (side-effect free) limited Bash parser / pseudo-interpreter in C++, sufficient to extract all necessary values from Genoo's equivalent of PKGBUILDs. Surely, this could have been useful for the AUR as well. But I can find no evidence of continued project activity after the GSoC period ended, so it appears they have given up... :( --- [1] http://dev.gentoo.org/~qiaomuf/libbash.html
Re: [aur-general] 'provides' info in AUR
2013/12/10 Jerome Leclanche : > On Tue, Dec 10, 2013 at 3:16 PM, 郑文辉(Techlive Zheng) > wrote: >> 2013/12/10 Dave Reisner : >>> On Tue, Dec 10, 2013 at 10:54:42PM +0800, 郑文辉(Techlive Zheng) wrote: 2013/12/10 Dave Reisner : > On Tue, Dec 10, 2013 at 10:45:58PM +0800, 郑文辉(Techlive Zheng) wrote: >> 2013/12/10 Karol Blazewicz : >> > On Tue, Dec 10, 2013 at 2:18 PM, 郑文辉(Techlive Zheng) >> > wrote: >> >> Hey, I wonder if it is possible to query the `provides` of a package >> >> in AUR? >> > >> > Is e.g. 'cower -ii wine-git' good enough? >> >> Acctually, I want to know if it is possible to query the AUR through >> API to know what packages provide the desired package. e.g. I query >> for packages provived 'wine', and get I get 'wine-git' and others. >> Does cower do that? > > No, cower doesn't do this because the AUR's RPC interface doesn't do > this. Okay, understand, so that's a todo, I guess. >>> >>> Well, sure... but I suspect this will never make it into the AUR, given >>> the current implementation of a lot of things. For a universal solution, >>> you'd need to: >>> >>> 1) Extend the PKGBUILD parser to parse provides. This alone is >>> problematic since people insist on dumping split packages on the AUR. >>> There's also plenty of reasons not to extend the PKGBUILD parser in its >>> current form. >>> 2) Extend the DB schema to store and relate the newly parsed provides. >>> 3) Extend RPC responses to include the provides in info/search responses. >>> 4) Add a flag to the search API to allow searching by providers (because >>> this shouldn't be the default behavior, lest you break existing tools). >>> 5) Reparse every single package in the AUR so that steps 1-4 are >>> actually meaningful. >>> >>> The half-assed solution would be to draw provides data from .AURINFO, >>> but then you rely on submitters to do the right thing. Inevitably, this >> How about change `makepkg --source` to let it generate a PHP-parsing friendly >> format like the `ini` format in the final src tarball, doss this >> solves everything? > > That's exactly what .PKGINFO files are and it's imho the only sane way > to do this. Problem are, they're not generated by -S. > >>> would leave you with a useless "feature" as only a fraction of >>> applicable packages would have the data. >>> That shold be an easy fix, I guess, someone just need to acctually submit the patch and discuss it with Allan.
Re: [aur-general] 'provides' info in AUR
On Tue, Dec 10, 2013 at 3:16 PM, 郑文辉(Techlive Zheng) wrote: > 2013/12/10 Dave Reisner : >> On Tue, Dec 10, 2013 at 10:54:42PM +0800, 郑文辉(Techlive Zheng) wrote: >>> 2013/12/10 Dave Reisner : >>> > On Tue, Dec 10, 2013 at 10:45:58PM +0800, 郑文辉(Techlive Zheng) wrote: >>> >> 2013/12/10 Karol Blazewicz : >>> >> > On Tue, Dec 10, 2013 at 2:18 PM, 郑文辉(Techlive Zheng) >>> >> > wrote: >>> >> >> Hey, I wonder if it is possible to query the `provides` of a package >>> >> >> in AUR? >>> >> > >>> >> > Is e.g. 'cower -ii wine-git' good enough? >>> >> >>> >> Acctually, I want to know if it is possible to query the AUR through >>> >> API to know what packages provide the desired package. e.g. I query >>> >> for packages provived 'wine', and get I get 'wine-git' and others. >>> >> Does cower do that? >>> > >>> > No, cower doesn't do this because the AUR's RPC interface doesn't do this. >>> >>> >>> Okay, understand, so that's a todo, I guess. >> >> Well, sure... but I suspect this will never make it into the AUR, given >> the current implementation of a lot of things. For a universal solution, >> you'd need to: >> >> 1) Extend the PKGBUILD parser to parse provides. This alone is >> problematic since people insist on dumping split packages on the AUR. >> There's also plenty of reasons not to extend the PKGBUILD parser in its >> current form. >> 2) Extend the DB schema to store and relate the newly parsed provides. >> 3) Extend RPC responses to include the provides in info/search responses. >> 4) Add a flag to the search API to allow searching by providers (because >> this shouldn't be the default behavior, lest you break existing tools). >> 5) Reparse every single package in the AUR so that steps 1-4 are >> actually meaningful. >> >> The half-assed solution would be to draw provides data from .AURINFO, >> but then you rely on submitters to do the right thing. Inevitably, this > How about change `makepkg --source` to let it generate a PHP-parsing friendly > format like the `ini` format in the final src tarball, doss this > solves everything? That's exactly what .PKGINFO files are and it's imho the only sane way to do this. Problem are, they're not generated by -S. >> would leave you with a useless "feature" as only a fraction of >> applicable packages would have the data. >> >> d
Re: [aur-general] 'provides' info in AUR
On Tue, Dec 10, 2013 at 3:09 PM, Dave Reisner wrote: > On Tue, Dec 10, 2013 at 10:54:42PM +0800, 郑文辉(Techlive Zheng) wrote: >> 2013/12/10 Dave Reisner : >> > On Tue, Dec 10, 2013 at 10:45:58PM +0800, 郑文辉(Techlive Zheng) wrote: >> >> 2013/12/10 Karol Blazewicz : >> >> > On Tue, Dec 10, 2013 at 2:18 PM, 郑文辉(Techlive Zheng) >> >> > wrote: >> >> >> Hey, I wonder if it is possible to query the `provides` of a package >> >> >> in AUR? >> >> > >> >> > Is e.g. 'cower -ii wine-git' good enough? >> >> >> >> Acctually, I want to know if it is possible to query the AUR through >> >> API to know what packages provide the desired package. e.g. I query >> >> for packages provived 'wine', and get I get 'wine-git' and others. >> >> Does cower do that? >> > >> > No, cower doesn't do this because the AUR's RPC interface doesn't do this. >> >> >> Okay, understand, so that's a todo, I guess. > > Well, sure... but I suspect this will never make it into the AUR, given > the current implementation of a lot of things. For a universal solution, > you'd need to: > > 1) Extend the PKGBUILD parser to parse provides. This alone is > problematic since people insist on dumping split packages on the AUR. > There's also plenty of reasons not to extend the PKGBUILD parser in its > current form. This was discussed previously and I don't understand why makepkg -S doesn't include the .PKGINFO file from makepkg (and subsequently the AUR would use that if present instead of the grep system which fails as soon as variables/expansions are involved which is every other package). All the implementation is there. > 2) Extend the DB schema to store and relate the newly parsed provides. > 3) Extend RPC responses to include the provides in info/search responses. > 4) Add a flag to the search API to allow searching by providers (because > this shouldn't be the default behavior, lest you break existing tools). > 5) Reparse every single package in the AUR so that steps 1-4 are > actually meaningful. > > The half-assed solution would be to draw provides data from .AURINFO, > but then you rely on submitters to do the right thing. Inevitably, this > would leave you with a useless "feature" as only a fraction of > applicable packages would have the data. > > d
Re: [aur-general] 'provides' info in AUR
2013/12/10 Dave Reisner : > On Tue, Dec 10, 2013 at 10:54:42PM +0800, 郑文辉(Techlive Zheng) wrote: >> 2013/12/10 Dave Reisner : >> > On Tue, Dec 10, 2013 at 10:45:58PM +0800, 郑文辉(Techlive Zheng) wrote: >> >> 2013/12/10 Karol Blazewicz : >> >> > On Tue, Dec 10, 2013 at 2:18 PM, 郑文辉(Techlive Zheng) >> >> > wrote: >> >> >> Hey, I wonder if it is possible to query the `provides` of a package >> >> >> in AUR? >> >> > >> >> > Is e.g. 'cower -ii wine-git' good enough? >> >> >> >> Acctually, I want to know if it is possible to query the AUR through >> >> API to know what packages provide the desired package. e.g. I query >> >> for packages provived 'wine', and get I get 'wine-git' and others. >> >> Does cower do that? >> > >> > No, cower doesn't do this because the AUR's RPC interface doesn't do this. >> >> >> Okay, understand, so that's a todo, I guess. > > Well, sure... but I suspect this will never make it into the AUR, given > the current implementation of a lot of things. For a universal solution, > you'd need to: > > 1) Extend the PKGBUILD parser to parse provides. This alone is > problematic since people insist on dumping split packages on the AUR. > There's also plenty of reasons not to extend the PKGBUILD parser in its > current form. > 2) Extend the DB schema to store and relate the newly parsed provides. > 3) Extend RPC responses to include the provides in info/search responses. > 4) Add a flag to the search API to allow searching by providers (because > this shouldn't be the default behavior, lest you break existing tools). > 5) Reparse every single package in the AUR so that steps 1-4 are > actually meaningful. > > The half-assed solution would be to draw provides data from .AURINFO, > but then you rely on submitters to do the right thing. Inevitably, this How about change `makepkg --source` to let it generate a PHP-parsing friendly format like the `ini` format in the final src tarball, doss this solves everything? > would leave you with a useless "feature" as only a fraction of > applicable packages would have the data. > > d
Re: [aur-general] 'provides' info in AUR
On Tue, Dec 10, 2013 at 10:54:42PM +0800, 郑文辉(Techlive Zheng) wrote: > 2013/12/10 Dave Reisner : > > On Tue, Dec 10, 2013 at 10:45:58PM +0800, 郑文辉(Techlive Zheng) wrote: > >> 2013/12/10 Karol Blazewicz : > >> > On Tue, Dec 10, 2013 at 2:18 PM, 郑文辉(Techlive Zheng) > >> > wrote: > >> >> Hey, I wonder if it is possible to query the `provides` of a package in > >> >> AUR? > >> > > >> > Is e.g. 'cower -ii wine-git' good enough? > >> > >> Acctually, I want to know if it is possible to query the AUR through > >> API to know what packages provide the desired package. e.g. I query > >> for packages provived 'wine', and get I get 'wine-git' and others. > >> Does cower do that? > > > > No, cower doesn't do this because the AUR's RPC interface doesn't do this. > > > Okay, understand, so that's a todo, I guess. Well, sure... but I suspect this will never make it into the AUR, given the current implementation of a lot of things. For a universal solution, you'd need to: 1) Extend the PKGBUILD parser to parse provides. This alone is problematic since people insist on dumping split packages on the AUR. There's also plenty of reasons not to extend the PKGBUILD parser in its current form. 2) Extend the DB schema to store and relate the newly parsed provides. 3) Extend RPC responses to include the provides in info/search responses. 4) Add a flag to the search API to allow searching by providers (because this shouldn't be the default behavior, lest you break existing tools). 5) Reparse every single package in the AUR so that steps 1-4 are actually meaningful. The half-assed solution would be to draw provides data from .AURINFO, but then you rely on submitters to do the right thing. Inevitably, this would leave you with a useless "feature" as only a fraction of applicable packages would have the data. d
Re: [aur-general] 'provides' info in AUR
2013/12/10 Dave Reisner : > On Tue, Dec 10, 2013 at 10:45:58PM +0800, 郑文辉(Techlive Zheng) wrote: >> 2013/12/10 Karol Blazewicz : >> > On Tue, Dec 10, 2013 at 2:18 PM, 郑文辉(Techlive Zheng) >> > wrote: >> >> Hey, I wonder if it is possible to query the `provides` of a package in >> >> AUR? >> > >> > Is e.g. 'cower -ii wine-git' good enough? >> >> Acctually, I want to know if it is possible to query the AUR through >> API to know what packages provide the desired package. e.g. I query >> for packages provived 'wine', and get I get 'wine-git' and others. >> Does cower do that? > > No, cower doesn't do this because the AUR's RPC interface doesn't do this. Okay, understand, so that's a todo, I guess.
Re: [aur-general] 'provides' info in AUR
On Tue, Dec 10, 2013 at 10:45:58PM +0800, 郑文辉(Techlive Zheng) wrote: > 2013/12/10 Karol Blazewicz : > > On Tue, Dec 10, 2013 at 2:18 PM, 郑文辉(Techlive Zheng) > > wrote: > >> Hey, I wonder if it is possible to query the `provides` of a package in > >> AUR? > > > > Is e.g. 'cower -ii wine-git' good enough? > > Acctually, I want to know if it is possible to query the AUR through > API to know what packages provide the desired package. e.g. I query > for packages provived 'wine', and get I get 'wine-git' and others. > Does cower do that? No, cower doesn't do this because the AUR's RPC interface doesn't do this.
Re: [aur-general] 'provides' info in AUR
2013/12/10 Karol Blazewicz : > On Tue, Dec 10, 2013 at 2:18 PM, 郑文辉(Techlive Zheng) > wrote: >> Hey, I wonder if it is possible to query the `provides` of a package in AUR? > > Is e.g. 'cower -ii wine-git' good enough? Acctually, I want to know if it is possible to query the AUR through API to know what packages provide the desired package. e.g. I query for packages provived 'wine', and get I get 'wine-git' and others. Does cower do that?
Re: [aur-general] 'provides' info in AUR
On Tue, Dec 10, 2013 at 2:18 PM, 郑文辉(Techlive Zheng) wrote: > Hey, I wonder if it is possible to query the `provides` of a package in AUR? Is e.g. 'cower -ii wine-git' good enough?
[aur-general] 'provides' info in AUR
Hey, I wonder if it is possible to query the `provides` of a package in AUR?
