Re: Vendor-based customization of Lintian (or profiles)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 2011-04-19 20:23, Neil Williams wrote: > On Tue, 19 Apr 2011 20:04:47 +0200 > Niels Thykier wrote: > Hi again, A little update on this. >>> When we agree on (the basis of) an implementation (not necessarily the >>> one proposed here) I suggest we add it to the Debian wiki under >>> Lintian/Spec/VendorCustomization (or similar). >>> This way the specification does not suddenly get lost on the mailing >>> list and we can easily update it later. I have created: http://wiki.debian.org/Lintian/Spec/VendorCustomization > > Any patches? > :-) > Yes! Check the vendor-profile branch from the Lintian repository. Caveat 1: All Debian based profiles are currently auto-generated and said auto-generation is not included in the repository. To generate them, simply run: $ debian/rules profiles It generates a profile based on all the checks in checks/ - If you add a new check file, you will need to re-run said target - but it is not needed if you just add a new tag to an existing check. The ubuntu profile is just a quick sample; if there are any Lintian-savvy Ubuntu people, feel free to send me a patch for this one. Caveat 2: The solution and the implementation disagree in some cases - most likely because I have not implemented the solution properly yet. >>> [Needed] >>> * No code changes: >>>- It must be possible to alter whether Lintian emits a tag >>> without modifying code. >>> * Re-usable: >>>- Most profiles will likely be based on the core profile. New >>> profiles should be able to "extend" existing profile. >>>- Extended profile must be able to include tags not originally >>> in the original profile. >>>- Extended profile must be able to exclude tags originally in the >>> original profile. >>> [...] >>> * A tag not included in the current profile shall not be emitted. >>>- An override for such a tag is not considered "unneeded" and >>> must be ignored. As far as I can tell, all of this works as it should. If you experience otherwise, do let me know. Exception: """ In the absence of the fields Extends, Enable-Tag, Disable-Tag, Enable-Check and Disable-Check, the profile shall simply include all checks/tags known to it. """ This does not hold yet, so a minimal profile currently implies no tags/checks enabled. > I think there should be support for dpkg-vendor in this situation. If > DEB_VENDOR is defined, lintian can look for that profile. Emdebian has > been doing this for some time with emvendor but not (yet) with > lintian. > > [...] > > IMHO DEB_VENDOR is a better fit. > Lintian will check DEB_VENDOR in the branch, however DEB_VENDOR is only used if --profile is not used and LINTIAN_PROFILE is not set as an environment variable or from the config file. It is possible this order will change later. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBCAAGBQJNs/JxAAoJEAVLu599gGRCCtEP/1JBsGmLWEqVQN8RBArLafUV i+LCAUULM0vi6xzoHtVMpzqE+VRxinM7+3vEBJz2napP9EvJhdSl2iJO4XFXtA/W gHhAYejZ0tZULfaCufcXbWy6Om9ZZBCJzZeBBKukSIonrE1RKQla1FLQR4NW7XzA HGxep2sOXZkv0crleBAqnA1FyQouqnItIcFtUOjcFMCUaVgQso+nM7WF7EsspEIv Quzzw5x6hqKkX0eitusAFDs+I2CiLyixR7uRTCEQbgDiJpjTQZkYTLdkoF6SMUbc nnIbAvl9WIh5qC9CdI7ppCSgxtKqJtJtkKe1R0Ea3qHW4syC03IYzTlZhEico2Cv 1+uk7sGHe7NC4Xs3rqsWfbjc9OjsHdLoVfE+3Dac66ihXOpytiMp9YU3uyzZGRQf siY3IheS4oa/Lhxuz3oRK7r3J2Xfo2ATaiFBBae/cxY5qM+BBW8MOrieOX0oQ2eu X5zffQtSDOO2el6kdM9rL5yjSRSz55WRJ4PANm/axe9YyIA/UbIDe3/Pjwh9Jsgu B9XSS0iHv970+eNvs5PWBsxRdGp03sEHvqsB042HIkCVGLzgNnJOhrylnniXxEQf 0p88HV/340JcDUo6XC59HhtSwbh+EqpP4HSzmWR+9lUhCVZWkwNO+Az+zJnlmolj gIRcuyYprisZJjKUdf5Y =5nIZ -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4db3f272.5050...@thykier.net
Re: Vendor-based customization of Lintian (or profiles)
On Tue, Apr 19, 2011 at 06:14:25PM +0200, Niels Thykier wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Hi > > Considering that #513663 will be closed in 2.5.0~rc3, I figured it was > time to start a new branch. XD w00t! > I was considering to do either Vendor-based customization or support > third party checks (e.g. Lintian "extra"), as the subject suggests I > think I will start with Vendor-based Lintian. > However if you think third party checks should be given priority, let > me know - this decision is not set in stone. :) I strongly feel vendor based customizations should be the Next Big Thing. I started a fork of lintian in Maemo, called maemian, but it quickly bogged down due to the complexities you mention in Ubuntu. The point is that I think this would be welcome amongst derivatives and improve the quality of derivative package and ultimately Debian. I'm very glad that you're looking into doing this Niels - thank you. And thank you for your work on lintian. To be honest, I've rarely seen more productive programmers than yourself! Thanks! Jeremiah -- To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110419210251.GA7526@localhost
Re: Vendor-based customization of Lintian (or profiles)
On Tue, 19 Apr 2011 21:02:27 +0200 Niels Thykier wrote: > > Emdebian tried this and came up against a few hidden assumptions in > > lintian which we discussed with the lintian maintainers at the time. We > > experimented with an emdebian checks file and desc file: > > > > http://packages.debian.org/sid/all/emdebian-crush/filelist > > > > If this proposal completes the final stage of that process, I will be > > v.happy. :-) > > Do you have a link to that debate; unfortunately I believe it was before > my involvement with Lintian, so I do not believe I have seen it (unless > it is covered in the DC10 notes from the BoF). Not really. I did describe some of the results to the Emdebian folks: http://lists.debian.org/debian-embedded/2008/04/msg3.html http://lists.debian.org/debian-embedded/2009/12/msg00032.html The other discussions with Russ were mostly direct email and/or discussions at DebConf9. -- Neil Williams = http://www.linux.codehelp.co.uk/ pgpKvBnkJLj0z.pgp Description: PGP signature
Re: Vendor-based customization of Lintian (or profiles)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 2011-04-19 20:23, Neil Williams wrote: > On Tue, 19 Apr 2011 20:04:47 +0200 > Niels Thykier wrote: > >>> [Needed] >>> * No code changes: >>>- It must be possible to alter whether Lintian emits a tag without >>> modifying code. >>> * Re-usable: >>>- Most profiles will likely be based on the core profile. New >>> profiles should be able to "extend" existing profile. >>>- Extended profile must be able to include tags not originally >>> in the original profile. >>>- Extended profile must be able to exclude tags originally in the >>> original profile. >>> * Add vendor/3rd party checks [Deferred]: >>>- A profile may include new checks/collections not present in the >>> Lintian package itself. >>> * A tag not included in the current profile shall not be emitted. >>>- An override for such a tag is not considered "unneeded" and must be >>> ignored. > > Emdebian tried this and came up against a few hidden assumptions in > lintian which we discussed with the lintian maintainers at the time. We > experimented with an emdebian checks file and desc file: > > http://packages.debian.org/sid/all/emdebian-crush/filelist > > If this proposal completes the final stage of that process, I will be > v.happy. :-) > Do you have a link to that debate; unfortunately I believe it was before my involvement with Lintian, so I do not believe I have seen it (unless it is covered in the DC10 notes from the BoF). > Another common requirement at work is to switch off the lintian warning > about "unsupported distributions" as we're trying to build > lintian-clean packages for our own internal repositories and it makes > apt pinning a lot easier to *not* use stable, testing, unstable or any > of the Debian ToyStory codenames. > > If that can be done with a simple file which applies across all > packages, it will just be so much easier. > Indeed :) >>> Rationale for deferring "Add checks": Fixing this is effectively to fix >>> the "third party checks" problem and this adds complexity like "how to >>> handle name clashing with checks/collections". We will have that debate >>> eventually. It is my belief that it will be easier for us to add the >>> third party checks on top on the profile system than the other way >>> around (infrastructure wise). > > That makes the third-party responsible for avoiding clashes - which is, > IMHO, the way it should work. > Maybe - I am still reluctant to include third party checks in this specification; partly because the current development has changed the below mentioned check/collection "API" and partly to keep the "size" of this specification down. >>> Also, as soon as we add third party checks our current collections as >>> well as our check interface becomes part of our API. Changing any of >>> these could possibly break other packages. >>> >>> Proposed solution >>> - >>> We add a new directory to the Lintian root called "profiles" with the >>> structure: >>> >>> profiles/ >>>debian/ >>> main.profile >>> ftp-auto-reject.profile >>>ubuntu/ >>> main.profile >>>$other_vendor/ >>> some.profile >>>default > > I think there should be support for dpkg-vendor in this situation. If > DEB_VENDOR is defined, lintian can look for that profile. Emdebian has > been doing this for some time with emvendor but not (yet) with lintian. > >>> Profiles should be specified with a new command line argument --profile >>> or the environment/lintianrc variable LINTIAN_PROFILE=. > > IMHO DEB_VENDOR is a better fit. > Hmm, dpkg-vendor/DEB_VENDOR could be a valid alternative, but I think I would like to keep LINTIAN_PROFILE as well. This would allow users/sysadmins to set their own profile that is completely unrelated to any known vendor without breaking DEB_VENDOR sensitive applications/build tools. >>> When we agree on (the basis of) an implementation (not necessarily the >>> one proposed here) I suggest we add it to the Debian wiki under >>> Lintian/Spec/VendorCustomization (or similar). >>> This way the specification does not suddenly get lost on the mailing >>> list and we can easily update it later. > > Any patches? > :-) > Unfortunately not at the moment; I wanted a bit of feedback before I started working on this. But when I start the work, I will push a branch to the lintian git. ~Niels -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBCAAGBQJNrdxDAAoJEAVLu599gGRCjkcQAJGuBLAd3Wiuuv2++4VQQPUn b3Kj1RrTD9tUSXQY/TsJoOngHOY8XMwwpoECn11ZBYsttA26Pi47wokRljpYALEJ R5KfMgBXOC04F7bxSgof1ZsQY5yuv8srO/tlRZHkSdwgYuNbCj2pSGeTSAPsR6Su 7X7rxJQEijZWTQwxzdZsch2BLXX2P8C0NC4ZlTthjram48Io/r512KBw2Bkn4M04 FTbj4zMWnMbR4uN2fiVmbaC2EgxuF6nc5b2S0nSZA7aGGTX/x/7vFuZ98mgmVCSf ZDvdeA2C8tOFvShKYfajIOKmni56MBq6YKrrNrbU5IBQLgHCtI3Iiab2oxqTEtp0 lGadM439nkGc8IbBLDmJeB2Z8KffxAhrjD
Re: Vendor-based customization of Lintian (or profiles)
On Tue, 19 Apr 2011 20:04:47 +0200 Niels Thykier wrote: > > [Needed] > > * No code changes: > >- It must be possible to alter whether Lintian emits a tag without > > modifying code. > > * Re-usable: > >- Most profiles will likely be based on the core profile. New > > profiles should be able to "extend" existing profile. > >- Extended profile must be able to include tags not originally > > in the original profile. > >- Extended profile must be able to exclude tags originally in the > > original profile. > > * Add vendor/3rd party checks [Deferred]: > >- A profile may include new checks/collections not present in the > > Lintian package itself. > > * A tag not included in the current profile shall not be emitted. > >- An override for such a tag is not considered "unneeded" and must be > > ignored. Emdebian tried this and came up against a few hidden assumptions in lintian which we discussed with the lintian maintainers at the time. We experimented with an emdebian checks file and desc file: http://packages.debian.org/sid/all/emdebian-crush/filelist If this proposal completes the final stage of that process, I will be v.happy. :-) Another common requirement at work is to switch off the lintian warning about "unsupported distributions" as we're trying to build lintian-clean packages for our own internal repositories and it makes apt pinning a lot easier to *not* use stable, testing, unstable or any of the Debian ToyStory codenames. If that can be done with a simple file which applies across all packages, it will just be so much easier. > > Rationale for deferring "Add checks": Fixing this is effectively to fix > > the "third party checks" problem and this adds complexity like "how to > > handle name clashing with checks/collections". We will have that debate > > eventually. It is my belief that it will be easier for us to add the > > third party checks on top on the profile system than the other way > > around (infrastructure wise). That makes the third-party responsible for avoiding clashes - which is, IMHO, the way it should work. > > Also, as soon as we add third party checks our current collections as > > well as our check interface becomes part of our API. Changing any of > > these could possibly break other packages. > > > > Proposed solution > > - > > We add a new directory to the Lintian root called "profiles" with the > > structure: > > > > profiles/ > >debian/ > > main.profile > > ftp-auto-reject.profile > >ubuntu/ > > main.profile > >$other_vendor/ > > some.profile > >default I think there should be support for dpkg-vendor in this situation. If DEB_VENDOR is defined, lintian can look for that profile. Emdebian has been doing this for some time with emvendor but not (yet) with lintian. > > Profiles should be specified with a new command line argument --profile > > or the environment/lintianrc variable LINTIAN_PROFILE=. IMHO DEB_VENDOR is a better fit. > > When we agree on (the basis of) an implementation (not necessarily the > > one proposed here) I suggest we add it to the Debian wiki under > > Lintian/Spec/VendorCustomization (or similar). > > This way the specification does not suddenly get lost on the mailing > > list and we can easily update it later. Any patches? :-) -- Neil Williams = http://www.linux.codehelp.co.uk/ pgpsoi6oCLH7K.pgp Description: PGP signature
Re: Vendor-based customization of Lintian (or profiles)
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Hi (again for some)
I realised after sending my proposal that it would probably be a good
idea to (ab)use the d-derivatives list to get more opinions on this
proposal (quoted in full below).
I have set Reply-To to lintian-maint@d.o as I would like the debate on
this topic to stay on the lintian list. The web-archive for the
proposal starts at [1].
Thank you in advance,
~Niels
[1] http://lists.debian.org/debian-lint-maint/2011/04/msg00277.html
On 2011-04-19 18:14, Niels Thykier wrote:
> Hi
>
> Considering that #513663 will be closed in 2.5.0~rc3, I figured it was
> time to start a new branch. XD
> I was considering to do either Vendor-based customization or support
> third party checks (e.g. Lintian "extra"), as the subject suggests I
> think I will start with Vendor-based Lintian.
> However if you think third party checks should be given priority, let
> me know - this decision is not set in stone. :)
>
>
> Introduction
>
> The general idea of this has been to allow Debian derivatives or/and
> Debian blends to use Lintian without modifying the code. Currently
> Ubuntu has to patch Lintian every time they import because their
> requirements are slightly different than Debians.
> In the past it was just to remove a line of code here or there in the
> checks. However, now that the test-suite has been enabled they have also
> been required to update some of the tests to match these changes.
>
> If it sounds familiar then it is probably because the particular idea
> has been debated before at DebConf10[DC10] (and possibly at other times
> and places).
>
>
> Requirements
>
> This is a list of things that the solution should handle (the "how"-part
> comes later). If you feel the requirements are not reflecting what is
> really needed, do object. These requirements should form the basis for
> any suggested solution.
>
> [Needed]
> * No code changes:
>- It must be possible to alter whether Lintian emits a tag without
> modifying code.
> * Re-usable:
>- Most profiles will likely be based on the core profile. New
> profiles should be able to "extend" existing profile.
>- Extended profile must be able to include tags not originally
> in the original profile.
>- Extended profile must be able to exclude tags originally in the
> original profile.
> * Add vendor/3rd party checks [Deferred]:
>- A profile may include new checks/collections not present in the
> Lintian package itself.
> * A tag not included in the current profile shall not be emitted.
>- An override for such a tag is not considered "unneeded" and must be
> ignored.
>
> [Optional/Extra/Ideas]
> * Allow profiles to mark tags as "cannot be overridden"/"not
>overwritable".
>- Support FTP-master auto-reject as a profile.
> * Allow profiles to alter severity (but not certainty).
>- What is serious to us may be minor to others.
>- The --display-level argument should operate in terms of active
> profile's severity and not original severity
> * Allow current tag selection command arguments to affect the profile.
>- suppress-tags{,-from-file} suppresses tags even if the current
> profile would have included them.
>
>
> Rationale for deferring "Add checks": Fixing this is effectively to fix
> the "third party checks" problem and this adds complexity like "how to
> handle name clashing with checks/collections". We will have that debate
> eventually. It is my belief that it will be easier for us to add the
> third party checks on top on the profile system than the other way
> around (infrastructure wise).
> Also, as soon as we add third party checks our current collections as
> well as our check interface becomes part of our API. Changing any of
> these could possibly break other packages.
>
> Original solution
> -
> The original solution involved tagging each tag with the profiles it was
> a part of (mentioned in the follow up for [DC10]). Personally I think
> the solution suffers from a few problems.
>
> - Users/System admins cannot make their own profile without messing in
>LINTIAN_ROOT/checks/*.desc, which is usually root/root unless you use
>--root (or LINTIAN_ROOT=)/some/where/writable.
> - It does not appear to be easy to extend/reuse an existing profile
>without altering said profile.
> - Vendors may have merge conflicts if profiles are added or extended
>further up in the "food chain".
>
>
> Proposed solution
> -
> We add a new directory to the Lintian root called "profiles" with the
> structure:
>
> profiles/
>debian/
> main.profile
> ftp-auto-reject.profile
>ubuntu/
> main.profile
>$other_vendor/
> some.profile
>default
>
> profiles/default would be a symlink to the current default vendor. The
> symlink should probably be generated at build time or install time
> (probably
Vendor-based customization of Lintian (or profiles)
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Hi
Considering that #513663 will be closed in 2.5.0~rc3, I figured it was
time to start a new branch. XD
I was considering to do either Vendor-based customization or support
third party checks (e.g. Lintian "extra"), as the subject suggests I
think I will start with Vendor-based Lintian.
However if you think third party checks should be given priority, let
me know - this decision is not set in stone. :)
Introduction
-
The general idea of this has been to allow Debian derivatives or/and
Debian blends to use Lintian without modifying the code. Currently
Ubuntu has to patch Lintian every time they import because their
requirements are slightly different than Debians.
In the past it was just to remove a line of code here or there in the
checks. However, now that the test-suite has been enabled they have also
been required to update some of the tests to match these changes.
If it sounds familiar then it is probably because the particular idea
has been debated before at DebConf10[DC10] (and possibly at other times
and places).
Requirements
-
This is a list of things that the solution should handle (the "how"-part
comes later). If you feel the requirements are not reflecting what is
really needed, do object. These requirements should form the basis for
any suggested solution.
[Needed]
* No code changes:
- It must be possible to alter whether Lintian emits a tag without
modifying code.
* Re-usable:
- Most profiles will likely be based on the core profile. New
profiles should be able to "extend" existing profile.
- Extended profile must be able to include tags not originally
in the original profile.
- Extended profile must be able to exclude tags originally in the
original profile.
* Add vendor/3rd party checks [Deferred]:
- A profile may include new checks/collections not present in the
Lintian package itself.
* A tag not included in the current profile shall not be emitted.
- An override for such a tag is not considered "unneeded" and must be
ignored.
[Optional/Extra/Ideas]
* Allow profiles to mark tags as "cannot be overridden"/"not
overwritable".
- Support FTP-master auto-reject as a profile.
* Allow profiles to alter severity (but not certainty).
- What is serious to us may be minor to others.
- The --display-level argument should operate in terms of active
profile's severity and not original severity
* Allow current tag selection command arguments to affect the profile.
- suppress-tags{,-from-file} suppresses tags even if the current
profile would have included them.
Rationale for deferring "Add checks": Fixing this is effectively to fix
the "third party checks" problem and this adds complexity like "how to
handle name clashing with checks/collections". We will have that debate
eventually. It is my belief that it will be easier for us to add the
third party checks on top on the profile system than the other way
around (infrastructure wise).
Also, as soon as we add third party checks our current collections as
well as our check interface becomes part of our API. Changing any of
these could possibly break other packages.
Original solution
- -
The original solution involved tagging each tag with the profiles it was
a part of (mentioned in the follow up for [DC10]). Personally I think
the solution suffers from a few problems.
- Users/System admins cannot make their own profile without messing in
LINTIAN_ROOT/checks/*.desc, which is usually root/root unless you use
--root (or LINTIAN_ROOT=)/some/where/writable.
- It does not appear to be easy to extend/reuse an existing profile
without altering said profile.
- Vendors may have merge conflicts if profiles are added or extended
further up in the "food chain".
Proposed solution
- -
We add a new directory to the Lintian root called "profiles" with the
structure:
profiles/
debian/
main.profile
ftp-auto-reject.profile
ubuntu/
main.profile
$other_vendor/
some.profile
default
profiles/default would be a symlink to the current default vendor. The
symlink should probably be generated at build time or install time
(probably the former is the easiest).
The absence of the default symlink should imply debian as default
vendor to keep git clones functional.
Profiles should be specified with a new command line argument --profile
or the environment/lintianrc variable LINTIAN_PROFILE=.
should be specified as $VENDOR/$profile or $VENDOR. The
latter being short for $VENDOR/main. The actual file name of the profile
will be $VENDOR/$profile.profile and should be located in:
~/.lintian/profiles/
/etc/lintian/profiles/
$LINTIAN_ROOT/profiles/
All of these directories (if they exist) are expected to have the same
layout (as described above). Profile names cannot contain ".".
Profile file format
-
