In 2011 Steve Langasek proposed dropping Essential: yes from e2fsprogs.
On Sat, Mar 26, 2011 at 11:47:08AM -0700, Steve Langasek wrote:
> Currently the e2fsprogs package is marked Essential: yes in the archive. Is
> this a historical holdover? I believe e2fsprogs used to ship /sbin/fsck,
> but since 2009 (i.e., util-linux (>= 2.15~rc1-1), which e2fsprogs has a
> pre-depends on), this has been provided by util-linux instead.
>
> The remaining programs provided by e2fsprogs are all specific to the ext*
> family of filesystems, so I don't think meet the definition of Essential any
> longer - their presence is certainly important if you have an ext[234]
> filesystem, but while this is the default, you can have systems that don't
> use ext* at all, which makes e2fsprogs no more essential in nature than the
> other per-filesystem fsck tools.
>
> Now that the transition to util-linux is done in a stable release, is it
> time for us to drop the Essential: yes flag from e2fsprogs? This will
> benefit those targetting embedded systems that don't use ext, where the
> package will be dead weight; the risk of any packages assuming availability
> of these e2fs-specific interfaces without a dependency is quite low; and
> we're at the right point in the cycle to make changes to the Essential set,
> where we have time to deal with any unexpected fallout.
Since then we have fully transitioned to systemd and init has become
non-essential. The issue around pulling e2fsprogs into essential via
logsave has thus solved itself.
I think we should revisit this proposal now that it becomes practical.
To get us going, I have come up with a plan:
1) Analyze which packages would need dependencies on e2fsprogs.
2) File a bug against lintian to stop complaining about e2fsprogs
dependencies.
3) MBF those packages that need an e2fsprogs dependency.
4) Drop Essential: yes from e2fsprogs.
So I thought, "how hard can it be?" All we need to do is grep the
archive for those tools and add those dependencies. So I unpacked sid
main amd64 and grepped[1] each and every file (potentially decompressing
gzip) for those e2fsprogs. The results[2] are 5666 occurrences in 1250
binary packages. From there, I started looking[3] into the actual uses
and filtered common non-uses such as documentation, debug symbols,
kernel images, locales and other stuff. I manually checked the remaining
packages and thus went down[4] to 318 occurrences in 133 binary
packages. Thus I arrive at the final dd-list (attached) for an MBF. We
can now say that our package lists will increase by less than 1.5kb
uncompressed if we make e2fsprogs non-essential. In comparison, the
average binary package weighs 767 bytes. I believe that the method used
is mostly free from false negatives (by looking for bare program names
in each and every file) and has a manageable number of false positives.
I think we can check off the analysis part. How about proceeding with
the lintian bug and the MBF now? Did I miss anything?
Again, we are at the right point in the cycle to handle unexpected
fallout. :)
Helmut
[1] Attached e2fsprogsimport.py + the software behind
https://dedup.debian.net/
[2] http://subdivi.de/~helmut/e2fsprogsimport.result.xz
[3] Attached e2fsprogsanalyze.py
[4] http://subdivi.de/~helmut/e2fsprogsanalyze.result.xz
#!/usr/bin/python3
import argparse
from contextlib import closing
import logging
import multiprocessing
try:
import queue
except ImportError:
import Queue as queue
import sys
try:
from urllib.request import urlopen
except ImportError:
from urllib2 import urlopen
import yaml
import zlib
from debian import deb822
from debian.debian_support import version_compare
from dedup.compression import GzipDecompressor
from dedup.debpkg import DebExtractor, decodetarname
from dedup.filemagic import FileDigester
from dedup.hashing import DecompressedHash, hash_file, SuppressingHash
from dedup.utils import open_compressed_mirror_url
class E2fsprogsAnalyzer:
"""Search for words in a stream of bytes.
@ivar found: is a set of discovered words
"""
def __init__(self, words):
"""
@param words: an iterable object yielding bytes
"""
self.found = set()
self.missing = set(words)
self.overlap = max(map(len, words)) - 1
self.buff = b""
def update(self, data):
self.buff += data
for word in tuple(self.missing):
if word in self.buff:
self.found.add(word)
self.missing.discard(word)
self.buff = self.buff[-self.overlap:]
def hexdigest(self):
return self.found
def copy(self):
new = E2fsprogsAnalyzer(self.missing)
new.found = self.found.copy()
new.overlap = self.overlap
new.buff = self.buff
return new
def make_tuple(*args):
return args
class MultiDigest(object):
def __init__(self, *digesters, postproc=make_tuple):
self.digesters = digesters
self.postproc = postproc
def update(self, buff):
for digester in self.digesters:
digester.update(buff)
def hexdigest(self):
return self.postproc(*(digester.hexdigest()
for digester in self.digesters))
def copy(self):
return MultiDigest(*(digester.copy() for digester in self.digesters),
postproc=self.postproc)
def make_e2fsprogs_analyzer():
programs = set(b"""badblocks chattr debugfs dumpe2fs e2freefrag e2fsck
e2image e2label e2undo e4crypt e4defrag filefrag fsck.ext2
fsck.ext3 fsck.ext4 logsave lsattr mke2fs mkfs.ext2
mkfs.ext3 mkfs.ext4 mklost+found resize2fs
tune2fs""".split())
def postproc(found, magicguess):
return (set(f.decode("ascii") for f in found), magicguess)
return MultiDigest(E2fsprogsAnalyzer(programs), FileDigester(),
postproc=postproc)
def make_compressed_e2fsprogs_analyzer():
d = make_e2fsprogs_analyzer()
zd = SuppressingHash(DecompressedHash(GzipDecompressor(), d.copy()),
(ValueError, zlib.error))
def postproc(zh, h):
if zh:
return zh + (True,)
return h + (False,)
return MultiDigest(zd, d, postproc=postproc)
class ProcessingFinished(Exception):
pass
class E2fsprogsExtractor(DebExtractor):
def __init__(self):
DebExtractor.__init__(self)
self.files = dict()
def handle_control_member(self, name, content):
a = make_e2fsprogs_analyzer()
a.update(content)
found, magicguess = a.hexdigest()
if found:
self.files["DEBIAN/%s" % name] = (found, magicguess, False)
def handle_data_tar(self, tarfileobj):
for elem in tarfileobj:
if not elem.isreg():
continue
try:
name = decodetarname(elem.name)
except UnicodeDecodeError:
logging.warning("skipping filename with encoding error %r",
elem.name)
continue # skip files with non-utf8 encoding for now
a = make_compressed_e2fsprogs_analyzer()
hash_file(a, tarfileobj.extractfile(elem))
found, magicguess, compressed = a.hexdigest()
if found:
self.files[name] = (found, magicguess, compressed)
raise ProcessingFinished
def process_one_package(item):
pkg, url = item
try:
extractor = E2fsprogsExtractor()
with closing(urlopen(url)) as pkgfile:
try:
extractor.process(pkgfile)
except ProcessingFinished:
pass
return (pkg, extractor.files)
except:
logging.exception("while processing %s", pkg)
return (pkg, {})
def consume_items(dct):
while True:
try:
yield dct.popitem()
except KeyError:
break
def bounded_imap_unordered(bound, pool, function, iterable):
iterable = iter(iterable)
results = queue.Queue()
outstanding = 0
while iterable or outstanding:
if iterable:
for elem in iterable:
pool.apply_async(function, (elem,), callback=results.put)
outstanding += 1
if outstanding >= bound or not results.empty():
break
else:
iterable = None
if outstanding:
yield results.get()
outstanding -= 1
def main():
logging.basicConfig(level=logging.DEBUG)
parser = argparse.ArgumentParser()
parser.add_argument("-m", "--mirror", action="store",
default="http://httpredir.debian.org/debian";,
help="Debian mirror to use")
args = parser.parse_args()
pkgs = dict()
url = "%s/dists/sid/main/binary-amd64/Packages" % args.mirror
with closing(open_compressed_mirror_url(url)) as pkglist:
for pkg in deb822.Packages.iter_paragraphs(pkglist):
try:
otherver = pkgs[pkg["package"]]["version"]
except KeyError:
pass
else:
if version_compare(otherver, pkg["version"]) < 0:
continue
pkgs[pkg["package"]] = dict(
version=pkg["version"],
filename=pkg["filename"])
pool = multiprocessing.Pool()
iterator = bounded_imap_unordered(32, pool, process_one_package,
((pkg, "%s/%s" % (args.mirror, meta["filename"]))
for pkg, meta in consume_items(pkgs)))
iterator = (dict(package=pkg,
files=dict((filename, dict(programs=sorted(programs),
magic=magicguess,
compressed=compressed))
for filename, (programs, magicguess, compressed)
in files.items()))
for (pkg, files) in iterator)
yaml.safe_dump_all(iterator, stream=sys.stdout)
if __name__ == '__main__':
main()
#!/usr/bin/python3
import sys
import yaml
manually_checked = {
"0xffff": "error message",
"actiona": "filesystem type list",
"alfred": "/sys/kernel/debug",
"alliance": "vlogsavelofig",
"amavisd-new": "nosuchattr",
"apitrace": "searchattributed",
"apitrace-gui": "searchattributed",
"apparmor-utils": "classifies tools from many packages",
"ardentryst": "python function name",
"argonaut-quota": "ldap_searchattribute",
"aufs-dkms": "#include <linux/debugfs.h>",
"augeas-lenses": "filesystem type list and mke2fs.conf",
"awesome-extra": "README.md",
"bacula-fd": "filesystem type list",
"bash-completion": "bash completions",
"batctl": "/sys/kernel/debug",
"bibledit-gtk-data": "html documentation",
"blktrace": "mountdebugfs and /sys/kernel/debug",
"blogliterately": "syntax highlighting",
"bluefish-data": "matchattr",
"borgbackup": "Python source comment",
"boxer-data": "package list including e2fsck-static",
"bpfcc-lua": "/sys/kernel/debug",
"brotli": "searchattributed",
"bup": "error messages and Python source comments",
"burp": "filesystem type list",
"busybox": "implements mke2fs",
"busybox-static": "implements mke2fs",
"calamares": "README.md",
"carettah": "syntax highlighting",
"chromium": "searchattributed",
"chromium-driver": "searchattributed",
"chromium-shell": "searchattributed",
"cinder-volume": "mmlsattr and mmchattr",
"coccinella": "value2label",
"context": "scratchattribute",
"coreutils": "filesystem type list",
"cpmtools": "cpmchattr",
"crash": "error message",
"crda": "C source comment",
"criu": "filesystem type list",
"cruft-common": "whitelist /etc/e2fsck.conf",
"cylc": "syntax highlighting",
"debian-cd": "package list including e2fsck-static",
"debtags": "tag database includes e2fsprogs",
"dfc": "filesystem type list",
"diamond": "/sys/kernel/debug",
"dict-de-en": "hochattraktiv",
"dict-freedict-deu-swe": "chattrum",
"dlm-controld": "error message",
"doconce": "title2label",
"dolibarr": "type2label",
"dotlrn": "Sprachattribute",
"doublecmd-gtk": "filesystem type list",
"doublecmd-qt": "filesystem type list",
"dovecot-core": "filesystem type list",
"dstat": "filesystem type list",
"dwarves": "/sys/kernel/debug",
"ecl": "ecl_chattrib",
"eficas": "foreachattr",
"eggdrop": "chattr irc command",
"eggdrop-data": "chattr irc command",
"elektra-tests": "filesystem type list",
"elvish": "mismatchattrScriptTypebad",
"emacs25": "filesystem type list",
"emacs25-lucid": "filesystem type list",
"erlang-base": "name2label",
"erlang-base-hipe": "name2label",
"erlang-src": "name2labels",
"ext4magic": "error message",
"facter": "actually uses lsattr on AIX",
"fdutils": "mbadblocks",
"fillets-ng-data": "game dialoges about chattr",
"findutils": "filesystem type list",
"fio": "e4defrag engine",
"firefox": "searchattributed",
"firefox-esr": "searchattributed",
"fish-common": "shell complestion for badblocks",
"fortunes": "jokes about chattr",
"fortunes-fr": "jokes about e2fsck",
"fortunes-it": "jokes about chattr",
"fortunes-mario": "jokes about chattr",
"fortunes-pl": "jokes about chattr",
"foxeye": "matchattr and internal chattr command",
"fpc-source-3.0.2": "variable name",
"freecol": "schattrein",
"fuse2fs": "error messages",
"gauche": "scheme variable name",
"gb": "protocolsattribute",
"gcc-5-source": "e2fsck_pass1 and changelog",
"gcc-6-source": "changelog",
"gcc-7-source": "changelog",
"gcl": "*chattrib",
"gfio": "engines/e4defrag.c",
"gfs2-utils": "/sys/kernel/debug",
"gitit": "syntax highlighting",
"gitlab": "filesystem type list",
"gnat-gps": "*_dialogsave_*",
"gnucash": "error message",
"gnulib": "filesystem type list",
"golang-github-aanand-compose-file-dev": "mtab like test output",
"golang-github-denverdino-aliyungo-dev": "modifyvswitchattribute",
"golang-github-opencontainers-runc-dev": "mtab like test output",
"golang-github-shirou-gopsutil-dev": "filesystem type list",
"golang-github-snapcore-snapd-dev": "go source comment",
"grc": "syntax highlighting",
"grr-server": "filesystem type list and mtab like test output",
"h2o": "searchattributed",
"hfsutils-tcltk": "tcl variable",
"hhvm": "searchattributed",
"highlight-common": "syntax highlighting",
"hostapd": "/sys/kernel/debug",
"hunspell-hu": "harminchattrilli?",
"idlestat": "/sys/kernel/debug",
"installation-report": "$logsavedir and a comment about debugfs",
"intel-gpu-tools": "/sys/kernel/debug",
"ircd-ircu": "chattr.tab.c and ircd_chattr.h",
"jfsutils": "jfs_debugfs",
"kate-data": "matchattr and syntax highlighting",
"kdelibs5-data": "woordenschattrainer",
"kdf": "filesystem type list",
"kernelshark": "/sys/kernel/debug",
"ketm": "badblocks.bmp",
"kexec-tools": "/sys/kernel/debug",
"kfloppy": "mke2fs documentation",
"kleopatra": "_ZNK4Kleo17DNAttributeMapper10name2labelERK7QString",
"kodi-pvr-vuplus": "e2imageversion",
"konversation-data": "chattrum and jokes about chattr",
"ktap": "#include <linux/debugfs.h>",
"laptop-mode-tools": "/sys/kernel/debug",
"latencytop": "/sys/kernel/debug",
"lazarus-ide-1.8": "liseventlogsavetofile",
"lazarus-ide-gtk2-1.8": "liseventlogsavetofile",
"lazarus-ide-qt5-1.8": "liseventlogsavetofile",
"lcl-nogui-1.8": "liseventlogsavetofile",
"lcl-utils-1.8": "liseventlogsavetofile",
"lcov": "/sys/kernel/debug",
"ldap-account-manager": "matchattrs",
"libalglib-dev": "C struct members",
"libbg1-dev": "debugfsys",
"libbio-graphics-perl": "feature2label and type2label",
"libblockdev-dev": "html and gir documentation",
"libcamitk4-doc": "m_5fdecimalsattribute",
"libconfig-model-perl": "fstab parser",
"libelektra4": "filesystem type list",
"libexodusii-dev": "elsattrb and elsattrib_name",
"libexodusii5": "elsattrb and elsattrib_name",
"libio-aio-perl": "filesystem type list",
"libghc-highlighting-kate-dev": "syntax highlighting",
"libghc-highlighting-kate-prof": "syntax highlighting",
"libghc-skylighting-dev": "syntax highlighting",
"libghc-skylighting-prof": "syntax highlighting",
"libgs9": "gs_attachattributrescolorspace",
"libgtksourceview2.0-common": "syntax highlighting",
"libgtksourceview-3.0-common": "syntax highlighting",
"libkf5akonadi-dev": "persistentsearchattribute",
"libkf5libkleo-dev": "name2label",
"libkf5libkleo5": "_ZNK4Kleo17DNAttributeMapper10name2labelERK7QString",
"libkf5coreaddons-data": "woordenschattrainer",
"libkleo4": "_ZNK4Kleo17DNAttributeMapper10name2labelERK7QString",
"liblinux-prctl-perl": "pod comments",
"libmailutils-dev": "chattr_msg",
"libmailutils5": "chattr_msg",
"libmonoboehm-2.0-1": "filesystem type list",
"libmonoboehm-2.0-dev": "filesystem type list",
"libmonosgen-2.0-1": "filesystem type list",
"libmonosgen-2.0-dev": "filesystem type list",
"libmount-dev": "filesystem type list",
"libmount1": "filesystem type list",
"libnet-imap-perl": "fetchattrs",
"libodin-dev": "type2label",
"libopenscap8": "filesystem type list",
"libopenzwave1.5": "switchtype2label",
"libpfm4": "/sys/kernel/debug",
"libpfm4-dev": "/sys/kernel/debug",
"libphp-phpmailer": "matchattrs",
"libqt5webenginecore5": "searchattributed",
"libreoffice-common": "srchattributes and searchattrdialog",
"libreoffice-core": "searchattrdialog",
"libsyntax-highlight-engine-kate-perl": "syntax highlighting",
"libsystem-info-perl": "actually uses lsattr on AIX",
"libsystemd0": "error message, chattr-util.c and chattr_fd",
"libterm-extendedcolor-perl": "Perl source comment",
"libvirt-daemon-system": "/sys/kernel/debug",
"libvm-ec2-perl": "runs e2fsprogs over ssh",
"libvtk6.3": "elsattrib_name and elsattrb",
"libwebkit2gtk-4.0-37": "searchattributed",
"libwebkit2gtk-4.0-37-gtk2": "searchattributed",
"libwiredtiger-dev": "__verify_filefrag_add",
"libxgks-dev": "C source symbols",
"libxmlada-schema4.6.2016": "...infoattrDeclsattributeattribu...",
"libxmlada-schema7-dev": "...foattrDeclsattributeattr...",
"linux-grsec-support-4.9.0-2": "aVdebugfs_create_automount",
"linux-headers-4.12.0-2-common": "linux kernel headers",
"linux-headers-4.12.0-2-amd64": "linux kernel headers",
"linux-headers-4.9.0-2-common-grsec": "linux kernel-headers",
"linux-headers-4.9.0-2-grsec-amd64": "linux kernel headers",
"linux-image-4.12.0-2-amd64": "linux kernel",
"linux-image-4.12.0-2-amd64-dbg": "linux kernel debug symbols",
"linux-image-4.9.0-2-grsec-amd64": "linux kernel",
"linux-patch-grsecurity2": "linux kernel source",
"linux-perf-4.12": "/sys/kernel/debug",
"linux-support-4.12.0-2": "aVdebugfs_create_automount",
"logcheck-database": "logcheck patterns",
"lttng-modules-dkms": "#include <linux/debugfs.h> and ChangeLog",
"lxc": "/sys/kernel/debug",
"lxc-tests": "mtab like test output",
"mali-midgard-dkms": "linux kernel extension",
"manaplus": "chattrade",
"manila-common": "command meta data",
"marionnet": "list of lots of binaries",
"mauve": "actually uses lsattr on AIX",
"mc": "ignore_ftp_chattr_errors and filesystem type list",
"mc-data": "syntax highlighting",
"mdadm": "examine-badblocks and error messages",
"mediawiki2latex": "syntax highlighting",
"medit": "syntax highlighting",
"mergerfs": "only package description",
"mercurial": "filesystem type list",
"mercurial-common": "debugfsinfo",
"mlton-compiler": "actually uses lsattr on AIX",
"modem-manager-gui": "contactname2label",
"modsecurity-crs": "patterns matching e2fsck.conf and chattr",
"monitoring-plugins-basic": "filesystem type list",
"mono-runtime-boehm": "filesystem type list",
"mono-runtime-sgen": "filesystem type list",
"mono-utils": "filesystem type list",
"mount": "bash-completion",
"mtools": "mbadblocks",
"munin-plugins-core": "filesystem type list and searchattrs",
"mysqltuner": "actually uses lsattr on AIX",
"nacl-tools": "uses AIX' lsattr",
"neofetch": "actually uses lsattr on AIX",
"netdata": "filesystem type list",
"netwox": "searchattributes",
"nmap-common": "page2images, dblogsave, lua source comment and ldap.searchattrib",
"ocfs2-tools": "debugfs.ocfs2, /sys/kernel/debug and error messages",
"ocfs2-tools-dev": "C source comments",
"ohai": "actually uses lsattr on AIX",
"onboard": "/sys/kernel/debug",
"open-infrastructure-system-build": "shell source comment",
"openacs": "Sprachattribute",
"ovirt-guest-agent": "filesystem type list",
"pandoc": "syntax highlighting",
"pandoc-citeproc": "syntax highlighting",
"paraview": "elsattrb*",
"patat": "syntax highlighting",
"pcp": "error message",
"pcp-testsuite": "/sys/kernel/debug and logs",
"perf-tools-unstable": "/sys/kernel/debug",
"petit": "dmesg output",
"php-geshi": "syntax highlighting",
"php7.0-dev": "actually uses lsattr on AIX",
"php7.1-dev": "actually uses lsattr on AIX",
"phpldapadmin": "matchattrs",
"pidgin": "chattrum",
"piglit": "debugfs_path",
"polari": "documentation",
"postgresql-10": "ldapsearchattribute",
"postgresql-9.6": "ldapsearchattribute",
"postgresql-server-dev-10": "ldapsearchattribute",
"postgresql-server-dev-9.6": "ldapsearchattribute",
"powerdebug": "error message",
"powertop": "/sys/kernel/debug",
"psychopy": "code2label",
"psychtoolbox-3-common": "/sys/kernel/debug",
"python-apipkg": "clsattr",
"python-blockdiag": "node2image",
"python-breezy.tests": "chattr Python function",
"python-brotli": "searchattributed",
"python-bzrlib.tests": "chattr Python function",
"python-cinder": "mmlsattr and mmchattr",
"python-colander": "clsattrs",
"python-fs": "debugfs Python module and chattr Python function",
"python-impacket": "rpc_m_invalid_srchattr",
"python-kde4-dev": "persistentsearchattribute",
"python-ldaptor": "*_nosuchattr",
"python-paramiko": "chattr method stubs",
"python-plumbum": "__plumbum_switchattr_dict__",
"python-sphinxcontrib.actdiag": "node2image",
"python-sphinxcontrib.blockdiag": "node2image",
"python-sphinxcontrib.nwdiag": "node2image",
"python-sphinxcontrib.seqdiag": "node2image",
"python-sprox": "name2label",
"python-toscawidgets": "name2label",
"python-twisted-core": "nosuchattrib",
"python-x2go": "chattr Python function",
"python3-apipkg": "clsattr",
"python3-blockdiag": "node2image",
"python3-brotli": "searchattributed",
"python3-colander": "clsattr",
"python3-paramiko": "chattr method stubs",
"python3-plumbum": "__plumbum_switchattr_dict__",
"python3-sphinxcontrib.actdiag": "node2image",
"python3-sphinxcontrib.blockdiag": "node2image",
"python3-sphinxcontrib.nwdiag": "node2image",
"python3-sphinxcontrib.seqdiag": "node2image",
"python3-twisted": "nosuchattrib",
"qtcreator-data": "syntax highlighting",
"r-bioc-multtest": "sample2label",
"racket-common": "relsattrs",
"rasdaemon": "/sys/kernel/debug",
"reiser4progs": "debugfs.reiser4",
"rex": "pod documentation",
"rhn-client-tools": "Python source comments",
"rmlint": "filesystem type list",
"roundcube-plugins": "*lchattr",
"rr": "searchattributed",
"rt-tests": "/sys/kernel/debug",
"ruby-dnsruby": "name2labels",
"ruby-fog-aliyun": "modifyvswitchattribute",
"sash": "implements lsattr and chattr",
"seekwatcher": "/sys/kernel/debug",
"selinux-policy-dev": "kernel_.*_debugfs and debugfs_t",
"selinux-policy-src": "selinux policy",
"sftpcloudfs": "Python method chattr",
"shtool": "actually uses lsattr on AIX",
"simplesamlphp": "searchattributes",
"sitesummary": "filesystem type list",
"sitesummary-nodes": "filesystem type list",
"skylighting": "syntax highlighting",
"snakemake": "node2label",
"strace": "format string",
"system-tools-backends": "mountdebugfs",
"systemd-container": "chattr_fd",
"systemtap-common": "/sys/kernel/debug",
"systemtap-runtime": "/sys/kernel/debug",
"taglog": "default filename of SaveAsFilename",
"testdisk": "error message",
"testdisk-dbg": "debug symbols",
"texlive-binaries": "searchattributed",
"texlive-pictures": "magichattrue",
"thunderbird": "searchattributed",
"tiger": "documentation, comments and filesystem type list",
"tortoisehg": "_enable2label",
"toxiproxy-cli": "mismatchattributes",
"trace-cmd": "/sys/kernel/debug",
"trans-de-en": "hochattraktiv",
"typespeed": "word list",
"ulatencyd": "/sys/kernel/debug",
"undertaker": "/sys/kernel/debug",
"urlscan": "searchattr",
"user-mode-linux": "kernel error messages and symbols",
"util-linux": "filesystem type list and --badblocks option",
"wims": "jokes about chattr",
"wuzz": "...-dialogsave-...",
"xenomai-kernel-source": "debug filesystem",
"xjadeo": "filesystem type list",
"xscreensaver-screensaver-bsod": "playing back fsck",
"xserver-xorg-video-intel": "/sys/kernel/debug",
"xymon-client": "actually uses lsattr on AIX",
"wpasupplicant": "/sys/kernel/debug",
"znc-tcl": "matchattr",
"zsh-common": "madblocks and tune2fs completions",
}
def main():
for doc in yaml.safe_load_all(stream=sys.stdin):
for filename, matches in doc["files"].items():
ignore = None
if filename.startswith("./usr/share/doc/"):
ignore = "documentation"
elif filename.startswith("./usr/share/") \
and "/locale/" in filename \
and filename.endswith(".mo") \
and matches["magic"].startswith("GNU message catalog"):
ignore = "locale"
elif filename.startswith("./usr/share/libreoffice/help/"):
ignore = "libreoffice help files"
elif filename.startswith("./usr/share/man/") \
and matches["compressed"]:
ignore = "manual page"
elif filename.startswith("./usr/lib/debug/.build-id/") \
and matches["magic"].startswith("ELF "):
ignore = "debug symbols"
elif filename.startswith("./usr/share/info/") and \
matches["compressed"]:
ignore = "info page"
elif filename.startswith("./usr/share/hunspell/") \
and filename.endswith(".dic"):
ignore = "myspell dictionary"
elif filename.startswith("./usr/lib/libreoffice/share/config/soffice.cfg/cui/ui/res/") \
and filename.endswith(".zip"):
ignore = "libreoffice translation"
elif filename.startswith("./usr/share/debian-reference/"):
ignore = "debian-reference"
elif filename.startswith("./usr/lib/debian-installer/images/"):
ignore = "debian-installer images"
elif filename.startswith("./usr/share/help/") \
and not matches["compressed"] \
and matches["magic"] == "XML document text":
ignore = "xml help file"
elif filename.endswith("/config.guess") \
and matches["programs"] == ["lsattr"] \
and not matches["compressed"] \
and matches["magic"] == "POSIX shell script, ASCII text executable":
ignore = "config.guess"
elif filename == "DEBIAN/md5sums":
ignore = filename
elif filename.endswith(".po") \
and not matches["compressed"] \
and matches["magic"].startswith(("GNU gettext message catalogue",
"UTF-8 Unicode text")):
ignore = ".po file"
elif '/modules/' in filename \
and '/kernel/' in filename \
and filename.endswith(".ko") \
and not matches["compressed"] \
and matches["magic"].startswith("ELF "):
ignore = "linux kernel module"
elif filename.endswith(".html") \
and matches["magic"].startswith("HTML document"):
ignore = "html"
elif doc["package"] in ("e2fsprogs", "e2fslibs", "e2fslibs-dev",
"e2fsck-static"):
ignore = "e2fsprogs itself"
elif doc["package"].startswith("linux-libc-dev"):
ignore = "linux-libc-dev"
else:
try:
ignore = manually_checked[doc["package"]]
except KeyError:
pass
print("%s %s %s %s (%s %s)%s" %
("ignore" if ignore else "check",
doc["package"],
filename,
",".join(matches["programs"]),
"compressed" if matches["compressed"] else "plain",
matches["magic"],
" because " + ignore if ignore else ""))
if __name__ == '__main__':
main()
Aaron M. Ucko <u...@debian.org>
gbrowse (U)
Adam Conrad <adcon...@0c3.net>
initramfs-tools (U)
Adrian Vondendriesch <adrian.vondendrie...@credativ.de>
resource-agents (U)
Alejandro Garrido Mota <garridom...@gmail.com>
libsys-filesystem-perl (U)
Alexander Wirt <formo...@debian.org>
grml-debootstrap (U)
Andreas B. Mundt <a...@debian.org>
debian-lan-config (U)
Andreas Cadhalpun <andreas.cadhal...@googlemail.com>
havp (U)
Andreas Henriksson <andr...@fatal.se>
libblockdev (U)
Andreas Tille <andr...@an3as.eu>
manila (U)
Andreas Tille <ti...@debian.org>
gbrowse (U)
Andrew Lee (æå¥ç§) <ajq...@debian.org>
obs-build (U)
Andriy Senkovych <jolly_ro...@itblog.org.ua>
salt (U)
Anibal Monsalve Salazar <ani...@debian.org>
gparted (U)
xfsprogs (U)
Ansgar Burchardt <ans...@debian.org>
libsys-filesystem-perl (U)
Anthony Prades <toony.deb...@chezouam.net>
cyrus-imapd (U)
Antonio Radici <anto...@debian.org>
cfengine3
Antonio Terceiro <terce...@debian.org>
cloud-utils (U)
ruby-specinfra (U)
Apollon Oikonomopoulos <apoi...@debian.org>
ganeti-instance-debootstrap (U)
Aron Xu <a...@debian.org>
dkms (U)
Aurelien Jarno <aure...@debian.org>
qemu (U)
Aurélien GÃRÃME <a...@roxor.cx>
rsrce (U)
Axel Beckert <a...@debian.org>
xen-tools
Bastian Blank <wa...@debian.org>
cloud-init (U)
linux (U)
lvm2 (U)
Bdale Garbee <bd...@gag.com>
plinth (U)
Ben Hutchings <b...@decadent.org.uk>
initramfs-tools (U)
linux (U)
Benda Xu <hero...@gmail.com>
sysvinit (U)
Benjamin Drung <benjamin.dr...@profitbricks.com>
salt (U)
Benjamin J. Scott <bensc...@nwlink.com>
kvpm
Bernd Schumacher <bernd.schumac...@hpe.com>
bootcd
Carlos Alberto Lopez Perez <clo...@igalia.com>
util-vserver
ChangZhuo Chen (é³æå¬) <czc...@debian.org>
tomb (U)
Charles Plessy <ple...@debian.org>
cloud-init (U)
euca2ools (U)
gbrowse (U)
Chris AtLee <ch...@atlee.ca>
drobo-utils
Chris Grzegorczyk <g...@eucalyptus.com>
euca2ools (U)
Chris Lamb <la...@debian.org>
diffoscope (U)
installation-birthday
Christian Hofstaedtler <z...@debian.org>
boot-info-script
grml-debootstrap (U)
Christoph Berg <m...@debian.org>
postgresql-common (U)
resource-agents (U)
Christophe Monniez <christophe.monn...@fccu.be>
recoverdm (U)
ClamAV Team <pkg-clamav-de...@lists.alioth.debian.org>
havp
Corey Bryant <corey.bry...@canonical.com>
tempest (U)
Daniel Jared Dominguez <jared.doming...@dell.com>
fwupdate (U)
Dave Love <f...@gnu.org>
singularity-container (U)
David Paleino <da...@debian.org>
dkms (U)
Debian Accessibility Team <pkg-a11y-de...@lists.alioth.debian.org>
edbrowse
Debian buildd-tools Developers <buildd-tools-de...@lists.alioth.debian.org>
sbuild
Debian Cloud Team <cloud-packa...@lists.alioth.debian.org>
bootstrap-vz
Debian Cloud Team <debian-cl...@lists.debian.org>
cloud-init
Debian Cyrus Team <pkg-cyrus-imapd-debian-de...@lists.alioth.debian.org>
cyrus-imapd
Debian DNS Packaging <pkg-dns-de...@lists.alioth.debian.org>
dnssec-trigger
Debian Edu Developers <debian-...@lists.debian.org>
debian-edu-config
Debian EFI <debian-...@lists.debian.org>
fwupdate
Debian Eucalyptus Maintainers
<pkg-eucalyptus-maintain...@lists.alioth.debian.org>
euca2ools
Debian Forensics <forensics-de...@lists.alioth.debian.org>
recoverdm
rkhunter
safecopy
Debian Ganeti Team <pkg-ganeti-de...@lists.alioth.debian.org>
ganeti-instance-debootstrap
Debian Go Packaging Team <pkg-go-maintain...@lists.alioth.debian.org>
nomad (U)
Debian HA Maintainers <debian-ha-maintain...@lists.alioth.debian.org>
resource-agents
Debian kernel team <debian-ker...@lists.debian.org>
initramfs-tools
Debian Kernel Team <debian-ker...@lists.debian.org>
linux
Debian LAN Developers <debian-lan-de...@lists.alioth.debian.org>
debian-lan-config
Debian LAVA team <pkg-linaro-lava-de...@lists.alioth.debian.org>
lava-dispatcher
Debian Libvirt Maintainers <pkg-libvirt-maintain...@lists.alioth.debian.org>
libguestfs
libvirt-sandbox
supermin
virt-manager
Debian Live <debian-l...@lists.debian.org>
live-build
Debian LVM Team <pkg-lvm-maintain...@lists.alioth.debian.org>
lvm2
Debian Med Packaging Team <debian-med-packag...@lists.alioth.debian.org>
gbrowse
Debian pbuilder maintenance team <pbuilder-ma...@lists.alioth.debian.org>
cowdancer
Debian Perl Group <pkg-perl-maintain...@lists.alioth.debian.org>
libsys-filesystem-perl
Debian PostgreSQL Maintainers <pkg-postgresql-pub...@lists.alioth.debian.org>
postgresql-common
Debian Privacy Tools Maintainers
<pkg-privacy-maintain...@lists.alioth.debian.org>
tails-installer
Debian QA Group <packa...@qa.debian.org>
boxbackup
gfxboot
lxctl
partimage
ploop
Debian QEMU Team <pkg-qemu-de...@lists.alioth.debian.org>
qemu
Debian Ruby Extras Maintainers
<pkg-ruby-extras-maintain...@lists.alioth.debian.org>
ruby-specinfra
Debian Salt Team <pkg-salt-t...@lists.alioth.debian.org>
salt
Debian Security Tools Packaging Team <pkg-security-t...@lists.alioth.debian.org>
tomb
Debian systemd Maintainers <pkg-systemd-maintain...@lists.alioth.debian.org>
casync
mkosi
systemd
Debian sysvinit maintainers <pkg-sysvinit-de...@lists.alioth.debian.org>
sysvinit
Debootloaders miBoot Maintainers Team
<debootloaders-mib...@lists.alioth.debian.org>
rsrce
Diane Trout <di...@ghic.org>
dnssec-trigger (U)
Dimitri John Ledkov <dimitri.j.led...@linux.intel.com>
obs-build (U)
Dmitry Smirnov <only...@debian.org>
nilfs-tools (U)
nomad
Docker Packaging Team <docker-ma...@lists.alioth.debian.org>
docker.io (U)
Dustin Kirkland <kirkl...@ubuntu.com>
euca2ools (U)
Dynamic Kernel Modules Support Team <pkg-dkms-ma...@lists.alioth.debian.org>
dkms
Eric Delaunay <delau...@debian.org>
scsitools
Etienne Dublé <etienne.du...@gmail.com>
debootstick
Fathi Boudra <f...@debian.org>
lava-dispatcher (U)
linaro-image-tools (U)
FAUmachine Team <faumach...@potyra.de>
faumachine
Federico Ceratto <feder...@debian.org>
plinth (U)
Felipe Sateler <fsate...@debian.org>
casync (U)
mkosi (U)
Felix Zielcke <fziel...@z-51.de>
reiserfsprogs
Filesystems Group <filesystems-de...@lists.alioth.debian.org>
nilfs-tools
Francesco Paolo Lovergine <fran...@debian.org>
sbuild (U)
Francisco Manuel Garcia Claramonte <franci...@debian.org>
lynis
Franck Joncourt <fra...@debian.org>
libsys-filesystem-perl (U)
Francois Marier <franc...@debian.org>
rkhunter (U)
Franklin G Mendoza <franklin.g.mend...@gmail.com>
salt (U)
FreedomBox packaging team <freedombox-pkg-t...@lists.alioth.debian.org>
plinth
Frédéric Bonnard <fre...@linux.vnet.ibm.com>
rear
Georges Khaznadar <georg...@debian.org>
clonezilla
drbl
Gergely Nagy <alger...@balabit.hu>
edbrowse (U)
Giuseppe Iuculano <iucul...@debian.org>
dkms (U)
Gonéri Le Bouder <gon...@debian.org>
fusioninventory-agent
Graziano Obertelli <grazi...@eucalyptus.com>
euca2ools (U)
Grml Team <t...@grml.org>
grml-debootstrap
Guido Günther <a...@sigxcpu.org>
libguestfs (U)
libvirt-sandbox (U)
virt-manager (U)
Guido Trotter <ultrot...@debian.org>
ganeti-instance-debootstrap (U)
gustavo panizzo <g...@zumbi.com.ar>
nova (U)
Harlan Lieberman-Berg <hlieber...@debian.org>
ansible
Helge Deller <del...@gmx.de>
palo
Henrique de Moraes Holschuh <h...@debian.org>
cyrus-imapd (U)
Herbert Parentes Fortes Neto <h...@debian.org>
pyparted
Hideki Yamane <henr...@debian.org>
snapper
Hilko Bengen <ben...@debian.org>
libguestfs (U)
supermin (U)
Holger Levsen <hol...@debian.org>
debian-edu-config (U)
diffoscope (U)
Héctor Orón MartÃnez <zu...@debian.org>
obs-build (U)
Iain R. Learmonth <i...@debian.org>
vmdebootstrap (U)
Ian Jackson <ijack...@chiark.greenend.org.uk>
sysvinit (U)
James Clarke <jrt...@debian.org>
cowdancer (U)
Jan Christoph Nordholz <he...@pool.math.tu-berlin.de>
autofs (U)
Jean-Michel Kelbert <kelb...@debian.org>
opensvc
Jeremie Koenig <spr...@sprite.fr.eu.org>
rsrce (U)
Joachim Wiedorn <joodeb...@joonet.de>
lilo
Joao Eriberto Mota Filho <eribe...@debian.org>
safecopy (U)
Joe Healy <joehe...@gmail.com>
salt (U)
Johannes Schauer <jo...@debian.org>
sbuild (U)
John Paul Adrian Glaubitz <glaub...@physik.fu-berlin.de>
kvpm (U)
Jonathan Yu <jaw...@cpan.org>
libsys-filesystem-perl (U)
Jordi Mallach <jo...@debian.org>
lava-dispatcher (U)
Julien Danjou <a...@debian.org>
cloud-init (U)
cloud-utils (U)
KURASHIKI Satoru <lur...@gmail.com>
ruby-specinfra (U)
Kyo Lee <kyo....@eucalyptus.com>
euca2ools (U)
LaMont Jones <lam...@debian.org>
postfix
Lars Wirzenius <l...@liw.fi>
vmdebootstrap (U)
Laurent Léonard <laur...@open-minds.org>
virt-manager (U)
Lee Garrett <deb...@rocketjump.eu>
ansible (U)
Linaro Packagers <packa...@lists.linaro.org>
linaro-image-tools
Louis Bouchard <louis.bouch...@ubuntu.com>
sosreport
Louis Zuckerman <m...@louiszuckerman.com>
glusterfs (U)
Malihe Asemani <ml.asem...@gmail.com>
manila (U)
Marcin Kulisz (kuLa) <deb...@kulisz.net>
bootstrap-vz (U)
Marco d'Itri <m...@linux.it>
systemd (U)
Marco Nenciarini <mnen...@debian.org>
resolvconf (U)
Mario Limonciello <mario.limoncie...@dell.com>
fwupdate (U)
Mario Limonciello <mario_limoncie...@dell.com>
dkms (U)
Martin Pitt <mp...@debian.org>
casync (U)
libblockdev (U)
postgresql-common (U)
systemd (U)
udisks2 (U)
Mateusz Åukasik <mat...@linuxmint.pl>
spacefm
Mattia Rizzolo <mat...@debian.org>
cowdancer (U)
diffoscope (U)
maximilian attems <m...@debian.org>
initramfs-tools (U)
linux (U)
Mehdi Dogguy <me...@debian.org>
singularity-container (U)
Michael Banck <mba...@debian.org>
sbuild (U)
Michael Biebl <bi...@debian.org>
fsarchiver
systemd (U)
udisks2 (U)
Michael Prokop <m...@debian.org>
fai (U)
grml-debootstrap (U)
initramfs-tools (U)
Michael Tokarev <m...@tls.msk.ru>
autofs
qemu (U)
Michael Vogt <m...@debian.org>
ansible (U)
Miguel Landaeta <mig...@miguel.cc>
euca2ools (U)
Miguel Landaeta <nomad...@debian.org>
cloud-init (U)
Mike Gabriel <sunwea...@debian.org>
debian-edu-config (U)
Nathan Scott <nath...@debian.org>
xfsprogs (U)
Neil Williams <codeh...@debian.org>
lava-dispatcher (U)
NeuroDebian Team <t...@neuro.debian.net>
singularity-container
Nick Daly <nick.m.d...@gmail.com>
plinth (U)
Olivier Sallou <osal...@debian.org>
gbrowse (U)
OndÅej Surý <ond...@debian.org>
cyrus-imapd (U)
dnssec-trigger (U)
Patrick Matthäi <pmatth...@debian.org>
glusterfs
Paul Tagliamonte <paul...@debian.org>
docker.io (U)
Peter Eisentraut <pet...@debian.org>
postgresql-common (U)
Petter Reinholdtsen <p...@debian.org>
debian-edu-config (U)
dkms (U)
plinth (U)
Phillip Susi <ps...@ubuntu.com>
gparted
Pierre Chifflier <pol...@debian.org>
ocsinventory-agent
Piotr Krysiuk <piot...@gmail.com>
rsrce (U)
Piotr Ożarowski <pi...@debian.org>
plinth (U)
PKG OpenStack <openstack-de...@lists.alioth.debian.org>
manila
nova
openstack-debian-images
openstack-trove
puppet-module-swift
python-diskimage-builder
sahara
tempest
Python Applications Packaging Team <python-apps-t...@lists.alioth.debian.org>
drobo-utils (U)
Raphaël Hertzog <hert...@debian.org>
live-build (U)
Reiner Herrmann <rei...@reiner-h.de>
diffoscope (U)
Rene Mayrhofer <rene.mayrho...@gibraltar.at>
havp (U)
Reproducible builds folks <reproducible-bui...@lists.alioth.debian.org>
diffoscope
resolvconf maintainers <resolvconf-de...@lists.alioth.debian.org>
resolvconf
Richard Jones <rjo...@redhat.com>
libguestfs (U)
Riku Voipio <riku.voi...@iki.fi>
qemu (U)
RPM packaging team <pkg-rpm-de...@lists.alioth.debian.org>
obs-build
RW Penney <rwpen...@users.sourceforge.net>
cryptmount
Salvatore Bonaccorso <car...@debian.org>
linux (U)
Samuel Thibault <sthiba...@debian.org>
edbrowse (U)
Scott Kitterman <sc...@kitterman.com>
havp (U)
postfix (U)
Scott Leggett <sc...@sl.id.au>
ruby-specinfra (U)
Sean Whitton <spwhit...@spwhitton.name>
propellor
Sebastian Andrzej Siewior <sebast...@breakpoint.cc>
havp (U)
Senthil Kumaran S (stylesen) <style...@gmail.com>
lava-dispatcher (U)
Simon Quigley <tsimo...@ubuntu.com>
note
Sjoerd Simons <sjo...@debian.org>
systemd (U)
Stefan Potyra <sistp...@ubuntu.com>
faumachine (U)
Steffen Moeller <moel...@debian.org>
python-cgcloud
Steve McIntyre <93...@debian.org>
fwupdate (U)
vmdebootstrap (U)
Sunil Mohan Adapa <su...@medhas.org>
plinth (U)
Sven Luther <lut...@debian.org>
rsrce (U)
Thomas Goirand <z...@debian.org>
cloud-init (U)
cloud-utils (U)
dtc-xen
manila (U)
nova (U)
openstack-debian-images (U)
openstack-trove (U)
puppet-module-swift (U)
python-diskimage-builder (U)
sahara (U)
tempest (U)
Thomas Hood <jdth...@gmail.com>
resolvconf (U)
Thomas Lange <la...@debian.org>
dracut
fai
Tiago Ilieve <tiago.my...@gmail.com>
bootstrap-vz (U)
cloud-utils
Tianon Gravi <tia...@debian.org>
docker.io (U)
Tim Potter <t...@hpe.com>
docker.io
Tzafrir Cohen <tzaf...@debian.org>
mock
plinth (U)
Ulrich Dangel <m...@spamt.net>
grml-debootstrap (U)
Ulrike Uhlig <ulr...@debian.org>
tails-installer (U)
Utopia Maintenance Team <pkg-utopia-maintain...@lists.alioth.debian.org>
libblockdev
udisks2
Valentin Vidic <valentin.vi...@carnet.hr>
resource-agents (U)
VMDebootstrap List <vmdebootstrap-de...@lists.alioth.debian.org>
vmdebootstrap
Volkmar Sieh <volkmar.s...@informatik.uni-erlangen.de>
faumachine (U)
William Dauchy <wdau...@gmail.com>
autofs (U)
Wolfgang Schweer <wschw...@arcor.de>
debian-edu-config (U)
Wolodja Wentland <deb...@babilen5.org>
salt (U)
Wookey <woo...@debian.org>
sbuild (U)
XFS Development Team <linux-...@vger.kernel.org>
xfsprogs
Ximin Luo <infini...@debian.org>
diffoscope (U)
Yaroslav Halchenko <deb...@onerussian.com>
singularity-container (U)
