[Reportbug-maint] Bug#892898: reportbug crashes due with UnicodeDecodeError
control: tags -1 patch Thank you for the report. This should fix the crash: https://salsa.debian.org/reportbug-team/reportbug/merge_requests/2 I could not reproduce the problem, though. What does DEBCONF_SYSTEMRC=1 DEBCONF_NOWARNINGS=yes debconf-show console-setup output on your system? Is the output not in a consistent encoding? ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#878088: Bug#878088: reportbug: please inform security and lts teams about security update regressions
Thanks Sandro! One thing is missing, though: we now need a dependency on python3-apt. ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#890680: Bug#890680: reportbug: python3-reportbug submodules are not well documented
On 20-02-2018 06:02, Sandro Tosi wrote: > On Sun, Feb 18, 2018 at 5:03 AM, Nis Martensen wrote: >> Extending the test suite is actually the goal here. It's just hard to >> add tests for functions of which you don't know what they're supposed to >> do exactly. So reading the code and taking notes is the first step. > > oh great to hear we're one the same page on that! :) I doubt it's going to be easy, though - many bugs are like "proxy-related command line options don't work well" or "does not interact nicely with my mua" or "crashes when user's homedir does not exist". Not sure how those can be covered with unit tests. But let's go step by step to figure out what's possible. > i was more thinking of tools external to debian, like scripts from > operators using those functions Hm. Are you aware of people having done that? Would the switch from py2 to py3 not already have broken such tools? >> Are you planning to move reportbug to salsa in the future? It might >> make this kind of review easier. > > i just did and migrated reportbug to > https://salsa.debian.org/reportbug-team/reportbug - wanna try the > merge request thing ah! :) Here you go: https://salsa.debian.org/reportbug-team/reportbug/merge_requests/1 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#890680: Bug#890680: reportbug: python3-reportbug submodules are not well documented
Hey Sandro! On 18-02-2018 05:49, Sandro Tosi wrote: > Hey Nis! > >> The reportbug source code has some issues that make it hard to review: > > > thanks for working on this! even if i dont want to mess with your > plans, let me tell you that if you really want to work on this "qa" > kind-of activity for reportbug, i think the area that will give the > biggest impact is improvements to the test suite: currently part of > the code is not covered well (or at all) and i did a poor job at > adding tests for bugs that were filed and fixed; also i've always > wished to be able to test the interactive part in reportbug (maybe > mocking the input method). Extending the test suite is actually the goal here. It's just hard to add tests for functions of which you don't know what they're supposed to do exactly. So reading the code and taking notes is the first step. >> - Most functions in reportbug/*.py do not have a docstring, or only a very >>brief one. Some functions still have many parameters, and some return >> quite >>complex objects. >> - module-internal functions are not prefixed with an underscore, so it is >> hard >>to tell which functions are meant for external use. > > can we do something now, though? as of now they are all public > functions, removing them by making them private can break someone's > code (ok ok it's rather remote possibility but still) You are of course correct. `apt-cache rdepends python3-reportbug` only lists reportbug, though. I also spent some time searching with codesearch.debian.net some weeks ago, and couldn't find anything else. I believe it is quite unlikely that there are any users outside debian. The current patch set includes making functions private. Let me know if you don't want this part. >> I'm slowly working on patches. > > i would suggest to submit of your early patches soon, so that i can > have a look and eventually comment earlier It is quite a lot already. Will send later today. Are you planning to move reportbug to salsa in the future? It might make this kind of review easier. Nis ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#890680: reportbug: python3-reportbug submodules are not well documented
Package: reportbug Version: 7.1.8 Severity: wishlist The reportbug source code has some issues that make it hard to review: - Most functions in reportbug/*.py do not have a docstring, or only a very brief one. Some functions still have many parameters, and some return quite complex objects. - module-internal functions are not prefixed with an underscore, so it is hard to tell which functions are meant for external use. I'm slowly working on patches. # test: \xc2\xb7 M\xc3\xb6h ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#853915: Bugreports and base64
control: clone 853915 -1 control: reassign 853915 python-debianbts control: retitle -1 reportbug: base64 encoded reports rejected by bts Reading and sending base64 message are two different bugs, so let's split this report. I believe that python-debianbts is supposed to decode a base64 message body, therefore I'm reassigning the base64 reading bug. Please reassign back to reportbug if this assumption is wrong. ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#878088: Bug#878088: reportbug: please inform security and lts teams about security update regressions
On 29-01-2018 00:11, Markus Koschany wrote: > > I noticed that you had to import apt but reportbug does not depend on > python3-apt. After I had installed this package it worked. I also > believe you don't need to check for the upstream changelog.gz file, the > Debian changelog should be sufficient. There is already a patch in some other bug report that adds a Depends: on python3-apt. You are right that this is also required if the new function gets accepted. We need to look for both changelog files, since native packages (like reportbug) do not have a separate Debian one. Your patch looks good to me now. Only minor nits: > reportbug.debdiff > > > diff -Nru reportbug-7.1.8/bin/reportbug reportbug-7.1.8+nmu1/bin/reportbug > --- reportbug-7.1.8/bin/reportbug 2017-12-29 05:25:43.0 +0100 > +++ reportbug-7.1.8+nmu1/bin/reportbug2018-01-23 20:43:14.0 > +0100 > @@ -32,6 +32,8 @@ > import optparse > import re > import locale > +import requests > +import json > import subprocess > import shlex > import email > @@ -1926,6 +1928,37 @@ > listcc += ui.get_multiline( > 'Enter any additional addresses this report should be sent > to; press ENTER after each address.') > > +# If the bug is reported against a package with a version that > +# indicates a security update add the security or lts team to CC > +# after user confirmation > +if pkgversion and package and not self.options.offline and not > self.options.mode == 'novice': Instead of "not self.options.mode == 'novice', please use "mode > MODE_NOVICE" > +if utils.is_security_update(package, pkgversion): > +if ui.yes_no('Do you want to report a regression because of > a security update? ', > + 'Yes, please inform the LTS and security > teams.', > + 'No or I am not sure.', True): > +regex = re.compile('(\+|~)deb(\d+)u(\d+)') > +secversion = regex.search(pkgversion) > +distnumber = secversion.group(2) shorter: distnumber = re.search('[+~]deb(\d+)u\d+', pkgversion).group(1) > +support = 'none' > +email_address = [] email_address = 'none' > +try: > +r = > requests.get('https://security-tracker.debian.org/tracker/distributions.json', > +timeout=self.options.timeout) > +data = r.json() > +for key, value in data.items(): > +if distnumber == value['major-version']: > +if value['support']: > +support = value['support'] > +if value['contact']: > +email_address = value['contact'] If we can trust that no fields are null, then the last two ifs are not needed and we can drop them to simplify the code. If we don't trust the input, should we protect against other errors as well? We'd get TypeError if some `value` is not a dict, and KeyError if any key is not there. > + > +if support != 'none': if support != 'none' and utils.check_email_addr(email_address) > +listcc += [email_address] else: ewrite('No support team contact address could be identified.\n') > + > +except requests.exceptions.RequestException: If we want to also catch TypeError and KeyError, do it here or in a separate except clause? > +ewrite('Unable to connect to > security-tracker.debian.org.\n' > + 'Please try again later or contact the LTS or > security team via email directly.\n') > + > if severity and rtype: > severity = debbugs.convert_severity(severity, rtype) > ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#878088: Bug#878088: reportbug: please inform security and lts teams about security update regressions
On 26-01-2018 15:45, Markus Koschany wrote: > I am not convinced that the apt-cache method is more efficient than > parsing the version string. I believe my method is simpler and it would > catch the same potential candidates as your apt-cache idea. Manual > intervention (answering a question) cannot be avoided unless the > security team agrees to receive all bug reports against a version with a > security update. I am absolutely sure that is not desired. I agree that the question should be asked when the package version is a security update. What I am trying to achieve using the apt-cache method (on top of the version string method) is to avoid asking the question for "normal" package updates in stable. Attached a new version of the is_security_update function. This could be further refined by fetching the changelog from the package tracker if the package version is not the installed one, but this is probably going too far... No idea how many of the stable package updates are usually normal bug fix updates compared to the number of security updates. If updates are almost all security updates, then we should definitely not do such micro-optimization and go with your original approach. > I favor my current patch because of the reasons I mentioned before. I > can remove the sys.exit call? What else should be done? "secversion[2]" should be "secversion.group(2)", right? The former variant did not work for me in a quick test. Using an else clause may be more pythonic than my previous suggestion of moving more stuff into the try block. Reportbug has an "ewrite()" function that you can use for the warning message. Reportbug has a concept of user expertise levels. Can the question be skipped in novice mode? Should reportbug incorporate a default version of the json file to fall back to if the lookup fails? Reportbug is probably going to be updated more often than the online version of the json file. An internal version could also be updated regularly. >From 56d6dbffd3075bb9f06aa1b31fd8f688b6b16d4b Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Sat, 27 Jan 2018 22:22:31 +0100 Subject: [PATCH] utils: add new is_security_update() function The security and LTS teams want to ask the reporting user for notification if one of their updates caused a regression. On the other hand, reportbug should not ask users unnecessary questions. So let's try to collect evidence that this report is really about a package version from a security update. --- reportbug/utils.py | 79 ++ 1 file changed, 79 insertions(+) diff --git a/reportbug/utils.py b/reportbug/utils.py index b376c6c..c372727 100644 --- a/reportbug/utils.py +++ b/reportbug/utils.py @@ -39,6 +39,8 @@ import email import socket import subprocess import pipes +import apt +import gzip from .urlutils import open_url from string import ascii_letters, digits @@ -1333,3 +1335,80 @@ def get_lsm_info(): break return lsminfo + + +def is_security_update(pkgname, pkgversion): +"""Determine whether a given package is a security update. + +Detection of security update versions works most reliably if the +package version under investigation is the currently installed +version. If this is not the case, the probability of false +negatives increases. + +Parameters +-- +pkgname : str +package name +pkgversion : str +package version + +Returns +--- +bool +True if there is evidence that this version is a security +update, otherwise False +""" +# Check 1: +# If it does not follow the debXuY version number pattern, it is +# definitely no security update. +# +# This check is not sufficient to detect security updates reliably, +# since other stable updates also use the same version pattern. +regex = re.compile('(\+|~)deb(\d+)u(\d+)') +secversion = regex.search(pkgversion) +if not secversion: +return False + +# Check 2: +# If the package comes from the Debian-Security package source, it +# is definitely a security update. +# +# This check does not identify all security updates, since some of +# them are distributed through the normal channels as part of a +# stable release update. +try: +p = apt.Cache()[pkgname] +if 'Debian-Security' in [o.label for o in +p.versions[pkgversion].origins]: +return True +except: +pass + +# Check 3: +# Inspect the package changelog if it mentions any vulnerability, +# identified by a CVE number, in the section of the latest version. + +cl = None +for cl in ['/usr/share/doc/{}/changelog.Debian.gz'.format(pkgname), + '/usr/share/doc/{}/changelog.gz&
[Reportbug-maint] Bug#878088: reportbug: please inform security and lts teams about security update regressions
On 24-01-2018 19:37, Markus Koschany wrote: > Thanks. How do you catch the case when security updates are part of a > stable point release? This requires more effort. Does the package tracker offer a way to query such information? The only other idea I have right now involves inspecting the latest entry in changelog.Debian.gz. ("Was the package uploaded by the maintainer or one of the normal uploaders?") Do you have other ideas on how a user might know whether a package update delivered in a stable point release was a security update? Would it be feasible to make all security updates available via the security update channel? Then the simple suggested method would be sufficient. But it is probably infeasible, otherwise it would be done? If there is no good way, maybe asking your question only for the packages identified by the proposed method would be acceptable as a first step, until a reliable approach is developed? But perhaps Sandro may even be willing to accept a patch based on your original version string pattern matching, if his other concerns are addressed. Sandro, what do you think? ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#878088: reportbug: please inform security and lts teams about security update regressions
How about this? import apt def is_security_update(pkgname, version): try: p = apt.Cache()[pkgname] except: return False if version not in p.versions: return False if 'Debian-Security' in [o.label for o in p.versions[version].origins]: return True return False ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#878088: reportbug: please inform security and lts teams about security update regressions
Let me describe my suggestion in more detail: - Instead of having the "is this a security version" check implemented directly in bin/reportbug as a version number check, there could be a new function "is_security_update(package, version)" in reportbug/utils.py. You can move the version number check there to quickly decide if this is definitely no security version. - The point of the apt-cache idea was to try harder to avoid asking the user unnecessary questions. So this could be included in the new function. Your question will still be asked, but only if the evidence that the package actually is a security update is stronger. To avoid the sys.exit completely, you could just move the seven lines starting with data = r.json() inside the try: clause? Please be optimistic: the feedback loop is short now, so we might arrive at something acceptable to Sandro soon, and then you'll have your notifications. ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#878088: reportbug: please inform security and lts teams about security update regressions
2018-01-23 15:58 GMT+01:00 Sandro Tosi : > point releases usually include all the security updates from the > previous point release, so they would come from the "main" repo and > not the security one That would only be a problem if they are then no longer available via the security update channel. Otherwise the security source is still listed? > (also apt-cache policy would require to have the > security.d.o source, which is not necessarily the case on every single > machine) You are right that there is no perfect solution. Could matching on "/updates" in the suite part of the source line be a "good enough" heuristic? ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#878088: reportbug: please inform security and lts teams about security update regressions
It might be possible to determine whether an installed package is a security update by inspecting the output of apt-cache policy pkgname. Not sure if this information can also be obtained used python3-apt. ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#539737: reportbug: RFC-1522-encoded headers for mutt
On 3 Aug 2009 Jakub Wilk wrote: > Package: reportbug > Version: 4.5 > Severity: normal > > reportbug encodes non-ASCII headers as per RFC 1522. However, `mutt -H` > does not expect headers to be encoded in such a way. This seems to have been fixed on the mutt side. At least I am unable to reproduce the problem. The reportbug behaviour has not changed. ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#577278: reportbug: Attach options (-A) is not safe for UTF-8 content
On 11 Apr 2010 Jari Aalto wrote: > Package: reportbug > Version: 4.11 > Severity: normal > > > Using the -A optiontion to attach and UTF-8 patch seems to have problems. This seems to have been fixed in the meantime, probably when reportbug was ported to python3. ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#507600: bug script not ready for scripts
On Wed, 03 Dec 2008 05:03:58 +0800 jida...@jidanni.org wrote: > Package: apt > Version: 0.7.19+b1 > File: /usr/share/bug/apt/script > > Your script never expected stdin to be closed. > I.e., one cannot use reportbug --template in a script then. E.g., > $ echo reportbug --template -f /usr/share/bug/apt/script|batch > > May I include your apt configuration (/etc/apt/apt.conf et al)? [Y/n] stty: > standard input: Inappropriate ioctl for device > stty: standard input: Inappropriate ioctl for device Bug scripts are free to use interactive actions, so this is not a bug in apt. On the reportbug side, non-interactive report generation was not possible until the "--no-bug-script" option was implemented in 2010. This was introduced in reportbug 4.12. Since then, this is also documented in the manpage: >--no-bug-script > Do not execute the bug script (if present); this option can be > useful together with --template to suppress > every interactive actions, since some bug scripts can ask > questions. ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#857810: reportbug: crashes if $HOME does not exist, bugreport is lost
On 15 Mar 2017 Thorsten Glaser wrote: > Package: reportbug > Version: 6.6.3 > Severity: important > > I just reported a bug in a cowbuilder chroot, so $HOME pointed to > a nonexistent directory. What I got was this: > > File /tmp/reportbug-node-uglify-20170315-25826-NfKllD saved. > Report will be sent to "Debian Bug Tracking System" > Submit this report on node-uglify (e to edit) [Y|n|a|c|e|i|l|m|p|q|d|t|s|?]? > Traceback (most recent call last): > File "/usr/bin/reportbug", line 2211, in > main() > File "/usr/bin/reportbug", line 1081, in main > return iface.user_interface() > File "/usr/bin/reportbug", line 2203, in user_interface > self.options.envelopefrom) > File "/usr/lib/python2.7/dist-packages/reportbug/submit.py", line 316, in > send_report > msgname = os.path.expanduser(outfile) or ('/var/tmp/%s.bug' % package) > File "/usr/lib/python2.7/posixpath.py", line 261, in expanduser > if not path.startswith('~'): > AttributeError: 'NoneType' object has no attribute 'startswith' This crash was fixed in reportbug 6.6.4 by: commit 3b9cb82c8972b5190fd1364256ce18742b3fd16c Author: Sandro Tosi 2015-04-13 00:14:21 Committer: Sandro Tosi 2015-04-13 00:14:21 correctly handle the case where there is not MTA or smtphost defined; Closes: #782319, #780825, #769055 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#885959: reportbug: marks mime type of attachments as inode/symlink
control: tags -1 patch Thanks for the report! >From 77087cf7a1b16d0be199c301ece62f04f06d5b6f Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Mon, 1 Jan 2018 11:28:25 +0100 Subject: [PATCH] submit.py: dereference symbolic links when detecting file type --- reportbug/submit.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/reportbug/submit.py b/reportbug/submit.py index 4b72ceb..30cfdef 100644 --- a/reportbug/submit.py +++ b/reportbug/submit.py @@ -144,7 +144,7 @@ def mime_attach(body, attachments, charset, body_charset=None): continue ctype = None cset = charset -info = Popen(['file', '--mime', '--brief', attachment], +info = Popen(['file', '--mime', '--brief', '--dereference', attachment], stdout=PIPE, stderr=STDOUT).communicate()[0].decode('ascii') if info: match = re.match(r'([^;, ]*)(,[^;]+)?(?:; )?(.*)', info) -- 2.11.0 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#883577: Bug#883577: locales not installed
On 29-12-2017 01:27, Sandro Tosi wrote: > On Wed, Dec 6, 2017 at 12:48 PM, Nis Martensen wrote: >>> Does reportbug depend on locales being installed or something related? >> >> No, strictly speaking not. In practice, however, often yes. > > Nis, what do you think we should be doing here? i'm tempted to close > this as a user configuration error, but we can also consider > explicitly add locales to depends Depending on locales won't help: Having the locales package installed does not imply that any locales are configured. And it is not needed - the new reportbug version (7.1.8) works even in the "C" locale when there are 8bit characters in the report or in text attachments. ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#868155: reportbug: no input line after pager
> Run “reportbug --offline reportbug” on a terminal with 33 lines. > Exit the pager with “q” then wonder. > > Pressing ^L shows the input line, with no introduction. > > This only happens for packages that cause a pager to show > something before the input fields. Cannot reproduce here. Can you test if the patch attached to #695887 fixes the problem? ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#877650: reportbug: crashes when maintainer's name contains incompatible character
control: tags -1 +patch > Reportbug is not designed to cope with arbitrary encodings of > configuration files, but relies on the user to use consistent encoding > in files and environment. We can and should still prevent reportbug from simply crashing in such cases. So here is a patch. It applies on top of the patch that I sent to #695887. Besides #877650, it should also fix #868022 and #883577. >From c238cf242a8713ccd8bd9a36319e1bd1d8c72bb6 Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Tue, 5 Dec 2017 21:40:03 +0100 Subject: [PATCH 2/2] Prevent Unicode(De|En)codeError with most open() calls We are still getting new bugreports with Unicode(De|En)codingError-Crashes. While users can often fix that by fixing their locale setup, we should prevent crashes where we can. --- bin/reportbug | 35 ++- reportbug/debbugs.py| 2 +- reportbug/submit.py | 5 ++--- reportbug/tempfiles.py | 4 ++-- reportbug/ui/gtk2_ui.py | 5 ++--- reportbug/ui/text_ui.py | 4 ++-- reportbug/utils.py | 15 --- 7 files changed, 35 insertions(+), 35 deletions(-) diff --git a/bin/reportbug b/bin/reportbug index 6a554b6..dff93c2 100755 --- a/bin/reportbug +++ b/bin/reportbug @@ -142,11 +142,10 @@ def include_file_in_report(message, message_filename, """ if inline: try: -fp = open(include_filename) -message += '\n*** %s\n%s' % ( -include_filename, -fp.read()) -fp.close() +with open(include_filename, errors='backslashreplace') as fp: +message += '\n*** {}\n{}'.format( +include_filename, +fp.read()) fp, temp_filename = TempFile( prefix=tempfile_prefix(package_name), dir=draftpath) fp.write(message) @@ -192,7 +191,8 @@ def handle_editing(filename, dmessage, options, sendto, attachments, package, stopmsg(filename) sys.exit(1) -message = open(filename).read() +with open(filename, errors='backslashreplace') as f: +message = f.read() changed = True prompt = 'Submit this report on %s (e to edit)' % package @@ -365,7 +365,8 @@ def handle_editing(filename, dmessage, options, sendto, attachments, package, if 'patch' in newtaglist: patch = True message = message.replace(oldtags, newtags) -open(filename, 'w').write(message) +with open(filename, 'w', errors='backslashreplace') as f: +f.write(message) elif x == 's': skip_editing = True ccemail = ui.get_string( @@ -391,7 +392,7 @@ def handle_editing(filename, dmessage, options, sendto, attachments, package, else: break -return open(filename).read(), patch, justsave +return open(filename, errors='backslashreplace').read(), patch, justsave def find_package_for(filename, notatty=False, pathonly=False): @@ -670,7 +671,7 @@ def offer_configuration(options): 'home directory. Please fix this before using ' 'reportbug again.\n', utils.USERFILE) -fp = os.fdopen(fd, 'w') +fp = os.fdopen(fd, 'w', errors='backslashreplace') print('# reportbug preferences file', file=fp) print('# character encoding: %s' % charset, file=fp) print('# Version of reportbug this preferences file was written by', file=fp) @@ -1168,7 +1169,8 @@ class UI(object): 'reduce its size else the report cannot be sent' % ( self.options.bodyfile, self.options.max_attachment_size)) raise Exception -body = open(self.options.bodyfile).read() +with open(self.options.bodyfile, errors='backslashreplace') as bf: +body = bf.read() except: efail('Unable to read body from file %s.\n', self.options.bodyfile) @@ -1351,11 +1353,8 @@ class UI(object): if self.options.include: for f in self.options.include: if os.path.exists(f): -fp = open(f) -incfiles = '%s\n*** %s\n%s' % ( -incfiles, f, -fp.read()) -fp.close() +with open(f, errors='backslashreplace') as fp: +incfiles += '\n*** {}\n{}'.format(f, fp.read()) else: ewrite("Can't find %s to include!\n", f)
[Reportbug-maint] Bug#883577: locales not installed
> Does reportbug depend on locales being installed or something related? No, strictly speaking not. In practice, however, often yes. One can set the default system locale to "C.UTF-8" (variable LANG in /etc/default/locale) if having locales installed is not desired. Thanks for the very detailed crash log. Reportbug likely left a file in /tmp/. If you still have that file from the first crash, can you check where the non-ASCII character(s) came from? ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#877650: reportbug: crashes when maintainer's name contains incompatible character
Thanks for your report! On 03 Oct 2017 Eckhard Neber wrote: > Detected character set: UTF-8 Your system is set up to use UTF-8 (good). > File "/usr/lib/python3/dist-packages/reportbug/utils.py", line 728, in > get_changed_config_files > for line in fp: > File "/usr/lib/python3.5/codecs.py", line 321, in decode > (result, consumed) = self._buffer_decode(data, self.errors, final) > UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 294: > invalid continuation byte You noticed this crash while trying to report a bug against udftools. If you run `apt-cache show udftools` on the system where you have the udftools package installed, the output should contain a line 'Conffiles:' with a list of one or more filenames and md5sums below. One of these files most likely uses a different fileencoding. (In latin1 encoding, 0xe9 is 'é'.) If you convert that file to UTF-8, the crash should no longer happen. Reportbug is not designed to cope with arbitrary encodings of configuration files, but relies on the user to use consistent encoding in files and environment. Please let us know if this does not fix your problem, otherwise I will close this bug in a while. ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#695887: reportbug locks terminal, refuses echo, requires "reset" after viewing followup
control: forcemerge 695887 849763 882983 control: tags -1 patch These bugs are caused by file descriptors not being closed properly. The attached patch should help. >From 1dbc071c3b966f4fb351948412ffe438de11e62f Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Mon, 4 Dec 2017 21:33:15 +0100 Subject: [PATCH] Make sure some file descriptors will be closed properly --- bin/reportbug | 3 ++- reportbug/submit.py | 3 ++- reportbug/ui/text_ui.py | 18 ++ 3 files changed, 14 insertions(+), 10 deletions(-) diff --git a/bin/reportbug b/bin/reportbug index 7eda598..6a554b6 100755 --- a/bin/reportbug +++ b/bin/reportbug @@ -339,7 +339,8 @@ def handle_editing(filename, dmessage, options, sendto, attachments, package, skip_editing = True if x == 'l': pager = os.environ.get('PAGER', 'sensible-pager') -os.popen(pager, 'w').write(message) +with os.popen(pager, 'w') as p: +p.write(message) else: sys.stdout.write(message) elif x == 't': diff --git a/reportbug/submit.py b/reportbug/submit.py index e6bd50b..ea613cb 100644 --- a/reportbug/submit.py +++ b/reportbug/submit.py @@ -265,7 +265,8 @@ def send_report(body, attachments, mua, fromaddr, sendto, ccaddr, bccaddr, if paranoid and not (template or printonly): pager = os.environ.get('PAGER', 'sensible-pager') try: -os.popen(pager, 'w').write(message) +with os.popen(pager, 'w') as p: +p.write(message) except Exception as e: # if the PAGER exits before all the text has been sent, # it'd send a SIGPIPE, so crash only if that's not the case diff --git a/reportbug/ui/text_ui.py b/reportbug/ui/text_ui.py index b852b79..4a60ff8 100644 --- a/reportbug/ui/text_ui.py +++ b/reportbug/ui/text_ui.py @@ -451,10 +451,9 @@ def show_report(number, system, mirrors, text = 'Original report - %s\n\n%s' % (buginfo.subject, messages[0]) if not skip_pager: -fd = os.popen('sensible-pager', 'w') try: -fd.write(text) -fd.close() +with os.popen('sensible-pager', 'w') as fd: +fd.write(text) except IOError as x: if x.errno == errno.EPIPE: pass @@ -1009,7 +1008,8 @@ def display_report(text, use_pager=True, presubj=False): pager = os.environ.get('PAGER', 'sensible-pager') try: -os.popen(pager, 'w').write(text) +with os.popen(pager, 'w') as p: +p.write(text) except IOError: pass @@ -1023,9 +1023,10 @@ def spawn_editor(message, filename, editor, charset='utf-8'): # Move the cursor for lazy buggers like me; add your editor here... ourline = 0 -for (lineno, line) in enumerate(open(filename)): -if line == '\n' and not ourline: -ourline = lineno + 2 +with open(filename) as f: +for (lineno, line) in enumerate(f): +if line == '\n' and not ourline: +ourline = lineno + 2 opts = '' if 'vim' in edname: @@ -1061,7 +1062,8 @@ def spawn_editor(message, filename, editor, charset='utf-8'): if '&' in editor: return (None, 1) -newmessage = open(filename).read() +with open(filename) as f: +newmessage = f.read() if newmessage == message: ewrite('No changes were made in the editor.\n') -- 2.11.0 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#564112: use python-apt to look up package short descriptions
control: tags -1 patch This also fixes #876373 Other related bugs: #502860, #599290, #882130 >From 4612729648fd17955e80a250baca120ce9747251 Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Mon, 4 Dec 2017 13:39:38 +0100 Subject: [PATCH] Look up short package descriptions using python3-apt --- debian/control | 2 +- reportbug/utils.py | 8 +++- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/debian/control b/debian/control index 9afcc7d..144dcfa 100644 --- a/debian/control +++ b/debian/control @@ -12,7 +12,7 @@ Homepage: http://alioth.debian.org/projects/reportbug/ Package: reportbug Architecture: all -Depends: ${misc:Depends}, ${python3:Depends}, apt, python3-reportbug (= ${source:Version}) +Depends: ${misc:Depends}, ${python3:Depends}, apt, python3-reportbug (= ${source:Version}), python3-apt Suggests: postfix | exim4 | mail-transport-agent, gnupg | pgp, debconf-utils (>> 1.1.0), debsums (>= 2.0.47), file (>> 1.30), dlocate, python3-urwid, python3-gi, python3-gi-cairo, gir1.2-gtk-3.0, gir1.2-vte-2.91, python3-gtkspellcheck, xdg-utils, emacs24-bin-common | emacs25-bin-common, claws-mail (>= 3.8.0) Description: reports bugs in the Debian distribution reportbug is a tool designed to make the reporting of bugs in Debian diff --git a/reportbug/utils.py b/reportbug/utils.py index 8139668..c9eb73a 100644 --- a/reportbug/utils.py +++ b/reportbug/utils.py @@ -24,6 +24,7 @@ import sys import os import re import platform +import apt try: import pwd @@ -550,6 +551,8 @@ def get_source_package(package): retlist = [] found = {} +apt_cache = apt.Cache() + data = get_command_output('apt-cache showsrc ' + pipes.quote(package)) binre = re.compile(r'^Binary: (.*)$') for line in data.split('\n'): @@ -560,7 +563,10 @@ def get_source_package(package): packages += packlist for p in packages: -desc = available_package_description(p) +try: +desc = apt_cache[p].versions[0].summary +except KeyError: +continue if desc and (p not in found): retlist += [(p, desc)] found[p] = desc -- 2.11.0 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#851502: Still see this bug
control: tags 851502 unreproducible moreinfo > I would like to request severity raised to serious. Michael, it seems not many people can actually reproduce the bug. I've tried on stretch and current sid, and reportbug works just fine for me. Sandro has already pointed out that reportbug requires python 3.5 or higher and won't work with python 3.4. You are the first who is seeing the problem even with python 3.5. Can you offer more information? ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#868154: reportbug: displays MIME type of attachment
control: tags -1 patch On 12 Jul 2017 Thorsten Glaser wrote: > I think the “application/x-xz; charset=binary” line should go > elsewhere, perhaps into the eMail? It is the output of a debug print() inserted while porting the code to python 3. It can just be dropped. Patch attached. >From bf3724a200d9db947c77c1d9ce0eb61536a0bade Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Sat, 9 Sep 2017 23:41:48 +0200 Subject: [PATCH 3/3] Drop leftover debug print() --- reportbug/submit.py | 1 - 1 file changed, 1 deletion(-) diff --git a/reportbug/submit.py b/reportbug/submit.py index 4940538..1eb1b71 100644 --- a/reportbug/submit.py +++ b/reportbug/submit.py @@ -147,7 +147,6 @@ def mime_attach(body, attachments, charset, body_charset=None): cset = charset info = Popen(['file', '--mime', '--brief', attachment], stdout=PIPE, stderr=STDOUT).communicate()[0].decode('ascii') -print(info) if info: match = re.match(r'([^;, ]*)(,[^;]+)?(?:; )?(.*)', info) if match: -- 2.11.0 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#849765: Bug#849765: reportbug: produces invalid mails when there are long lines
control: tags -1 patch On 30 Dec 2016 Adam Borowski wrote: > RFC 821: > # The maximum total length of a text line including the is 1000 > # characters (but not counting the leading dot duplicated for transparency). > > Some MTAs are said to have a bit shorter limits, such as 990 bytes. Here is a patch that fixes the problem in reportbug. >From d32fb670bcdf622b9e2b3d45e88abd8cf0e2d711 Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Thu, 7 Sep 2017 18:17:00 +0200 Subject: [PATCH 2/2] Use quoted-printable encoding for too long lines --- reportbug/submit.py | 21 ++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/reportbug/submit.py b/reportbug/submit.py index e6bd50b..4940538 100644 --- a/reportbug/submit.py +++ b/reportbug/submit.py @@ -108,12 +108,27 @@ def sign_message(body, fromaddr, package='x', pgp_addr=None, sign='gpg', draftpa body = None return body +def _MIMEText_wrapper(text): +msg = MIMEText(text) +# Too long lines need to be encoded (see RFC2822), but MIMEText does +# not yet handle this for us. +# Since utf-8 will already be base64-encoded at this point, we only +# need to deal with the us-ascii case. +if msg.get_content_charset() == 'us-ascii' and \ +max(len(l) for l in text.splitlines()) > 980: +email.encoders.encode_quopri(msg) +# due to a bug in the email library, the result now has two CTE +# headers, only one of which is correct. Delete both and set the +# correct value. +del msg['Content-Transfer-Encoding'] +msg['Content-Transfer-Encoding'] = 'quoted-printable' +return msg def mime_attach(body, attachments, charset, body_charset=None): mimetypes.init() message = MIMEMultipart('mixed') -bodypart = MIMEText(body) +bodypart = _MIMEText_wrapper(body) bodypart.add_header('Content-Disposition', 'inline') message.preamble = 'This is a multi-part MIME message sent by reportbug.\n\n' message.epilogue = '' @@ -154,7 +169,7 @@ def mime_attach(body, attachments, charset, body_charset=None): if maintype == 'text': try: with open(attachment, 'rU') as fp: -part = MIMEText(fp.read()) +part = _MIMEText_wrapper(fp.read()) except UnicodeDecodeError: fp = open(attachment, 'rb') part = MIMEBase(maintype, subtype) @@ -216,7 +231,7 @@ def send_report(body, attachments, mua, fromaddr, sendto, ccaddr, bccaddr, ewrite("Error: Message creation failed, not sending\n") mua = mta = smtphost = None else: -message = MIMEText(body) +message = _MIMEText_wrapper(body) # Standard headers message['From'] = fromaddr -- 2.11.0 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#872780: reportbug: When filing ITP report and realname has UTF-8 character, the name will be encoded in RFC2047, which it shouldn't
control: tags -1 patch On 2017-08-21 Yao Wei wrote:> See Bug #872773 for example, the owner is garbled like this: > > Owner: =?utf-8?b?WWFvIFdlaSAo6a2P6YqY5bu3KQ==?= > > while it should be this: > > Owner: Yao Wei (魏銘廷) Yes. Here's a patch. >From ae8bb0186e4469ec02f1bfb5c25fa33b78744cc2 Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Tue, 5 Sep 2017 22:13:14 +0200 Subject: [PATCH] Do not RFC2047-encode Owner pseudo header --- reportbug/debbugs.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/reportbug/debbugs.py b/reportbug/debbugs.py index 1c65075..d941d74 100644 --- a/reportbug/debbugs.py +++ b/reportbug/debbugs.py @@ -682,7 +682,8 @@ def handle_wnpp(package, bts, ui, fromaddr, timeout, online=True, http_proxy=Non if tag == 'ITP': headers.append('X-Debbugs-CC: debian-de...@lists.debian.org') -pseudos.append('Owner: %s' % fromaddr) +pseudos.append('Owner: {}'.format( +email.header.make_header(email.header.decode_header(fromaddr ui.log_message('Your report will be carbon-copied to debian-devel, ' 'per Debian policy.\n') -- 2.11.0 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#866596: querybts: launch_mbox_reader() strips NL characters
Daniel Shahaf wrote: > > The .splitlines() calls returns a list of strings that do *not* contain > newlines (\n characters), and the following .write() does not add its own > \n's. Oops. That was me. Thanks for finding, debugging and reporting this! Patch attached. >From 7b0f010797dc8ad52838604bb9eb3de9e3f5d9a3 Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Fri, 30 Jun 2017 23:19:32 +0200 Subject: [PATCH] Add line endings when processing strings line by line Bug introduced in d7e9eb8cc5d2efa4543bd35a806cbc36d7419e5e --- reportbug/debbugs.py | 4 ++-- reportbug/utils.py | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/reportbug/debbugs.py b/reportbug/debbugs.py index f2e5830..eade68a 100644 --- a/reportbug/debbugs.py +++ b/reportbug/debbugs.py @@ -969,7 +969,7 @@ def parse_html_report(number, url, http_proxy, timeout, followups=False, cgi=Tru parser = BTSParser(cgi=cgi, followups=followups) for line in page.splitlines(): -parser.feed(line) +parser.feed(line + '\n') parser.close() items = parser.preblock @@ -1121,7 +1121,7 @@ def get_reports(package, timeout, system='debian', mirrors=None, version=None, parser = BTSParser() for line in page.splitlines(): -parser.feed(line) +parser.feed(line + '\n') parser.close() return parser.bugcount, parser.title, parser.hierarchy diff --git a/reportbug/utils.py b/reportbug/utils.py index 8139668..3e53a3c 100644 --- a/reportbug/utils.py +++ b/reportbug/utils.py @@ -1205,7 +1205,7 @@ def launch_mbox_reader(cmd, url, http_proxy, timeout): (fd, fname) = TempFile() try: for line in mbox.splitlines(): -fd.write(line) +fd.write(line + '\n') fd.close() if cmd is not None: try: -- 2.1.4 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#862152: crash when $DISPLAY is defined
On 29-05-2017 21:58, Erwan David wrote: > X display is remote (on a mac in my case, but remote). Direct access to > driver is thus not possible. If you do X remote display (which ssh is capable of doing), then something must communicate with some driver. Is any of your local machine or the remote machine using the mentioned driver? > But, reportbug is configured to use a text interface, no graphic layer > should interfere with this. We agree here :) ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#863322: reportbug: Wrong subject and package description while filling a WNPP RFA
One more patch on top, with some improvements. >From 1e965eb715d80dd07f4b930db0e22a8011c03bae Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Sun, 28 May 2017 13:07:25 +0200 Subject: [PATCH 3/3] utils.py: Do not use localized package description .. and unify the regexps used for getting the description, so that we never accidentally use the Description-md5 field. --- reportbug/utils.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/reportbug/utils.py b/reportbug/utils.py index 51c9d23..e238a8d 100644 --- a/reportbug/utils.py +++ b/reportbug/utils.py @@ -362,7 +362,7 @@ def get_package_status(package, avail=False): packarg = pipes.quote(package) if avail: output = get_command_output( -"apt-cache show %s 2>/dev/null" % packarg) +"LC_ALL=C.UTF-8 apt-cache show %s 2>/dev/null" % packarg) else: output = get_command_output( "COLUMNS=79 dpkg --status %s 2>/dev/null" % packarg) @@ -525,7 +525,7 @@ def get_avail_database(): def available_package_description(package): data = get_command_output('apt-cache show ' + pipes.quote(package)) -descre = re.compile('^Description(?:-.*)?: (.*)$') +descre = re.compile('^Description(?:-[a-zA-Z]+)?: (.*)$') for line in data.split('\n'): m = descre.match(line) if m: @@ -599,7 +599,7 @@ def get_package_info(packages, skip_notfound=False): packob = re.compile('^Package: (?P.*)$', re.MULTILINE) statob = re.compile('^Status: (?P.*)$', re.MULTILINE) versob = re.compile('^Version: (?P.*)$', re.MULTILINE) -descob = re.compile('^Description(?:-.*)?: (?P.*)$', re.MULTILINE) +descob = re.compile('^Description(?:-[a-zA-Z]+)?: (?P.*)$', re.MULTILINE) ret = [] for p in packinfo: -- 2.1.4 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#861153: Bug#861153: reportbug: Architecture field split into two lines
Hey Sandro! > should we revert to the previous behavior of get_command_output before > introducing other regressions? Having re-read the code again, I can't see any risk of introducing other regressions. Only lsb_release_info() and get_arch() directly return the result of get_command_output(), all other callers do some parsing and do not care about newlines at the end of the output. The alternative you have in mind is changing get_command_output like this, right? : -return subprocess.run([...]).stdout.decode([...]) +return subprocess.run([...]).stdout.decode([...]).strip() This will work as well. I slightly prefer my other patch, but it does not matter much. ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#863322: reportbug: Wrong subject and package description while filling a WNPP RFA
control: tags 863322 patch On Thu, 25 May 2017 Nelson A. de Oliveira wrote: > While filling a reportbug against wnpp and seleting RFA (package is > pngnq) I got this: > > = > Subject: RFA: pngnq -- 8dc4366236a6a8b60a7651dc12063d65 > Package: wnpp > Severity: normal > > I request an adopter for the pngnq package. > > The package description is: > scope::utility, use::compressing, works-with-format::png, > works-with::image, works-with::image:raster > = > > ie, the package description in "Subject" is wrong (it's using > "Description-md5" instead the short description) and the description > itself is using text from tags. Thank you for the report. Attaching two patches. Both together should fix this. >From 924f153462f456840211ee6ba852a041156ffe02 Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Fri, 26 May 2017 19:08:56 +0200 Subject: [PATCH 1/2] utils.get_package_status(): properly capture package description The code trying to capture the package description has two issues: 1. The regexp used for detecting the description field also matches on the Description-md5 field, which it should not. 2. Lines starting with a space are assumed to belong to the long package description, even if they actually belong to another field listed after the description. This can be fixed by only matching a Description field with alphabetic characters before the colon, and a "descmode" similar to the already existing "confmode" in the parser. --- reportbug/utils.py | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/reportbug/utils.py b/reportbug/utils.py index e134bcd..5ff3dbe 100644 --- a/reportbug/utils.py +++ b/reportbug/utils.py @@ -338,7 +338,7 @@ def get_package_status(package, avail=False): statusre = re.compile('Status: ') originre = re.compile('Origin: ') bugsre = re.compile('Bugs: ') -descre = re.compile('Description(?:-.*)?: ') +descre = re.compile('Description(?:-[a-zA-Z]+)?: ') fullre = re.compile(' ') srcre = re.compile('Source: ') sectionre = re.compile('Section: ') @@ -351,6 +351,7 @@ def get_package_status(package, avail=False): recommends = [] suggests = [] confmode = False +descmode = False state = '' try: @@ -371,6 +372,12 @@ def get_package_status(package, avail=False): if not line: continue +if descmode: +if line[0] == ' ': +fulldesc.append(line) +else: +descmode = False + if confmode: if line[:2] != ' /': confmode = False @@ -392,6 +399,7 @@ def get_package_status(package, avail=False): (crud, bugs) = line.split(": ", 1) elif descre.match(line): (crud, desc) = line.split(": ", 1) +descmode = True elif dependsre.match(line): (crud, thisdepends) = line.split(": ", 1) # Remove versioning crud @@ -419,8 +427,6 @@ def get_package_status(package, avail=False): src_name = src_name.split()[0] elif sectionre.match(line): crud, section = line.split(": ", 1) -elif desc and line[0] == ' ': -fulldesc.append(line) installed = False if status: -- 2.1.4 >From d2604f7ea963ec6a2766c427c1e634a183bd96f8 Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Fri, 26 May 2017 19:21:30 +0200 Subject: [PATCH 2/2] Use apt instead of dpkg for availability information "dpkg --print-avail pkg-name" no longer shows the long package description. Since we want it, so we must get it from somewhere else. "apt-cache show pkg-name" provides it. Also, man dpkg-query(1) says: Users of APT-based frontends should use "apt-cache show package-name" instead [of dpkg --print-avail package-name] as the available file is only kept up-to-date when using dselect. Nowadays APT has many more users than dselect, according to popcon.debian.org. --- reportbug/utils.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/reportbug/utils.py b/reportbug/utils.py index 5ff3dbe..51c9d23 100644 --- a/reportbug/utils.py +++ b/reportbug/utils.py @@ -362,7 +362,7 @@ def get_package_status(package, avail=False): packarg = pipes.quote(package) if avail: output = get_command_output( -"COLUMNS=79 dpkg --print-avail %s 2>/dev/null" % packarg) +"apt-cache show %s 2>/dev/null" % packarg) else: output = get_command_output( "COLUMNS=79 dpkg --status %s 2>/dev/null" % packarg) -- 2.1.4 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#861153: reportbug: Architecture field split into two lines
Paul Wise wrote: > Package: reportbug > Version: 7.1.6 > Severity: minor > > The architecture field is currently split into two lines: > > Architecture: amd64 > (x86_64) > > With earlier versions of reportbug it was only one line: > > https://bugs.debian.org/823456 > Architecture: amd64 (x86_64) Thanks for reporting this. It is only one symptom of the actual bug. Let's fix this before we find out exactly what else was broken by my patch in reportbug 7.1.6. Follow-up patch attached. >From 892d6fb67fbe2e9198884e020e8f1926c7a1188b Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Tue, 25 Apr 2017 22:30:40 +0200 Subject: [PATCH] utils.py: Avoid unwanted newlines In contrast to subprocess.getoutput(), the new get_command_output() function does not strip trailing newlines from the returned output. Failure to account for this in the utils/get_arch() function caused various subtle breakage in multiple places. Strip newlines in get_arch() to fix the resulting problems in bin/reportbug, reportbug/checkbuildd.py, reportbug/checkversions.py and reportbug/debbugs.py. In lsb_release_info(), do not add the (now unnecessary) additional newline. --- reportbug/utils.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/reportbug/utils.py b/reportbug/utils.py index e3084bb..086dd9b 100644 --- a/reportbug/utils.py +++ b/reportbug/utils.py @@ -783,11 +783,11 @@ def get_debian_release_info(): def lsb_release_info(): -return get_command_output('lsb_release -a 2>/dev/null') + '\n' +return get_command_output('lsb_release -a 2>/dev/null') def get_arch(): -arch = get_command_output('COLUMNS=79 dpkg --print-architecture 2>/dev/null') +arch = get_command_output('COLUMNS=79 dpkg --print-architecture 2>/dev/null').strip() if not arch: un = os.uname() arch = un[4] -- 2.1.4 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#859274: /usr/bin/querybts: double-faults when using -m option
control: tags 859274 patch Thanks for reporting this. Here's a patch. >From cbe84151785b180188d5df9c26f42714e82e9451 Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Sat, 22 Apr 2017 11:27:37 +0200 Subject: [PATCH 1/2] Finish open_url return type conversion The return type of reportbug/urlutils/open_url() was changed in commit 5cf006943001a94bfafe0a40adbb1241950853de from http.client.HTTPResponse to string. A few remaining open_url() callers have not yet been updated, leading to breakage such as https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=859274 This commit updates the remaining callers to deal with strings, and updates a few places to iterate over lines rather than characters. --- bin/querybts | 4 ++-- reportbug/debbugs.py | 23 +++ reportbug/utils.py | 2 +- 3 files changed, 6 insertions(+), 23 deletions(-) diff --git a/bin/querybts b/bin/querybts index 229dde9..ee79949 100755 --- a/bin/querybts +++ b/bin/querybts @@ -168,7 +168,7 @@ def main(): url = debbugs.get_report_url(options.system, num, options.archived, mbox=True) try: report = urlutils.open_url(url, timeout=options.timeout) -sys.stdout.write(report.read()) +sys.stdout.write(report) except urlutils.urllib2.URLError as ex: print("Error while accessing mbox report (%s)." % ex, file=sys.stderr) else: @@ -176,7 +176,7 @@ def main(): url = debbugs.get_report_url(options.system, num, options.archived, mbox=True) try: report = urlutils.open_url(url, timeout=options.timeout) -sys.stdout.write(report.read()) +sys.stdout.write(report) except urlutils.urllib2.URLError as ex: print("Error while accessing mbox report (%s)." % ex, file=sys.stderr) sys.exit(1) diff --git a/reportbug/debbugs.py b/reportbug/debbugs.py index 19ee02e..d71b303 100644 --- a/reportbug/debbugs.py +++ b/reportbug/debbugs.py @@ -967,16 +967,10 @@ def parse_html_report(number, url, http_proxy, timeout, followups=False, cgi=Tru return None parser = BTSParser(cgi=cgi, followups=followups) -for line in page: +for line in page.splitlines(): parser.feed(line) parser.close() -try: -page.fp._sock.recv = None -except: -pass -page.close() - items = parser.preblock title = "#%d: %s" % (number, parser.title) @@ -1011,13 +1005,7 @@ def parse_mbox_report(number, url, http_proxy, timeout, followups=False): return None # Make this seekable -wholefile = io.StringIO(page.read()) - -try: -page.fp._sock.recv = None -except: -pass -page.close() +wholefile = io.StringIO(page) mbox = mailbox.UnixMailbox(wholefile, msgfactory) title = '' @@ -1131,14 +1119,9 @@ def get_reports(package, timeout, system='debian', mirrors=None, version=None, #return (0, None, None) parser = BTSParser() -for line in page: +for line in page.splitlines(): parser.feed(line) parser.close() -try: -page.fp._sock.recv = None -except: -pass -page.close() return parser.bugcount, parser.title, parser.hierarchy diff --git a/reportbug/utils.py b/reportbug/utils.py index e134bcd..2a5bea1 100644 --- a/reportbug/utils.py +++ b/reportbug/utils.py @@ -1196,7 +1196,7 @@ def launch_mbox_reader(cmd, url, http_proxy, timeout): return (fd, fname) = TempFile() try: -for line in mbox: +for line in mbox.splitlines(): fd.write(line) fd.close() if cmd is not None: -- 2.1.4 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#857013: Fixing #857013
control: tags 857013 patch Thanks Michal for finding and reporting this bug! >From 9d8f610dedbd831961ed2d30d96782c569c60d5c Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Sat, 22 Apr 2017 11:43:19 +0200 Subject: [PATCH 2/2] utils.py: Fix IndexError: list index out of range re.findall() called with a regexp containing groups will return a list of tuples. In our case: a list containing zero or exactly one tuple. Since we are storing the result in a list anyway, simply add the two lists and do not try to extract an element first. If re.findall() returns an empty list, this does nothing and will no longer crash. --- reportbug/utils.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/reportbug/utils.py b/reportbug/utils.py index 2a5bea1..e3084bb 100644 --- a/reportbug/utils.py +++ b/reportbug/utils.py @@ -376,7 +376,7 @@ def get_package_status(package, avail=False): confmode = False else: # re is used to identify also conffiles with spaces in the name -conffiles = conffiles + [re.findall(r' (.+) ([0-9a-f]+).*$', line)[0]] +conffiles += re.findall(r' (.+) ([0-9a-f]+).*$', line) if versionre.match(line): (crud, pkgversion) = line.split(": ", 1) -- 2.1.4 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#857013: Fixing #857013
This might work, too? : - conffiles = conffiles + [re.findall(r' (.+) ([0-9a-f]+).*$', line)[0]] + conffiles += re.findall(r' (.+) ([0-9a-f]+).*$', line) ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#857794: reportbug: crash when encountering some non-ASCII characters
Hi Stuart, Thanks for your comments! On 22-03-2017 02:45, Stuart Prescott wrote: > C.UTF-8 is provided within glibc (it's in the libc-bin package so it > is always available). Is it worth setting that as the locale for all > communication with subprocesses? It strikes me that could simplify > some parts of this encoding handling. We could do this if we knew that we'd then always get UTF-8 encoded data. Is it guaranteed that all subprocesses will produce text output encoded in the charset associated with the current locale? Or are the tools just passing on what they get from their input files? https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=848729 has a case where reportbug ran into such "foreign charset" data; setting LC_ALL=C.UTF-8 for reportbug would not have helped there. Things might be easier if the subprocesses called by reportbug would only output the bits of information needed. Unfortunately they don't, they output much more, and reportbug needs to deal with it and parse it. Most information needed is ASCII-only anyway, so limiting other tools' output would be ideal. We just can't fix that in reportbug. So the minimum we should do is avoid the crashes, and changing the error handler allows us to do that. > If we're touching every invocation subprocess.* anyway, can we also > take this opportunity to get rid of any invocations via the shell? > > +def get_command_output(cmd): +use_shell = False +if > isinstance(cmd, str) and ' ' in cmd: +use_shell = True + > return subprocess.run(cmd, shell=use_shell, > stdout=subprocess.PIPE).stdout.decode(errors='backslashreplace') > > specifically, my suggestion is that we *require* cmd to be a list or > tuple and always force use_shell=False. Then we never need to worry > about nasty characters going near the shell, there's also one less > subprocess being used. > > Associated with that: > > * redirection of stderr to /dev/null can be done in python instead > > * the uses of COLUMNS=79 are either no-ops because the command > doesn't respect COLUMNS anyway or no-ops because the command ignores > COLUMNS when invoked within a pipe. No problem with that idea :) I just tried to keep the changes to a minimum since we're in deep freeze at the moment. Cheers, Nis ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#857794: reportbug: crash when encountering some non-ASCII characters
Thank you for your report, and for the nice recipe to reproduce the bug. Can you try the attached patch? >From f42c5879b91b11a986e93f7f92244cf938dae0fb Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Tue, 21 Mar 2017 22:23:49 +0100 Subject: [PATCH] Stop using subprocess.getoutput() To avoid crashes with non-ascii characters and locale mismatch, we need to use something that allows specifying an error handler. getoutput is a legacy API that does not allow that, so switch to subprocess.run() and use a little wrapper function for readability. --- reportbug/utils.py | 27 +-- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/reportbug/utils.py b/reportbug/utils.py index 2de91ae..e134bcd 100644 --- a/reportbug/utils.py +++ b/reportbug/utils.py @@ -180,6 +180,13 @@ def search_pipe(searchfile, use_dlocate=True): return (pipe, use_dlocate) +def get_command_output(cmd): +use_shell = False +if isinstance(cmd, str) and ' ' in cmd: +use_shell = True +return subprocess.run(cmd, shell=use_shell, stdout=subprocess.PIPE).stdout.decode(errors='backslashreplace') + + def query_dpkg_for(filename, use_dlocate=True): try: x = os.getcwd() @@ -353,10 +360,10 @@ def get_package_status(package, avail=False): packarg = pipes.quote(package) if avail: -output = subprocess.getoutput( +output = get_command_output( "COLUMNS=79 dpkg --print-avail %s 2>/dev/null" % packarg) else: -output = subprocess.getoutput( +output = get_command_output( "COLUMNS=79 dpkg --status %s 2>/dev/null" % packarg) for line in output.split(os.linesep): @@ -511,7 +518,7 @@ def get_avail_database(): def available_package_description(package): -data = subprocess.getoutput('apt-cache show ' + pipes.quote(package)) +data = get_command_output('apt-cache show ' + pipes.quote(package)) descre = re.compile('^Description(?:-.*)?: (.*)$') for line in data.split('\n'): m = descre.match(line) @@ -523,7 +530,7 @@ def available_package_description(package): def get_source_name(package): packages = [] -data = subprocess.getoutput('apt-cache showsrc ' + pipes.quote(package)) +data = get_command_output('apt-cache showsrc ' + pipes.quote(package)) packre = re.compile(r'^Package: (.*)$') for line in data.split('\n'): m = packre.match(line) @@ -537,7 +544,7 @@ def get_source_package(package): retlist = [] found = {} -data = subprocess.getoutput('apt-cache showsrc ' + pipes.quote(package)) +data = get_command_output('apt-cache showsrc ' + pipes.quote(package)) binre = re.compile(r'^Binary: (.*)$') for line in data.split('\n'): m = binre.match(line) @@ -704,7 +711,7 @@ def get_changed_config_files(conffiles, nocompress=False): confinfo[filename] = msg continue -filemd5 = subprocess.getoutput('md5sum ' + pipes.quote(filename)).split()[0] +filemd5 = get_command_output('md5sum ' + pipes.quote(filename)).split()[0] if filemd5 == md5sum: continue @@ -732,7 +739,7 @@ DISTORDER = ['oldstable', 'stable', 'testing', 'unstable', 'experimental'] def get_debian_release_info(): debvers = debinfo = verfile = warn = '' dists = [] -output = subprocess.getoutput('apt-cache policy 2>/dev/null') +output = get_command_output('apt-cache policy 2>/dev/null') if output: mre = re.compile('\s+(\d+)\s+.*$\s+release\s.*o=(Ubuntu|Debian|Debian Ports),a=([^,]+),', re.MULTILINE) found = {} @@ -776,11 +783,11 @@ def get_debian_release_info(): def lsb_release_info(): -return subprocess.getoutput('lsb_release -a 2>/dev/null') + '\n' +return get_command_output('lsb_release -a 2>/dev/null') + '\n' def get_arch(): -arch = subprocess.getoutput('COLUMNS=79 dpkg --print-architecture 2>/dev/null') +arch = get_command_output('COLUMNS=79 dpkg --print-architecture 2>/dev/null') if not arch: un = os.uname() arch = un[4] @@ -791,7 +798,7 @@ def get_arch(): def get_multiarch(): -out = subprocess.getoutput('COLUMNS=79 dpkg --print-foreign-architectures 2>/dev/null') +out = get_command_output('COLUMNS=79 dpkg --print-foreign-architectures 2>/dev/null') return ', '.join(out.splitlines()) -- 2.1.4 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#848729: It is grave
Hi Ondřej, Ondřej Surý wrote: > As this bug is preventing me from filling any bugs at all and it fails > only after I press "Y", so the content of the lengthy unblock bug is > gone... and this puts my laptop in danger of my rage :). > > Traceback (most recent call last): > File "/usr/bin/reportbug", line 2234, in > main() > File "/usr/bin/reportbug", line 1107, in main > return iface.user_interface() > File "/usr/bin/reportbug", line 2225, in user_interface > self.options.envelopefrom) > File "/usr/lib/python3/dist-packages/reportbug/submit.py", line 209, > in send_report > (message, failed) = mime_attach(body, attachments, charset, > body_charset) > File "/usr/lib/python3/dist-packages/reportbug/submit.py", line 157, > in mime_attach > part = MIMEText(fp.read()) > File "/usr/lib/python3.5/codecs.py", line 321, in decode > (result, consumed) = self._buffer_decode(data, self.errors, final) > UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe5 in position > 296505: invalid continuation byte > > and my locale is: > > $ locale > LANG=en_DK.UTF-8 Your Traceback is different from the others. This bug is not fixed yet. Did you try to attach a text(plain/html/etc) file in a different encoding than UTF-8? As a workaround, you could gzip that file. Attached a patch that should fix the problem. >From 0566a9cec6c9bd561db0128a58c84b9db0a5d1c4 Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Tue, 14 Feb 2017 22:21:29 +0100 Subject: [PATCH] Handle text attachments in different encodings --- reportbug/submit.py | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/reportbug/submit.py b/reportbug/submit.py index 5cc1231..b3025b3 100644 --- a/reportbug/submit.py +++ b/reportbug/submit.py @@ -153,9 +153,15 @@ def mime_attach(body, attachments, charset, body_charset=None): maintype, subtype = ctype.split('/', 1) if maintype == 'text': -fp = open(attachment, 'rU') -part = MIMEText(fp.read()) -fp.close() +try: +with open(attachment, 'rU') as fp: +part = MIMEText(fp.read()) +except UnicodeDecodeError: +fp = open(attachment, 'rb') +part = MIMEBase(maintype, subtype) +part.set_payload(fp.read()) +fp.close() +email.encoders.encode_base64(part) elif maintype == 'message': fp = open(attachment, 'rb') part = MIMEMessage(email.message_from_file(fp), -- 2.1.4 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#853269: reportbug: attempting to use gtk2 interface on text console segfaults, should fallback to text
> reportbug should fallback to the text ui if run from a text console. Here is a patch that implements this. Regards, Nis >From a2da701d8f1fa1a5b33392a6ee8b4d16171466ce Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Wed, 1 Feb 2017 23:28:51 +0100 Subject: [PATCH 4/4] gtk2_ui: without graphical display, fall back to text UI --- reportbug/ui/gtk2_ui.py | 4 1 file changed, 4 insertions(+) diff --git a/reportbug/ui/gtk2_ui.py b/reportbug/ui/gtk2_ui.py index 3d74a7a..23d598b 100644 --- a/reportbug/ui/gtk2_ui.py +++ b/reportbug/ui/gtk2_ui.py @@ -22,6 +22,10 @@ from reportbug.exceptions import UINotImportable +import os +if not ('DISPLAY' in os.environ or 'WAYLAND_DISPLAY' in os.environ): +raise UINotImportable('No graphical display detected, falling back to text UI.') + try: import gi -- 2.1.4 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#853269: reportbug: attempting to use gtk2 interface on text console segfaults, should fallback to text
Package: reportbug Version: 7.1.4 Severity: normal $ reportbug --ui gtk2 Unable to init server: Could not connect: Connection refused Unable to init server: Could not connect: Connection refused Segmentation fault $ reportbug should fallback to the text ui if run from a text console. ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#853159: [reportbug] package reportbug should recommend to install the package "gir1.2-vte-2.9"
Package: reportbug Followup-For: Bug #853159 It may make sense to create a new reportbug-gtk-ui binary package with proper Depends: instead of Suggests:, but this is too late for the upcoming stable release. In the meantime, we can at least fix the crashes and fall back to the text ui as intended. The attached patch fixes both #852652 and #853159. >From 7c4ef828a097583f71144e2b785e15c9a7895a55 Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Mon, 30 Jan 2017 23:52:20 +0100 Subject: [PATCH 3/3] ui/gtk2_ui: fix incomplete fallback to text ui This fixes crashes when the python3-gi-cairo and gir1.2-vte-2.91 packages are not installed, so that the fallback can work in these cases. --- reportbug/ui/gtk2_ui.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/reportbug/ui/gtk2_ui.py b/reportbug/ui/gtk2_ui.py index f60a1eb..3d74a7a 100644 --- a/reportbug/ui/gtk2_ui.py +++ b/reportbug/ui/gtk2_ui.py @@ -42,6 +42,8 @@ try: gi.require_version('Gtk', '3.0') from gi.repository import Gtk + +gi.require_foreign('cairo') except ImportError: raise UINotImportable('Please install the python3-gi and gir1.2-gtk-3.0 packages to use this interface.') @@ -1777,7 +1779,7 @@ def initialize(): try: gi.require_version('Vte', '2.91') from gi.repository import Vte -except ImportError: +except (ImportError, ValueError): message = """Please install the %s package to use the GTK+(known as 'gtk2' in reportbug) interface. Falling back to 'text' interface.""" dialog = Gtk.MessageDialog(None, Gtk.DialogFlags.MODAL | Gtk.DialogFlags.DESTROY_WITH_PARENT, -- 2.1.4 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#749884: reportbug: doesn't grok :any qualifiers in Depends
The attached patch fixes this by dropping the qualifier before looking up the package status information. It is possible that something else is needed for proper multi-arch support. Regards, Nis >From 3df5c44f472609894488df47deb30f9f3281dc86 Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Sat, 28 Jan 2017 13:53:29 +0100 Subject: [PATCH 2/2] utils.get_dependency_info: ignore arch qualifiers Without this, we may look for packages like 'python3:any' from the dependencies and then not find any installed versions. --- reportbug/utils.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/reportbug/utils.py b/reportbug/utils.py index 2de91ae..22a6635 100644 --- a/reportbug/utils.py +++ b/reportbug/utils.py @@ -648,6 +648,8 @@ def get_dependency_info(package, depends, rel="depends on"): dependencies = [] for dep in depends: +# drop possible architecture qualifier from package names +dep = [d.split(':')[0] for d in dep] for bit in dep: dependencies.append((tuple(dep), bit)) -- 2.1.4 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#848729: reportbug UnicodeDecodeError: possible patch, please test
Control: tags 848729 patch On 26-01-2017 00:47, Nis Martensen wrote: > Can you please test if the attached patch fixes this bug? > > (This bug already has multiple duplicates -- Bugs #848729, #849358, > #850687, #851865, #852129 all seem related.) Recipe to reproduce the bug: 1. (Optional) Configure a real-name with non-ascii characters in your .reportbugrc 2. LC_ALL=C export LC_ALL 3. Try to report a bug on reportbug (using reportbug). Reportbug 7.1.4 fails immediately. It works after applying the patch. Regards, Nis ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#848729: reportbug UnicodeDecodeError: possible patch, please test
Can you please test if the attached patch fixes this bug? (This bug already has multiple duplicates -- Bugs #848729, #849358, #850687, #851865, #852129 all seem related.) >From f1bbfcfd8ef09ae86f06ea1aa815e0a4c4e51e92 Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Thu, 26 Jan 2017 00:09:43 +0100 Subject: [PATCH] reportbug/utils.py: do not fail on decoding errors The dpkg status database and the reportbug configuration file might use character sets different from the current locale. This can be problematic when reading and decoding the file contents: The default decoding error handler ("strict") errors out in such cases. Picking a different one is more robust. --- reportbug/utils.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/reportbug/utils.py b/reportbug/utils.py index 5c06afb..2de91ae 100644 --- a/reportbug/utils.py +++ b/reportbug/utils.py @@ -496,7 +496,7 @@ class AvailDB(object): def get_dpkg_database(): try: -fp = open(STATUSDB) +fp = open(STATUSDB, errors="backslashreplace") if fp: return AvailDB(fp=fp) except IOError: @@ -976,7 +976,7 @@ def parse_config_files(): for filename in FILES: if os.path.exists(filename): try: -lex = our_lex(open(filename), posix=True) +lex = our_lex(open(filename, errors="backslashreplace"), posix=True) except IOError as msg: continue @@ -1235,7 +1235,7 @@ def exec_and_parse_bugscript(handler, bugscript): isattachments = False headers = pseudoheaders = text = '' attachments = [] -fp = open(filename) +fp = open(filename, errors="backslashreplace") for line in fp.readlines(): # we identify the blocks for headers and pseudo-h if line == '-- BEGIN HEADERS --\n': -- 2.1.4 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#850317: reportbug attachment bug fix
File "/usr/lib/python3/dist-packages/reportbug/submit.py", line 177, in mime_attach email.Encoders.encode_base64(part) AttributeError: module 'email' has no attribute 'Encoders' Looks like a typo with a simple fix: s/Encoders/encoders/ in line 177 of reportbug/submit.py ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#851361: reportbug: please show user from address in human readable format, not rfc 2047 encoded
Package: reportbug Version: 7.1.2 Severity: minor Tags: patch Dear Maintainer, Another finding from playing with non-ascii characters in the configuration: It would be nice to show the from address in a more readable format than rfc 2047 encoding. Proposed patch attached. Before: > Using '=?utf-8?q?Hans_M=C3=BCller?= ' as your from address. After: > Using 'Hans Müller ' as your from address. >From e1e88e8d5d865dca3d7ad7fb7507423a398248b6 Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Sat, 14 Jan 2017 11:52:52 +0100 Subject: [PATCH 4/4] Output user from address in human-readable format I.e., not in rfc 2047 encoded format --- bin/reportbug | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/bin/reportbug b/bin/reportbug index 78ee665..e2247c6 100755 --- a/bin/reportbug +++ b/bin/reportbug @@ -1323,7 +1323,8 @@ class UI(object): "Please change your locale if this is incorrect.\n\n", charset) fromaddr = utils.get_user_id(self.options.email, self.options.realname, charset) -ewrite("Using '%s' as your from address.\n", fromaddr) +ewrite("Using '%s' as your from address.\n", +str(email.header.make_header(email.header.decode_header(fromaddr if self.options.debugmode: sendto = fromaddr -- 2.1.4 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#851355: reportbug manpage: wrong order of email environment variables
Package: reportbug Version: 7.1.2 Severity: minor Tags: patch Dear Maintainer, Even in reportbug 7.1.2, the manpage still says: > REPORTBUGEMAIL, EMAIL, DEBEMAIL > Email address to use as your from address (in this order). If > no environment variable exists, the default is taken from your > user name and /etc/mailname. The order of the environment variable does not match the code, which means that the fix for bug #472296 was not complete/correct. Patch attached. >From d4b4fe45ba0a424b7c2ae7bdb2ab8b0b3df994c7 Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Sat, 14 Jan 2017 10:50:14 +0100 Subject: [PATCH] manpage: fix order of email environment variables --- man/reportbug.1 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/man/reportbug.1 b/man/reportbug.1 index df33de5..f5f0ede 100644 --- a/man/reportbug.1 +++ b/man/reportbug.1 @@ -519,7 +519,7 @@ Editor to use for editing your bug report. .B EDITOR Editor to use for editing the bug report (overridden by \fBVISUAL\fP). .TP -.B REPORTBUGEMAIL, EMAIL, DEBEMAIL +.B REPORTBUGEMAIL, DEBEMAIL, EMAIL Email address to use as your from address (in this order). If no environment variable exists, the default is taken from your user name and \fB/etc/mailname\fP. -- 2.1.4 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#851322: reportbug: bug script fails with non-ascii characters in reportbugrc
Package: reportbug Version: 7.1.2 Severity: normal Dear Maintainer, Configure a realname with non-ascii characters, then try to report a bug on reportbug itself with reportbug 7.1.2. If you have no locale set, reportbug fails immediately. (But I'm not sure this case should be supported. You really should have appropriate locale settings if have non-ascii characters in your name...) With proper locale settings, only the bug script still fails: Traceback (most recent call last): File "/usr/share/bug/reportbug/script", line 25, in for line in open(USERFILE, 'r'): File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode return codecs.ascii_decode(input, self.errors)[0] UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 369: ordinal not in range(128) This is because the bug script is executed with LC_ALL=C, so it can only deal with ascii characters by default. There are several options to fix this, among them: (a) execute the bug script in the user's locale -- this will affect bug scripts of all packages, so it may break other bug scripts and lead to localized output of others, (b) modify the reportbug bug script to cope with non-ascii characters, for example like this: @@ -4,7 +4,7 @@ import os import re -OUT=os.fdopen(3, 'w') +OUT=os.fdopen(3, 'w', encoding='utf-8', errors='backslashreplace') envprint = False for var in ['EDITOR', 'PAGER', 'VISUAL', 'REPORTBUGEMAIL', 'DEBEMAIL', 'EMAIL', @@ -22,7 +22,7 @@ if os.path.exists(USERFILE): if envprint: print(file=OUT) print('** %s:' % USERFILE, file=OUT) -for line in open(USERFILE, 'r'): +for line in open(USERFILE, 'r', encoding='utf-8', errors='backslashreplace'): line = line.strip() if line and not line.startswith('#'): m = passwdre.match(line) ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#849749: reportbug: unable to run its own bug script without python 2
The previous patch is still not sufficient if there are non-ascii characters in the name, for example. Explicitly choosing utf-8 character encoding makes it work. >From 95f5fcb92086110e23a3611beae550397ad49801 Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Fri, 30 Dec 2016 14:11:24 +0100 Subject: [PATCH 2/4] Port bug script to python3 --- share/script | 16 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/share/script b/share/script index e64c5e9..0b81135 100755 --- a/share/script +++ b/share/script @@ -1,31 +1,31 @@ -#! /usr/bin/python +#! /usr/bin/python3 # -*- python -*- import os import re -OUT=os.fdopen(3, 'w') +OUT=os.fdopen(3, 'w', encoding='utf-8') envprint = False for var in ['EDITOR', 'PAGER', 'VISUAL', 'REPORTBUGEMAIL', 'DEBEMAIL', 'EMAIL', 'DEBFULLNAME', 'DEBNAME', 'NAME', 'INTERFACE']: if var in os.environ: if not envprint: -print >> OUT, '** Environment settings:' +print('** Environment settings:', file=OUT) envprint = True -print >> OUT, '%s="%s"' % (var, os.environ[var]) +print('%s="%s"' % (var, os.environ[var]), file=OUT) passwdre = re.compile(r'\s*(smtppasswd)\s+(.*)$') USERFILE = os.path.expanduser('~/.reportbugrc') if os.path.exists(USERFILE): if envprint: -print >> OUT -print >> OUT, '** %s:' % USERFILE -for line in file(USERFILE): +print(file=OUT) +print('** %s:' % USERFILE, file=OUT) +for line in open(USERFILE, 'r', encoding='utf-8'): line = line.strip() if line and not line.startswith('#'): m = passwdre.match(line) if m: line = '%s ' % m.group(1) -print >> OUT, line +print(line, file=OUT) -- 2.1.4 >From 0d8d4e36f81ed97168da45b254b15e502ac3891e Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Sat, 31 Dec 2016 11:36:00 +0100 Subject: [PATCH 3/4] Update bug control file to binary package change --- share/control | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/share/control b/share/control index 49ceaa0..4e3e8d1 100644 --- a/share/control +++ b/share/control @@ -1 +1 @@ -Report-with: python-reportbug \ No newline at end of file +Report-with: python3-reportbug -- 2.1.4 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#849749: reportbug: unable to run its own bug script without python 2
The previous patch turned out to be incomplete. Here is a new version, now tested. The bug control file should probably also be updated. Attaching another patch. >From 098e23286bfa44566e4877dd9532ce5e436a799f Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Fri, 30 Dec 2016 14:11:24 +0100 Subject: [PATCH 2/4] Port bug script to python3 --- share/script | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/share/script b/share/script index e64c5e9..ddc144e 100755 --- a/share/script +++ b/share/script @@ -1,4 +1,4 @@ -#! /usr/bin/python +#! /usr/bin/python3 # -*- python -*- import os @@ -11,21 +11,21 @@ for var in ['EDITOR', 'PAGER', 'VISUAL', 'REPORTBUGEMAIL', 'DEBEMAIL', 'EMAIL', 'DEBFULLNAME', 'DEBNAME', 'NAME', 'INTERFACE']: if var in os.environ: if not envprint: -print >> OUT, '** Environment settings:' +print('** Environment settings:', file=OUT) envprint = True -print >> OUT, '%s="%s"' % (var, os.environ[var]) +print('%s="%s"' % (var, os.environ[var]), file=OUT) passwdre = re.compile(r'\s*(smtppasswd)\s+(.*)$') USERFILE = os.path.expanduser('~/.reportbugrc') if os.path.exists(USERFILE): if envprint: -print >> OUT -print >> OUT, '** %s:' % USERFILE -for line in file(USERFILE): +print(file=OUT) +print('** %s:' % USERFILE, file=OUT) +for line in open(USERFILE, 'r'): line = line.strip() if line and not line.startswith('#'): m = passwdre.match(line) if m: line = '%s ' % m.group(1) -print >> OUT, line +print(line, file=OUT) -- 2.1.4 >From a3de0701e7a847847a76d6c1f915a1b7cb030e08 Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Sat, 31 Dec 2016 11:36:00 +0100 Subject: [PATCH 3/4] Update bug control file to binary package change --- share/control | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/share/control b/share/control index 49ceaa0..4e3e8d1 100644 --- a/share/control +++ b/share/control @@ -1 +1 @@ -Report-with: python-reportbug \ No newline at end of file +Report-with: python3-reportbug -- 2.1.4 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#849749: reportbug: unable to run its own bug script without python 2
Package: reportbug Version: 7.1.1 Severity: normal Tags: patch Dear Maintainer, reportbug either still needs to depend on python, or its bug script must be ported to python3. Otherwise it will fail if python2 is not installed. Untested patch attached (done using 2to3). >From 5ba5586196275bec188fe4f473a1eba207506cb5 Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Fri, 30 Dec 2016 14:11:24 +0100 Subject: [PATCH] Port bug script to python3 --- share/script | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/share/script b/share/script index e64c5e9..58eef25 100755 --- a/share/script +++ b/share/script @@ -1,4 +1,4 @@ -#! /usr/bin/python +#! /usr/bin/python3 # -*- python -*- import os @@ -11,21 +11,21 @@ for var in ['EDITOR', 'PAGER', 'VISUAL', 'REPORTBUGEMAIL', 'DEBEMAIL', 'EMAIL', 'DEBFULLNAME', 'DEBNAME', 'NAME', 'INTERFACE']: if var in os.environ: if not envprint: -print >> OUT, '** Environment settings:' +print('** Environment settings:', file=OUT) envprint = True -print >> OUT, '%s="%s"' % (var, os.environ[var]) +print('%s="%s"' % (var, os.environ[var]), file=OUT) passwdre = re.compile(r'\s*(smtppasswd)\s+(.*)$') USERFILE = os.path.expanduser('~/.reportbugrc') if os.path.exists(USERFILE): if envprint: -print >> OUT -print >> OUT, '** %s:' % USERFILE +print(file=OUT) +print('** %s:' % USERFILE, file=OUT) for line in file(USERFILE): line = line.strip() if line and not line.startswith('#'): m = passwdre.match(line) if m: line = '%s ' % m.group(1) -print >> OUT, line +print(line, file=OUT) -- 2.1.4 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#849677: Bug#849677: reportbug: ampersand-replaced usernames should be capitalized instead of uppercased
Hey Sandro, On 29-12-2016 19:53, Sandro Tosi wrote: > Thanks for your patch, but in the meantime reportbug has migrated to > python3 (as for v7.0.0, latest v7.1.1) - could you verify if your fix > is still required in the new version and if so port the patch to it? > git repo url for reference: > https://anonscm.debian.org/gitweb/?p=reportbug/reportbug.git The patch applies to current master. Sorry, forgot to update the version field in the bug report - I'm running stable, but noticed the bug when reading the latest source in cgit. Nis ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint
[Reportbug-maint] Bug#849677: reportbug: ampersand-replaced usernames should be capitalized instead of uppercased
Package: reportbug Version: 6.6.3 Severity: minor Tags: patch Dear Maintainer, The fix for https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=224231 was not fully accurate. "finger" does capitalize, not uppercase the username replacing the ampersand. Patch attached. >From a385efe73ab398b9acd51319dd5249385843326d Mon Sep 17 00:00:00 2001 From: Nis Martensen Date: Thu, 29 Dec 2016 19:25:44 +0100 Subject: [PATCH] Ampersand replacement should capitalize, but not uppercase to match behavior of "finger" --- reportbug/utils.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/reportbug/utils.py b/reportbug/utils.py index 89e0991..4b00a86 100644 --- a/reportbug/utils.py +++ b/reportbug/utils.py @@ -305,7 +305,7 @@ def get_user_id(email='', realname='', charset='utf-8'): if not realname: realname = info[4].split(',', 1)[0] # Convert & in gecos field 4 to capitalized logname: #224231 -realname = realname.replace('&', info[0].upper()) +realname = realname.replace('&', info[0].capitalize()) if not realname: return email -- 2.1.4 ___ Reportbug-maint mailing list Reportbug-maint@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint