On Wed, Nov 1, 2017 at 12:52 PM, Raphael Hertzog <hert...@debian.org> wrote: > On Tue, 31 Oct 2017, Ville Skyttä wrote: >> This switches to using debianbts instead of SOAPpy. To be applied over >> the LDAP->nm.d.o REST patch in https://bugs.debian.org/797223#10 > > Thanks for this. Right now tracker.debian.org still runs jessie. Do > you know if your code works with python-debianbts 1.12 from jessie or if > I need to request the installation from jessie-backports ? (which has > 2.6.1~bpo8+1)
I don't know; while doing the port I skimmed through old python-debianbts tags at GitHub, and IIRC all needed functions have always been there, which is why I didn't add any version to the dependency. All the testing I've done has been with the 2.6.1 shipping with Ubuntu 17.10. Attached is a revised version of the patch: adds tox.ini modifications, no other changes.
From 91a7d107f3a0f3949a1f4b8c03db6dad193db2e4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ville=20Skytt=C3=A4?= <ville.sky...@iki.fi> Date: Tue, 31 Oct 2017 08:44:42 +0200 Subject: [PATCH] Use debianbts instead of SOAPpy --- TODO | 3 --- debian/control | 4 ++-- distro_tracker/vendor/debian/tracker_tasks.py | 30 ++++++++++----------------- docs/setup/setup.rst | 4 ++-- tox.ini | 2 +- 5 files changed, 16 insertions(+), 27 deletions(-) diff --git a/TODO b/TODO index 87d41fd..d5d2851 100644 --- a/TODO +++ b/TODO @@ -16,9 +16,6 @@ Misc possible enhancements: * Add a subscriber count on each package page (and put subscription-related actions in the same panel). * Paul Wise's input: https://lists.debian.org/debian-qa/2014/07/msg00092.html -* Codebase is Python 3 compatible but some optional dependencies need to - be ported: - - python-soappy => switch to pysimplesoap * Re-enable a smarter clickjacking protection: only activate it when the page is rendered for a logged in user and allow callers to pass a "anonymous=1" parameter that will force render the web page as if diff --git a/debian/control b/debian/control index a5189e6..3a7373c 100644 --- a/debian/control +++ b/debian/control @@ -12,13 +12,13 @@ Build-Depends: debhelper (>= 9), python-apt, python-bs4, python-debian, + python-debianbts, python-django (>= 1.8), python-django-jsonfield, python-django-captcha, python-gpgme, python-lzma, python-requests (>= 2), - python-soappy, python-yaml, python-pyinotify, Standards-Version: 3.9.5 @@ -59,12 +59,12 @@ Depends: ${python:Depends}, python-apt, python-beautifulsoup, python-debian, + python-debianbts, python-django (>= 1.8), python-django-jsonfield, python-gpgme, python-lzma, python-requests (>= 2), - python-soappy, python-yaml, python-pyinotify, ${misc:Depends} diff --git a/distro_tracker/vendor/debian/tracker_tasks.py b/distro_tracker/vendor/debian/tracker_tasks.py index 0a7b185..bf4d4b8 100644 --- a/distro_tracker/vendor/debian/tracker_tasks.py +++ b/distro_tracker/vendor/debian/tracker_tasks.py @@ -56,7 +56,7 @@ from bs4 import BeautifulSoup as soup import yaml try: - import SOAPpy + import debianbts except ImportError: pass @@ -222,7 +222,7 @@ class UpdatePackageBugStats(BaseTask): def _get_tagged_bug_stats(self, tag, user=None): """ - Using the BTS SOAP interface, retrieves the statistics of bugs with a + Using the BTS interface, retrieves the statistics of bugs with a particular tag. :param tag: The tag for which the statistics are required. @@ -237,29 +237,21 @@ class UpdatePackageBugStats(BaseTask): debian_ca_bundle = '/etc/ssl/ca-debian/ca-certificates.crt' if os.path.exists(debian_ca_bundle): os.environ['SSL_CERT_FILE'] = debian_ca_bundle - url = 'https://bugs.debian.org/cgi-bin/soap.cgi' - namespace = 'Debbugs/SOAP' - server = SOAPpy.SOAPProxy(url, namespace) if user: - bugs = server.get_usertag(user, tag) - bugs = bugs[0] + bug_numbers = debianbts.get_usertag(user, tag).values() else: - bugs = server.get_bugs('tag', tag) + bug_numbers = debianbts.get_bugs('tag', tag) # Match each retrieved bug ID to a package and then find the aggregate # count for each package. bug_stats = {} - statuses = server.get_status(bugs) - statuses = statuses[0] - for status in statuses: - status = status['value'] - if status['done'] or status['fixed'] or \ - status['pending'] == 'fixed': + bugs = debianbts.get_status(*bug_numbers) + for bug in bugs: + if bug.done or bug.fixed_versions or bug.pending == 'fixed': continue - package_name = status['package'] - bug_stats.setdefault(package_name, 0) - bug_stats[package_name] += 1 + bug_stats.setdefault(bug.package, 0) + bug_stats[bug.package] += 1 return bug_stats @@ -465,14 +457,14 @@ class UpdatePackageBugStats(BaseTask): if not bug_stats: bug_stats = {} - # Add in help bugs from the BTS SOAP interface + # Add in help bugs from the BTS interface try: help_bugs = self._get_tagged_bug_stats('help') self._extend_bug_stats(bug_stats, help_bugs, 'help') except: logger.exception("Could not get bugs tagged help") - # Add in newcomer bugs from the BTS SOAP interface + # Add in newcomer bugs from the BTS interface try: newcomer_bugs = self._get_tagged_bug_stats('newcomer') self._extend_bug_stats(bug_stats, newcomer_bugs, 'newcomer') diff --git a/docs/setup/setup.rst b/docs/setup/setup.rst index cd95ba6..d4c62f8 100644 --- a/docs/setup/setup.rst +++ b/docs/setup/setup.rst @@ -16,12 +16,12 @@ Distro Tracker currently depends on the following Debian packages: - python-django-debug-toolbar (in development mode only) - python-django-captcha (optional) - python-debian +- python-debianbts (optional) - python-apt - python-gpgme - python-yaml - python-bs4 - python-pyinotify -- python-soappy (optional) - python-tox (for development only) - python-selenium (for development only) - chromedriver (for development only) @@ -34,7 +34,7 @@ For Python2.7, the following additional packages are required: Here is the list of required packages for development on Debian Jessie:: - $ sudo apt install python-django python-requests python-django-jsonfield python-django-debug-toolbar python-debian python-apt python-gpgme python-yaml python-bs4 python-soappy python-pyinotify python-tox python-mock python-lzma python-selenium python3-django python3-requests python3-django-jsonfield python3-django-debug-toolbar python3-debian python3-apt python3-gpgme python3-yaml python3-bs4 python3-pyinotify python3-selenium chromium chromedriver + $ sudo apt install python-django python-requests python-django-jsonfield python-django-debug-toolbar python-debian python-debianbts python-apt python-gpgme python-yaml python-bs4 python-pyinotify python-tox python-mock python-lzma python-selenium python3-django python3-requests python3-django-jsonfield python3-django-debug-toolbar python3-debian python3-debianbts python3-apt python3-gpgme python3-yaml python3-bs4 python3-pyinotify python3-selenium chromium chromedriver .. _database_setup: diff --git a/tox.ini b/tox.ini index 66e1671..791cbc0 100644 --- a/tox.ini +++ b/tox.ini @@ -37,13 +37,13 @@ deps = coverage: coverage py27-tests: mock py27-tests: pyliblzma - py27-tests: SOAPpy tests: requests tests: django_jsonfield tests: django_debug_toolbar tests: pygpgme tests: PyYAML tests: python_debian + tests: python_debianbts tests: python_apt tests: beautifulsoup4 functional-tests: selenium -- 2.14.1