Re: [PATCH] Use debianbts instead of SOAPpy

[Raphael Hertzog]

On Tue, 07 Nov 2017, Ville Skyttä wrote:
> Attached is a small patch on top of the previous one: 'fixed' is not a
> documented value for a pending bug, 'done' is. Just happened to
> notice, but did not see any effect in cases I tested with.

Thanks, I merged your two patches as I got python-debianbts installed
on the production server.

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: https://www.freexian.com/services/debian-lts.html
Learn to master Debian: https://debian-handbook.info/get/

Re: [PATCH] Use debianbts instead of SOAPpy

[Ville Skyttä]

On Wed, Nov 1, 2017 at 10:51 PM, Ville Skyttä  wrote:
> On Wed, Nov 1, 2017 at 12:52 PM, Raphael Hertzog  wrote:
>> On Tue, 31 Oct 2017, Ville Skyttä wrote:
>>> This switches to using debianbts instead of SOAPpy. To be applied over
>>> the LDAP->nm.d.o REST patch in https://bugs.debian.org/797223#10
>>
>> Thanks for this. Right now tracker.debian.org still runs jessie. Do
>> you know if your code works with python-debianbts 1.12 from jessie or if
>> I need to request the installation from jessie-backports ? (which has
>> 2.6.1~bpo8+1)
>
> I don't know; while doing the port I skimmed through old
> python-debianbts tags at GitHub, and IIRC all needed functions have
> always been there, which is why I didn't add any version to the
> dependency. All the testing I've done has been with the 2.6.1 shipping
> with Ubuntu 17.10.

Tested locally using jessie docker image + django 1.8 from backports
(so this is with debianbts 1.12): success, works.

Attached is a small patch on top of the previous one: 'fixed' is not a
documented value for a pending bug, 'done' is. Just happened to
notice, but did not see any effect in cases I tested with.
From baaa91765aab70e6bfbd6bf1bc125dc67816829e Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ville=20Skytt=C3=A4?= 
Date: Tue, 7 Nov 2017 22:49:34 +0200
Subject: [PATCH 2/2] BTS: Check for pending == 'done' instead of 'fixed'

https://wiki.debian.org/DebbugsSoapInterface#line-55
---
 distro_tracker/vendor/debian/tracker_tasks.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/distro_tracker/vendor/debian/tracker_tasks.py b/distro_tracker/vendor/debian/tracker_tasks.py
index 3478c06..3dc8d3b 100644
--- a/distro_tracker/vendor/debian/tracker_tasks.py
+++ b/distro_tracker/vendor/debian/tracker_tasks.py
@@ -247,7 +247,7 @@ class UpdatePackageBugStats(BaseTask):
 bug_stats = {}
 bugs = debianbts.get_status(*bug_numbers)
 for bug in bugs:
-if bug.done or bug.fixed_versions or bug.pending == 'fixed':
+if bug.done or bug.fixed_versions or bug.pending == 'done':
 continue
 
 bug_stats.setdefault(bug.package, 0)
-- 
2.14.1

Re: [PATCH] Use debianbts instead of SOAPpy

[Raphael Hertzog]

On Wed, 01 Nov 2017, Mattia Rizzolo wrote:
> On Wed, Nov 01, 2017 at 11:52:16AM +0100, Raphael Hertzog wrote:
> > Thanks for this. Right now tracker.debian.org still runs jessie.
> 
> DSA is actively looking to upgrade as many hosts as possible, and I know
> a DSA member asked around about who to contact about upgrading ticharich
> ("is it buxy only or somebody else as well?" kind of questions).  If you
> believe tracker.d.o would do great with the host on stretch, you should
> definitely send DSA a word about it.

We should certainly upgrade to stretch but I would like to switch to
Django 1.11 and Python 3 at the same time. Thus I'm not fully ready yet.

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: https://www.freexian.com/services/debian-lts.html
Learn to master Debian: https://debian-handbook.info/get/

Re: [PATCH] Use debianbts instead of SOAPpy

[Mattia Rizzolo]

On Wed, Nov 01, 2017 at 11:52:16AM +0100, Raphael Hertzog wrote:
> Thanks for this. Right now tracker.debian.org still runs jessie.

DSA is actively looking to upgrade as many hosts as possible, and I know
a DSA member asked around about who to contact about upgrading ticharich
("is it buxy only or somebody else as well?" kind of questions).  If you
believe tracker.d.o would do great with the host on stretch, you should
definitely send DSA a word about it.

-- 
regards,
Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540  .''`.
more about me:  https://mapreri.org : :'  :
Launchpad user: https://launchpad.net/~mapreri  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-

Re: [PATCH] Use debianbts instead of SOAPpy

[Ville Skyttä]

On Wed, Nov 1, 2017 at 12:52 PM, Raphael Hertzog  wrote:
> On Tue, 31 Oct 2017, Ville Skyttä wrote:
>> This switches to using debianbts instead of SOAPpy. To be applied over
>> the LDAP->nm.d.o REST patch in https://bugs.debian.org/797223#10
>
> Thanks for this. Right now tracker.debian.org still runs jessie. Do
> you know if your code works with python-debianbts 1.12 from jessie or if
> I need to request the installation from jessie-backports ? (which has
> 2.6.1~bpo8+1)

I don't know; while doing the port I skimmed through old
python-debianbts tags at GitHub, and IIRC all needed functions have
always been there, which is why I didn't add any version to the
dependency. All the testing I've done has been with the 2.6.1 shipping
with Ubuntu 17.10.

Attached is a revised version of the patch: adds tox.ini
modifications, no other changes.
From 91a7d107f3a0f3949a1f4b8c03db6dad193db2e4 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ville=20Skytt=C3=A4?= 
Date: Tue, 31 Oct 2017 08:44:42 +0200
Subject: [PATCH] Use debianbts instead of SOAPpy

---
 TODO  |  3 ---
 debian/control|  4 ++--
 distro_tracker/vendor/debian/tracker_tasks.py | 30 ++-
 docs/setup/setup.rst  |  4 ++--
 tox.ini   |  2 +-
 5 files changed, 16 insertions(+), 27 deletions(-)

diff --git a/TODO b/TODO
index 87d41fd..d5d2851 100644
--- a/TODO
+++ b/TODO
@@ -16,9 +16,6 @@ Misc possible enhancements:
 * Add a subscriber count on each package page (and put
   subscription-related actions in the same panel).
 * Paul Wise's input: https://lists.debian.org/debian-qa/2014/07/msg00092.html
-* Codebase is Python 3 compatible but some optional dependencies need to
-  be ported:
-  - python-soappy => switch to pysimplesoap
 * Re-enable a smarter clickjacking protection: only activate it when the
   page is rendered for a logged in user and allow callers to pass a
   "anonymous=1" parameter that will force render the web page as if
diff --git a/debian/control b/debian/control
index a5189e6..3a7373c 100644
--- a/debian/control
+++ b/debian/control
@@ -12,13 +12,13 @@ Build-Depends: debhelper (>= 9),
 python-apt,
 python-bs4,
 python-debian,
+python-debianbts,
 python-django (>= 1.8),
 python-django-jsonfield,
 python-django-captcha,
 python-gpgme,
 python-lzma,
 python-requests (>= 2),
-python-soappy,
 python-yaml,
 python-pyinotify,
 Standards-Version: 3.9.5
@@ -59,12 +59,12 @@ Depends: ${python:Depends},
 python-apt,
 python-beautifulsoup,
 python-debian,
+python-debianbts,
 python-django (>= 1.8),
 python-django-jsonfield,
 python-gpgme,
 python-lzma,
 python-requests (>= 2),
-python-soappy,
 python-yaml,
 python-pyinotify,
 ${misc:Depends}
diff --git a/distro_tracker/vendor/debian/tracker_tasks.py b/distro_tracker/vendor/debian/tracker_tasks.py
index 0a7b185..bf4d4b8 100644
--- a/distro_tracker/vendor/debian/tracker_tasks.py
+++ b/distro_tracker/vendor/debian/tracker_tasks.py
@@ -56,7 +56,7 @@ from bs4 import BeautifulSoup as soup
 import yaml
 
 try:
-import SOAPpy
+import debianbts
 except ImportError:
 pass
 
@@ -222,7 +222,7 @@ class UpdatePackageBugStats(BaseTask):
 
 def _get_tagged_bug_stats(self, tag, user=None):
 """
-Using the BTS SOAP interface, retrieves the statistics of bugs with a
+Using the BTS interface, retrieves the statistics of bugs with a
 particular tag.
 
 :param tag: The tag for which the statistics are required.
@@ -237,29 +237,21 @@ class UpdatePackageBugStats(BaseTask):
 debian_ca_bundle = '/etc/ssl/ca-debian/ca-certificates.crt'
 if os.path.exists(debian_ca_bundle):
 os.environ['SSL_CERT_FILE'] = debian_ca_bundle
-url = 'https://bugs.debian.org/cgi-bin/soap.cgi'
-namespace = 'Debbugs/SOAP'
-server = SOAPpy.SOAPProxy(url, namespace)
 if user:
-bugs = server.get_usertag(user, tag)
-bugs = bugs[0]
+bug_numbers = debianbts.get_usertag(user, tag).values()
 else:
-bugs = server.get_bugs('tag', tag)
+bug_numbers = debianbts.get_bugs('tag', tag)
 
 # Match each retrieved bug ID to a package and then find the aggregate
 # count for each package.
 bug_stats = {}
-statuses = server.get_status(bugs)
-statuses = statuses[0]
-for status in statuses:
-status = status['value']
-if status['done'] or status['fixed'] or \
-status['pending'] == 'fixed':
+bugs = debianbts.get_status(*bug_numbers)
+for bug in bugs:
+if bug.done or bug.fixed_versions or bug.pending == 'fixed':
 continue
 
-package_name = status['package']
-   

Re: [PATCH] Use debianbts instead of SOAPpy

[Raphael Hertzog]

On Tue, 31 Oct 2017, Ville Skyttä wrote:
> This switches to using debianbts instead of SOAPpy. To be applied over
> the LDAP->nm.d.o REST patch in https://bugs.debian.org/797223#10

Thanks for this. Right now tracker.debian.org still runs jessie. Do
you know if your code works with python-debianbts 1.12 from jessie or if
I need to request the installation from jessie-backports ? (which has
2.6.1~bpo8+1)

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: https://www.freexian.com/services/debian-lts.html
Learn to master Debian: https://debian-handbook.info/get/