Hello community, here is the log from the commit of package openSUSE-release-tools for openSUSE:Factory checked in at 2018-11-19 23:34:16 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/openSUSE-release-tools (Old) and /work/SRC/openSUSE:Factory/.openSUSE-release-tools.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "openSUSE-release-tools" Mon Nov 19 23:34:16 2018 rev:147 rq:650220 version:20181119.1714c58 Changes: -------- --- /work/SRC/openSUSE:Factory/openSUSE-release-tools/openSUSE-release-tools.changes 2018-11-14 14:44:16.014620692 +0100 +++ /work/SRC/openSUSE:Factory/.openSUSE-release-tools.new/openSUSE-release-tools.changes 2018-11-19 23:34:27.435019875 +0100 @@ -1,0 +2,60 @@ +Mon Nov 19 10:38:58 UTC 2018 - opensuse-releaset...@opensuse.org + +- Update to version 20181119.1714c58: + * staging-report: ignore projects not in a final state. + * osclib/stagingapi: provide project_status_final(). + +------------------------------------------------------------------- +Mon Nov 19 06:51:09 UTC 2018 - opensuse-releaset...@opensuse.org + +- Update to version 20181119.efa6674: + * stagingapi: No longer build disable to sub packages + +------------------------------------------------------------------- +Fri Nov 16 20:36:51 UTC 2018 - opensuse-releaset...@opensuse.org + +- Update to version 20181116.065ab5e: + * userscript/README: include more details about usage and troubleshooting. + +------------------------------------------------------------------- +Fri Nov 16 17:51:23 UTC 2018 - opensuse-releaset...@opensuse.org + +- Update to version 20181116.416172c: + * Checked with aplanas: using stringio is preferred + +------------------------------------------------------------------- +Fri Nov 16 14:13:03 UTC 2018 - opensuse-releaset...@opensuse.org + +- Update to version 20181116.31e97b0: + * In some cases the return value of yaml.load(description_text) can be a string containing 'none'. + +------------------------------------------------------------------- +Fri Nov 16 10:15:56 UTC 2018 - opensuse-releaset...@opensuse.org + +- Update to version 20181116.e3f260f: + * Import urllib2 python agnostic + * Store data as binary in the cache + * Remove the reduce call the brute force way + * basestring is gone - in favor or str (which also works for python2) + * rabbit-openqa: fix imports for python3 + * Port more code to run under python3 + * Use Leap:15.1:Images for testing instead of openSUSE:Factory + * Add the base container to container_products of openSUSE:Factory + * Add release workflow for container products + * Add new ImageProduct class to allow images with different enabled architectures + * Only look at/toggle publishing of the the product repo in :ToTest + * Define products in ToTestBase + * Allow to specify repository and target in _release_package + +------------------------------------------------------------------- +Thu Nov 15 18:12:37 UTC 2018 - opensuse-releaset...@opensuse.org + +- Update to version 20181115.1c92d8a: + * travis: add test jobs against osc python3 branch for both python 2 and 3. + +------------------------------------------------------------------- +Wed Nov 14 21:56:10 UTC 2018 - Jimmy Berry <jbe...@suse.com> + +- Extract osrt-worker-obs.kiwi via _service and reference in spec. + +------------------------------------------------------------------- Old: ---- openSUSE-release-tools-20181113.f936dc2.obscpio New: ---- openSUSE-release-tools-20181119.1714c58.obscpio osrt-worker-obs.kiwi ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ openSUSE-release-tools.spec ++++++ --- /var/tmp/diff_new_pack.Z4tQWo/_old 2018-11-19 23:34:29.499017489 +0100 +++ /var/tmp/diff_new_pack.Z4tQWo/_new 2018-11-19 23:34:29.507017479 +0100 @@ -20,13 +20,14 @@ %define source_dir openSUSE-release-tools %define announcer_filename factory-package-news Name: openSUSE-release-tools -Version: 20181113.f936dc2 +Version: 20181119.1714c58 Release: 0 Summary: Tools to aid in staging and release work for openSUSE/SUSE License: GPL-2.0-or-later AND MIT Group: Development/Tools/Other Url: https://github.com/openSUSE/openSUSE-release-tools Source: %{name}-%{version}.tar.xz +Source99: osrt-worker-obs.kiwi BuildArch: noarch # Requires sr#512849 which provides osc_plugin_dir. BuildRequires: osc >= 0.159.0 ++++++ osrt-worker-obs.kiwi ++++++ <?xml version="1.0" encoding="utf-8"?> <image schemaversion="6.5" name="osrt-worker-obs"> <description type="system"> <author>openSUSE Release Team</author> <contact>opensuse-releaset...@opensuse.org</contact> <specification>openSUSE Release Tools (OSRT) worker for OBS container image</specification> </description> <preferences> <type image="docker" derived_from="obsrepositories:/opensuse/tumbleweed#current"> <!-- <containerconfig name="osrt/worker-obs" tag="%%TAG%%" additionaltags="latest"/> --> <containerconfig name="osrt/worker-obs" tag="latest"/> </type> <version>1.0.0</version> <packagemanager>zypper</packagemanager> <rpm-check-signatures>false</rpm-check-signatures> <rpm-excludedocs>true</rpm-excludedocs> </preferences> <repository> <source path="obsrepositories:/"/> </repository> <packages type="image"> <package name="openSUSE-release-tools-check-source"/> <package name="openSUSE-release-tools-leaper"/> <package name="openSUSE-release-tools-pkglistgen"/> <package name="openSUSE-release-tools-repo-checker"/> <package name="openSUSE-release-tools-staging-bot"/> </packages> </image> ++++++ _service ++++++ --- /var/tmp/diff_new_pack.Z4tQWo/_old 2018-11-19 23:34:29.559017419 +0100 +++ /var/tmp/diff_new_pack.Z4tQWo/_new 2018-11-19 23:34:29.559017419 +0100 @@ -5,6 +5,7 @@ <param name="scm">git</param> <param name="changesgenerate">enable</param> <param name="extract">dist/package/openSUSE-release-tools.spec</param> + <param name="extract">dist/kiwi/osrt-worker-obs.kiwi</param> <param name="filename">openSUSE-release-tools</param> </service> <service name="set_version" mode="disabled" /> ++++++ _servicedata ++++++ --- /var/tmp/diff_new_pack.Z4tQWo/_old 2018-11-19 23:34:29.579017397 +0100 +++ /var/tmp/diff_new_pack.Z4tQWo/_new 2018-11-19 23:34:29.579017397 +0100 @@ -1,6 +1,6 @@ <servicedata> <service name="tar_scm"> <param name="url">https://github.com/openSUSE/openSUSE-release-tools.git</param> - <param name="changesrevision">f936dc2f47c066dfc9e2cad029cb66c8dd5505ec</param> + <param name="changesrevision">1714c582f6543e92e2762d7d63da41d066df37ee</param> </service> </servicedata> ++++++ openSUSE-release-tools-20181113.f936dc2.obscpio -> openSUSE-release-tools-20181119.1714c58.obscpio ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/.travis.yml new/openSUSE-release-tools-20181119.1714c58/.travis.yml --- old/openSUSE-release-tools-20181113.f936dc2/.travis.yml 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/.travis.yml 2018-11-19 11:33:52.000000000 +0100 @@ -74,6 +74,64 @@ - nosetests --with-coverage --cover-package=. --cover-inclusive --exclude-dir=./oqamaint -c .noserc after_success: - coveralls + - env: TEST_SUITE=nosetests-osc-python3 + sudo: required + services: + - docker + language: python + python: 2.7 + before_install: + # provides xmllint used by test_bootstrap_copy (tests.freeze_tests.TestFreeze) + - sudo apt-get install libxml2-utils + install: + # urlgrabber needed to install osc from git in requirements.txt + # m2crypto for osc to be runable as used in docker-compose-obs + - pip install pycurl urlgrabber m2crypto pika + - sed -i 's|osc|osc@python3|' requirements.txt + - pip install -r requirements.txt + - pip install python-coveralls + - pip install nose-exclude + before_script: + # travis-ci/travis-ci#7008: stop services to make room for OBS setup + - sudo service mysql stop + - sudo service memcached stop + - ./dist/ci/docker-compose-obs + # Needs python prefix to use the correct interpretor. + - python ./obs_clone.py --cache --debug --apiurl-target local + script: + - nosetests --with-coverage --cover-package=. --cover-inclusive --exclude-dir=./oqamaint -c .noserc + after_success: + - coveralls + - env: TEST_SUITE=nosetests-osc-python3 + sudo: required + services: + - docker + language: python + python: 3.6 + before_install: + # provides xmllint used by test_bootstrap_copy (tests.freeze_tests.TestFreeze) + - sudo apt-get install libxml2-utils + install: + # m2crypto for osc to be runable as used in docker-compose-obs + - pip install pycurl m2crypto pika + - sed -i 's|osc|osc@python3|' requirements.txt + - sed -i 's|urlgrabber||' requirements.txt + - pip install -r requirements.txt + - pip install python-coveralls + - pip install nose-exclude + before_script: + # travis-ci/travis-ci#7008: stop services to make room for OBS setup + - sudo service mysql stop + - sudo service memcached stop + - ./dist/ci/docker-compose-obs + # Needs python prefix to use the correct interpretor. + - python ./obs_clone.py --cache --debug --apiurl-target local + script: + - nosetests --with-coverage --cover-package=. --cover-inclusive --exclude-dir=./oqamaint -c .noserc + after_success: + - coveralls + allow_failures: + - env: TEST_SUITE=nosetests-osc-python3 deploy: provider: script diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/ReviewBot.py new/openSUSE-release-tools-20181119.1714c58/ReviewBot.py --- old/openSUSE-release-tools-20181113.f936dc2/ReviewBot.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/ReviewBot.py 2018-11-19 11:33:52.000000000 +0100 @@ -28,7 +28,12 @@ from osc import conf import osc.core -import urllib2 +try: + from urllib.error import HTTPError, URLError +except ImportError: + # python 2.x + from urllib2 import HTTPError, URLError + from itertools import count class PackageLookup(object): @@ -57,7 +62,7 @@ try: return osc.core.http_GET(osc.core.makeurl(self.apiurl, ['source', prj, '00Meta', 'lookup.yml'])) - except urllib2.HTTPError as e: + except HTTPError as e: # in case the project doesn't exist yet (like sle update) if e.code != 404: raise e @@ -458,7 +463,7 @@ url = osc.core.makeurl(apiurl, ('source', project, package), query=query) try: return ET.parse(osc.core.http_GET(url)).getroot() - except (urllib2.HTTPError, urllib2.URLError): + except (HTTPError, URLError): return None def get_originproject(self, project, package, rev=None): @@ -493,7 +498,7 @@ url = osc.core.makeurl(self.apiurl, ('source', src_project, src_package), query=query) try: root = ET.parse(osc.core.http_GET(url)).getroot() - except urllib2.HTTPError: + except HTTPError: return (None, None) if root is not None: @@ -521,7 +526,7 @@ return True if self.review_group and self._has_open_review_by(root, 'by_group', self.review_group): return True - except urllib2.HTTPError as e: + except HTTPError as e: print('ERROR in URL %s [%s]' % (url, e)) return False @@ -670,7 +675,7 @@ self.logger.debug("checking %s in %s"%(package, project)) try: si = osc.core.show_package_meta(self.apiurl, project, package) - except (urllib2.HTTPError, urllib2.URLError): + except (HTTPError, URLError): si = None if si is None: self.logger.debug("new package") @@ -686,7 +691,7 @@ u = osc.core.makeurl(self.apiurl, [ 'source', project, package, '_history' ], { 'limit': history_limit }) try: r = osc.core.http_GET(u) - except urllib2.HTTPError as e: + except HTTPError as e: self.logger.debug("package has no history!?") return None diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/ToolBase.py new/openSUSE-release-tools-20181119.1714c58/ToolBase.py --- old/openSUSE-release-tools-20181113.f936dc2/ToolBase.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/ToolBase.py 2018-11-19 11:33:52.000000000 +0100 @@ -8,7 +8,12 @@ import signal import sys import time -import urllib2 + +try: + from urllib.error import HTTPError +except ImportError: + # python 2.x + from urllib2 import HTTPError import osc.conf import osc.core @@ -48,7 +53,7 @@ def retried_GET(self, url): try: return http_GET(url) - except urllib2.HTTPError as e: + except HTTPError as e: if 500 <= e.code <= 599: print 'Retrying {}'.format(url) time.sleep(1) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/abichecker/abichecker.py new/openSUSE-release-tools-20181119.1714c58/abichecker/abichecker.py --- old/openSUSE-release-tools-20181113.f936dc2/abichecker/abichecker.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/abichecker/abichecker.py 2018-11-19 11:33:52.000000000 +0100 @@ -24,7 +24,12 @@ import osc.core from osc.util.cpio import CpioRead -import urllib2 +try: + from urllib.error import HTTPError +except ImportError: + # python 2.x + from urllib2 import HTTPError + import rpm from collections import namedtuple from osclib.pkgcache import PkgCache @@ -783,7 +788,7 @@ [ 'view=cpioheaders' ]) try: r = osc.core.http_GET(u) - except urllib2.HTTPError as e: + except HTTPError as e: raise FetchError('failed to fetch header information: %s'%e) tmpfile = NamedTemporaryFile(prefix="cpio-", delete=False) for chunk in r: @@ -813,7 +818,7 @@ url = osc.core.makeurl(self.apiurl, ('build', prj, repo, arch, pkg)) try: root = ET.parse(osc.core.http_GET(url)).getroot() - except urllib2.HTTPError: + except HTTPError: return None return dict([(node.attrib['filename'], node.attrib['mtime']) for node in root.findall('binary')]) @@ -828,7 +833,7 @@ 'srcmd5' : rev } url = osc.core.makeurl(self.apiurl, ('build', src_project, '_result'), query) return ET.parse(osc.core.http_GET(url)).getroot() - except urllib2.HTTPError as e: + except HTTPError as e: if e.code != 404: self.logger.error('ERROR in URL %s [%s]' % (url, e)) raise @@ -855,7 +860,7 @@ url = osc.core.makeurl(self.apiurl, ('source', project, '_meta')) try: root = ET.parse(osc.core.http_GET(url)).getroot() - except urllib2.HTTPError: + except HTTPError: return None repos = set() @@ -912,7 +917,7 @@ url = osc.core.makeurl(self.apiurl, ('source', src_project, '_meta')) try: root = ET.parse(osc.core.http_GET(url)).getroot() - except urllib2.HTTPError: + except HTTPError: return None # set of source repo name, target repo name, arch @@ -1087,4 +1092,3 @@ if __name__ == "__main__": app = CommandLineInterface() sys.exit( app.main() ) - diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/biarchtool.py new/openSUSE-release-tools-20181119.1714c58/biarchtool.py --- old/openSUSE-release-tools-20181113.f936dc2/biarchtool.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/biarchtool.py 2018-11-19 11:33:52.000000000 +0100 @@ -4,7 +4,11 @@ import sys import cmdln import logging -import urllib2 +try: + from urllib.error import HTTPError +except ImportError: + # python 2.x + from urllib2 import HTTPError import osc.core import ToolBase @@ -158,7 +162,7 @@ try: x = ET.fromstring(self.cached_GET(self.makeurl(['source', self.project, pkg, '_history'], {'rev': '1'}))) # catch deleted packages - except urllib2.HTTPError as e: + except HTTPError as e: if e.code == 404: continue raise e @@ -202,7 +206,7 @@ self.http_PUT(pkgmetaurl, data=ET.tostring(pkgmeta)) if self.caching: self._invalidate__cached_GET(pkgmetaurl) - except urllib2.HTTPError as e: + except HTTPError as e: logger.error('failed to update %s: %s', pkg, e) def add_explicit_disable(self, wipebinaries=False): @@ -242,7 +246,7 @@ 'cmd' : 'wipe', 'arch': self.arch, 'package' : pkg })) - except urllib2.HTTPError as e: + except HTTPError as e: logger.error('failed to update %s: %s', pkg, e) @@ -326,7 +330,7 @@ 'cmd' : 'wipe', 'arch': self.arch, 'package' : pkg })) - except urllib2.HTTPError as e: + except HTTPError as e: logger.error('failed to update %s: %s', pkg, e) class CommandLineInterface(ToolBase.CommandLineInterface): @@ -399,4 +403,3 @@ if __name__ == "__main__": app = CommandLineInterface() sys.exit( app.main() ) - diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/check_maintenance_incidents.py new/openSUSE-release-tools-20181119.1714c58/check_maintenance_incidents.py --- old/openSUSE-release-tools-20181113.f936dc2/check_maintenance_incidents.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/check_maintenance_incidents.py 2018-11-19 11:33:52.000000000 +0100 @@ -13,7 +13,11 @@ import osc.conf import osc.core -import urllib2 +try: + from urllib.error import HTTPError, URLError +except ImportError: + # python 2.x + from urllib2 import HTTPError, URLError import yaml from osclib.memoize import memoize @@ -56,7 +60,7 @@ url = osc.core.makeurl(apiurl, ('source', project, '00Meta', 'lookup.yml')) try: return yaml.safe_load(osc.core.http_GET(url)) - except (urllib2.HTTPError, urllib2.URLError): + except (HTTPError, URLError): return None # check if pkgname was submitted by the correct maintainer. If not, set @@ -165,4 +169,3 @@ app = ReviewBot.CommandLineInterface() app.clazz = MaintenanceChecker sys.exit( app.main() ) - diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/check_source.py new/openSUSE-release-tools-20181119.1714c58/check_source.py --- old/openSUSE-release-tools-20181113.f936dc2/check_source.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/check_source.py 2018-11-19 11:33:52.000000000 +0100 @@ -17,7 +17,12 @@ from osclib.core import devel_project_get from osclib.core import devel_project_fallback from osclib.core import group_members -import urllib2 +try: + from urllib.error import HTTPError +except ImportError: + # python 2.x + from urllib2 import HTTPError + import ReviewBot from osclib.conf import str2bool @@ -124,7 +129,7 @@ shutil.rmtree(os.path.join(target_package, '.osc')) os.rename(target_package, '_old') old_info = self.package_source_parse(target_project, target_package) - except urllib2.HTTPError: + except HTTPError: self.logger.error('failed to checkout %s/%s' % (target_project, target_package)) CheckSource.checkout_package(self.apiurl, source_project, source_package, revision=source_revision, @@ -218,7 +223,7 @@ try: xml = ET.parse(osc.core.http_GET(url)).getroot() - except urllib2.HTTPError as e: + except HTTPError as e: self.logger.error('ERROR in URL %s [%s]' % (url, e)) return ret @@ -261,7 +266,7 @@ try: result = osc.core.show_project_sourceinfo(self.apiurl, action.tgt_project, True, (action.tgt_package)) root = ET.fromstring(result) - except urllib2.HTTPError: + except HTTPError: return None # Decline the delete request if there is another delete/submit request against the same package diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/check_source_in_factory.py new/openSUSE-release-tools-20181119.1714c58/check_source_in_factory.py --- old/openSUSE-release-tools-20181113.f936dc2/check_source_in_factory.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/check_source_in_factory.py 2018-11-19 11:33:52.000000000 +0100 @@ -13,7 +13,12 @@ import osc.conf import osc.core -import urllib2 +try: + from urllib.error import HTTPError, URLError +except ImportError: + # python 2.x + from urllib2 import HTTPError, URLError + import yaml import ReviewBot @@ -89,7 +94,7 @@ sr = srprefix break requests = osc.core.get_request_list(apiurl, project, package, None, ['new', 'review'], 'submit') - except (urllib2.HTTPError, urllib2.URLError): + except (HTTPError, URLError): self.logger.error("caught exception while checking %s/%s", project, package) return None @@ -169,4 +174,3 @@ if __name__ == "__main__": app = CommandLineInterface() sys.exit( app.main() ) - diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/compare_pkglist.py new/openSUSE-release-tools-20181119.1714c58/compare_pkglist.py --- old/openSUSE-release-tools-20181113.f936dc2/compare_pkglist.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/compare_pkglist.py 2018-11-19 11:33:52.000000000 +0100 @@ -3,7 +3,12 @@ import argparse import logging import sys -import urllib2 +try: + from urllib.error import HTTPError +except ImportError: + # python 2.x + from urllib2 import HTTPError + import re from xml.etree import cElementTree as ET @@ -53,7 +58,7 @@ url = makeurl(self.apiurl, ['source', project, '_meta']) try: http_GET(url) - except urllib2.HTTPError: + except HTTPError: return False return True diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/fcc_submitter.py new/openSUSE-release-tools-20181119.1714c58/fcc_submitter.py --- old/openSUSE-release-tools-20181113.f936dc2/fcc_submitter.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/fcc_submitter.py 2018-11-19 11:33:52.000000000 +0100 @@ -3,7 +3,13 @@ import argparse import logging import sys -import urllib2 + +try: + from urllib.error import HTTPError, URLError +except ImportError: + # python 2.x + from urllib2 import HTTPError, URLError + import random import re from xml.etree import cElementTree as ET @@ -115,7 +121,7 @@ l = ET.tostring(flink) try: http_PUT(url, data=l) - except urllib2.HTTPError as e: + except HTTPError as e: raise e class FccSubmitter(object): @@ -159,7 +165,7 @@ def get_link(self, project, package): try: link = http_GET(makeurl(self.apiurl, ['source', project, package, '_link'])).read() - except (urllib2.HTTPError, urllib2.URLError): + except (HTTPError, URLError): return None return ET.fromstring(link) @@ -201,7 +207,7 @@ try: logging.debug("Gathering package_meta %s/%s" % (tgt_project, tgt_package)) osc.core.show_package_meta(self.apiurl, tgt_project, tgt_package) - except (urllib2.HTTPError, urllib2.URLError): + except (HTTPError, URLError): return True return False @@ -222,7 +228,7 @@ def check_multiple_specfiles(self, project, package): try: url = makeurl(self.apiurl, ['source', project, package], { 'expand': '1' } ) - except urllib2.HTTPError as e: + except HTTPError as e: if e.code == 404: return None raise e @@ -283,7 +289,7 @@ url = makeurl(self.apiurl, ['source', project, package, '{}?expand=1'.format('fcc_skip_pkgs')]) try: return http_GET(url).read() - except urllib2.HTTPError: + except HTTPError: return '' def crawl(self): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/leaper.py new/openSUSE-release-tools-20181119.1714c58/leaper.py --- old/openSUSE-release-tools-20181113.f936dc2/leaper.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/leaper.py 2018-11-19 11:33:52.000000000 +0100 @@ -16,7 +16,13 @@ import osc.core from osclib.conf import Config from osclib.core import devel_project_get -import urllib2 + +try: + from urllib.error import HTTPError +except ImportError: + # python 2.x + from urllib2 import HTTPError + import yaml import ReviewBot from check_source_in_factory import FactorySourceChecker @@ -64,7 +70,7 @@ root = ET.parse(osc.core.http_GET(osc.core.makeurl(self.apiurl, ['source', project], query=query))).getroot() packages = [i.get('name') for i in root.findall('entry')] - except urllib2.HTTPError as e: + except HTTPError as e: # in case the project doesn't exist yet (like sle update) if e.code != 404: raise e @@ -583,4 +589,3 @@ if __name__ == "__main__": app = CommandLineInterface() sys.exit( app.main() ) - diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/legal-auto.py new/openSUSE-release-tools-20181119.1714c58/legal-auto.py --- old/openSUSE-release-tools-20181113.f936dc2/legal-auto.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/legal-auto.py 2018-11-19 11:33:52.000000000 +0100 @@ -12,7 +12,12 @@ import requests as REQ import json import time -import urllib2 + +try: + from urllib.error import HTTPError +except ImportError: + # python 2.x + from urllib2 import HTTPError try: from xml.etree import cElementTree as ET @@ -53,7 +58,7 @@ def retried_GET(self, url): try: return http_GET(url) - except urllib2.HTTPError, e: + except HTTPError, e: if 500 <= e.code <= 599: print 'Retrying {}'.format(url) time.sleep(1) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/manager_42.py new/openSUSE-release-tools-20181119.1714c58/manager_42.py --- old/openSUSE-release-tools-20181113.f936dc2/manager_42.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/manager_42.py 2018-11-19 11:33:52.000000000 +0100 @@ -12,7 +12,13 @@ import osc.core from osc.core import get_commitlog from osc.core import get_request_list -import urllib2 + +try: + from urllib.error import HTTPError +except ImportError: + # python 2.x + from urllib2 import HTTPError + import subprocess import time import yaml @@ -86,7 +92,7 @@ self.lookup = {} try: self.lookup = yaml.safe_load(self._load_lookup_file(project)) - except urllib2.HTTPError as e: + except HTTPError as e: if e.code != 404: raise @@ -118,7 +124,7 @@ def retried_GET(self, url): try: return http_GET(url) - except urllib2.HTTPError as e: + except HTTPError as e: if 500 <= e.code <= 599: logger.warn('Retrying {}'.format(url)) time.sleep(1) @@ -135,7 +141,7 @@ query=query))) packages = [i.get('name') for i in root.findall('entry')] - except urllib2.HTTPError as e: + except HTTPError as e: if e.code == 404: logger.error("{}: {}".format(project, e)) packages = [] @@ -158,7 +164,7 @@ for package in sorted(packages): try: self.check_one_package(package) - except urllib2.HTTPError as e: + except HTTPError as e: logger.error("Failed to check {}: {}".format(package, e)) pass @@ -225,7 +231,7 @@ query['deleted'] = 1 return self.cached_GET(makeurl(self.apiurl, ['source', project, package, '_history'], query)) - except urllib2.HTTPError as e: + except HTTPError as e: if e.code == 404: return None raise @@ -360,7 +366,7 @@ try: link = self.cached_GET(makeurl(self.apiurl, ['source', project, package, '_link'])) - except urllib2.HTTPError: + except HTTPError: return None return ET.fromstring(link) @@ -430,4 +436,3 @@ http_DELETE = dryrun('DELETE') sys.exit(main(args)) - diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/obs_clone.py new/openSUSE-release-tools-20181119.1714c58/obs_clone.py --- old/openSUSE-release-tools-20181113.f936dc2/obs_clone.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/obs_clone.py 2018-11-19 11:33:52.000000000 +0100 @@ -10,7 +10,13 @@ from osc.core import makeurl from osc.core import show_upstream_rev from osclib.core import project_pseudometa_package -from urllib2 import HTTPError + +try: + from urllib.error import HTTPError +except ImportError: + # python 2.x + from urllib2 import HTTPError + import argparse import osc.conf import sys diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/osclib/cache.py new/openSUSE-release-tools-20181119.1714c58/osclib/cache.py --- old/openSUSE-release-tools-20181113.f936dc2/osclib/cache.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/osclib/cache.py 2018-11-19 11:33:52.000000000 +0100 @@ -7,9 +7,21 @@ import re import shutil import sys -import urlparse -import urllib -from StringIO import StringIO + +try: + from urllib.parse import unquote + from urllib.parse import urlsplit, SplitResult + from urllib.error import URLError, HTTPError + from io import StringIO +except ImportError: + # python 2.x + from urlparse import urlsplit, SplitResult + from urllib import unquote + from urllib2 import URLError, HTTPError + from StringIO import StringIO + +from io import BytesIO + from osc import conf from osc.core import urlopen from osclib.cache_manager import CacheManager @@ -122,7 +134,7 @@ @staticmethod def get(url): - url = urllib.unquote(url) + url = unquote(url) match, project = Cache.match(url) if match: path = Cache.path(url, project, include_file=True) @@ -176,7 +188,7 @@ @staticmethod def put(url, data): - url = urllib.unquote(url) + url = unquote(url) match, project = Cache.match(url) if match: path = Cache.path(url, project, include_file=True, makedirs=True) @@ -189,10 +201,10 @@ # be replaced with urlopen('file://...') to be consistent, but until # the need arrises StringIO has less overhead. text = data.read() - data = StringIO(text) + data = BytesIO(text) if conf.config['debug']: print('CACHE_PUT', url, project, file=sys.stderr) - f = open(path, 'w') + f = open(path, 'wb') f.write(text) f.close() @@ -200,7 +212,7 @@ @staticmethod def delete(url): - url = urllib.unquote(url) + url = unquote(url) match, project = Cache.match(url) if match: path = Cache.path(url, project, include_file=True) @@ -220,9 +232,9 @@ # Also delete version without query. This does not handle other # variations using different query strings. Handy for PUT with ?force=1. - o = urlparse.urlsplit(url) + o = urlsplit(url) if o.query != '': - url_plain = urlparse.SplitResult(o.scheme, o.netloc, o.path, '', o.fragment).geturl() + url_plain = SplitResult(o.scheme, o.netloc, o.path, '', o.fragment).geturl() Cache.delete(url_plain) @staticmethod @@ -250,9 +262,9 @@ @staticmethod def spliturl(url): - o = urlparse.urlsplit(url) - apiurl = urlparse.SplitResult(o.scheme, o.netloc, '', '', '').geturl() - path = urlparse.SplitResult('', '', o.path, o.query, '').geturl() + o = urlsplit(url) + apiurl = SplitResult(o.scheme, o.netloc, '', '', '').geturl() + path = SplitResult('', '', o.path, o.query, '').geturl() return (apiurl, path) @staticmethod @@ -262,7 +274,7 @@ parts = [Cache.CACHE_DIR] - o = urlparse.urlsplit(url) + o = urlsplit(url) parts.append(o.hostname) if project: @@ -273,7 +285,7 @@ os.makedirs(directory) if include_file: - parts.append(hashlib.sha1(url).hexdigest()) + parts.append(hashlib.sha1(url.encode('utf-8')).hexdigest()) return os.path.join(*parts) return directory diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/osclib/conf.py new/openSUSE-release-tools-20181119.1714c58/osclib/conf.py --- old/openSUSE-release-tools-20181113.f936dc2/osclib/conf.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/osclib/conf.py 2018-11-19 11:33:52.000000000 +0100 @@ -1,6 +1,6 @@ from __future__ import print_function -from ConfigParser import ConfigParser +from osc import OscConfigParser from collections import OrderedDict import io import os @@ -206,11 +206,11 @@ for k, v in DEFAULT[prj_pattern].items(): if k.startswith('_'): continue - if isinstance(v, basestring) and '%(project)s' in v: + if isinstance(v, str) and '%(project)s' in v: defaults[k] = v % {'project': project} - elif isinstance(v, basestring) and '%(project.lower)s' in v: + elif isinstance(v, str) and '%(project.lower)s' in v: defaults[k] = v % {'project.lower': project.lower()} - elif isinstance(v, basestring) and '%(version)s' in v: + elif isinstance(v, str) and '%(version)s' in v: defaults[k] = v % {'version': match.group('version')} else: defaults[k] = v @@ -223,19 +223,12 @@ # Update the configuration, only when it is necessary conf.config[self.project] = self.read_section(self.project, defaults) - # Take the common parameters and check that are there - params = [set(d) for d in DEFAULT.values()] - params = reduce(operator.__and__, params) - if not all(p in conf.config[self.project] for p in params): - msg = 'Please, add [%s] section in %s, see %s for details' % (self.project, self.conf_file, __file__) - raise Exception(msg) - def read_section(self, section, defaults): """OSC parser is a bit buggy. Re-read the configuration file to find extra sections. """ - cp = ConfigParser(defaults=defaults) + cp = OscConfigParser.OscConfigParser(defaults=defaults) cp.read(self.conf_file) if cp.has_section(section): return dict(cp.items(section)) @@ -246,9 +239,9 @@ from osclib.core import attribute_value_load config = attribute_value_load(apiurl, self.project, 'Config') if config: - cp = ConfigParser() - config = '[remote]\n' + config - cp.readfp(io.BytesIO(config)) + cp = OscConfigParser.OscConfigParser() + config = u'[remote]\n' + config + cp.readfp(io.StringIO(config)) return dict(cp.items('remote')) return None diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/osclib/core.py new/openSUSE-release-tools-20181119.1714c58/osclib/core.py --- old/openSUSE-release-tools-20181113.f936dc2/osclib/core.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/osclib/core.py 2018-11-19 11:33:52.000000000 +0100 @@ -316,7 +316,7 @@ if not len(value): return None - return value[0] + return str(value[0]) # New attributes must be defined manually before they can be used. Example: # `osc api /attribute/OSRT/IgnoredIssues/_meta outputs` diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/osclib/cycle.py new/openSUSE-release-tools-20181119.1714c58/osclib/cycle.py --- old/openSUSE-release-tools-20181113.f936dc2/osclib/cycle.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/osclib/cycle.py 2018-11-19 11:33:52.000000000 +0100 @@ -1,4 +1,9 @@ -import urllib2 +try: + from urllib.error import HTTPError +except ImportError: + # python 2.x + from urllib2 import HTTPError + from xml.etree import cElementTree as ET from osc.core import http_GET @@ -140,7 +145,7 @@ # print('Generating _builddepinfo for (%s, %s, %s)' % (project, repository, arch)) url = makeurl(self.apiurl, ['build/%s/%s/%s/_builddepinfo' % (project, repository, arch)]) root = http_GET(url).read() - except urllib2.HTTPError as e: + except HTTPError as e: print('ERROR in URL %s [%s]' % (url, e)) return root diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/osclib/list_command.py new/openSUSE-release-tools-20181119.1714c58/osclib/list_command.py --- old/openSUSE-release-tools-20181113.f936dc2/osclib/list_command.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/osclib/list_command.py 2018-11-19 11:33:52.000000000 +0100 @@ -1,9 +1,10 @@ +from __future__ import print_function + from colorama import Fore from osc import oscerr from osclib.request_splitter import RequestSplitter from osclib.supersede_command import SupersedeCommand - class ListCommand: SOURCE_PROJECT_STRIP = [ 'SUSE:SLE-12:', @@ -34,7 +35,7 @@ hide_source = self.api.project == 'openSUSE:Factory' for group in sorted(splitter.grouped.keys()): - print Fore.YELLOW + group + print(Fore.YELLOW + group) for request in splitter.grouped[group]['requests']: request_id = int(request.get('id')) @@ -58,13 +59,13 @@ if message: line += '\n' + Fore.WHITE + message + Fore.RESET - print ' ', line + print(' ' + line) if len(splitter.other): non_ring_packages = [] for request in splitter.other: non_ring_packages.append(request.find('./action/target').get('package')) - print 'Not in a ring:', ' '.join(sorted(non_ring_packages)) + print('Not in a ring: ' + ' '.join(sorted(non_ring_packages))) # Print requests not handled by staging process to highlight them. splitter.stageable = False diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/osclib/obslock.py new/openSUSE-release-tools-20181119.1714c58/osclib/obslock.py --- old/openSUSE-release-tools-20181113.f936dc2/osclib/obslock.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/osclib/obslock.py 2018-11-19 11:33:52.000000000 +0100 @@ -1,3 +1,5 @@ +from __future__ import print_function + from datetime import datetime import time import warnings @@ -95,7 +97,7 @@ stop = False if stop: - print 'Lock acquired by [%s] %s ago, reason <%s>. Try later.' % (user, delta, reason) + print('Lock acquired by [%s] %s ago, reason <%s>. Try later.' % (user, delta, reason)) exit(-1) self._write(self._signature()) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/osclib/prio_command.py new/openSUSE-release-tools-20181119.1714c58/osclib/prio_command.py --- old/openSUSE-release-tools-20181113.f936dc2/osclib/prio_command.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/osclib/prio_command.py 2018-11-19 11:33:52.000000000 +0100 @@ -1,6 +1,13 @@ +from __future__ import print_function + import json import osc -import urllib2 + +try: + from urllib.error import HTTPError +except ImportError: + # python 2.x + from urllib2 import HTTPError class PrioCommand(object): def __init__(self, api): @@ -22,12 +29,12 @@ if req.priority != priority: query = { 'cmd': 'setpriority', 'priority': priority } url = osc.core.makeurl(self.api.apiurl, ['request', reqid], query) - print reqid, message + print(reqid + ' ' + message) try: osc.core.http_POST(url, data=message) - print reqid, r['by'], priority - except urllib2.HTTPError as e: - print e + print(reqid + ' ' + r['by'] + ' ' + priority) + except HTTPError as e: + print(e) def perform(self, projects=None, priority=None): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/osclib/repair_command.py new/openSUSE-release-tools-20181119.1714c58/osclib/repair_command.py --- old/openSUSE-release-tools-20181113.f936dc2/osclib/repair_command.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/osclib/repair_command.py 2018-11-19 11:33:52.000000000 +0100 @@ -1,7 +1,12 @@ from __future__ import print_function import re -import urllib2 + +try: + from urllib.error import HTTPError +except ImportError: + # python 2.x + from urllib2 import HTTPError from osc import oscerr from osc.core import change_review_state @@ -37,7 +42,7 @@ staging_project = reviews[0] try: data = self.api.get_prj_pseudometa(staging_project) - except urllib2.HTTPError as e: + except HTTPError as e: if e.code == 404: data = None diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/osclib/stagingapi.py new/openSUSE-release-tools-20181119.1714c58/osclib/stagingapi.py --- old/openSUSE-release-tools-20181113.f936dc2/osclib/stagingapi.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/osclib/stagingapi.py 2018-11-19 11:33:52.000000000 +0100 @@ -1,4 +1,11 @@ -from cStringIO import StringIO +from __future__ import print_function + +try: + # python2 + from StringIO import StringIO +except ImportError: + from io import StringIO + from datetime import datetime import dateutil.parser import json @@ -156,7 +163,7 @@ return func(url) except HTTPError as e: if 500 <= e.code <= 599: - print 'Error {}, retrying {} in {}s'.format(e.code, url, retry_sleep_seconds) + print('Error {}, retrying {} in {}s'.format(e.code, url, retry_sleep_seconds)) time.sleep(retry_sleep_seconds) # increase sleep time up to one minute to avoid hammering # the server in case of real problems @@ -691,7 +698,7 @@ def load_prj_pseudometa(self, description_text): try: data = yaml.load(description_text) - if data is None: + if isinstance(data, str) or data is None: data = {} except (TypeError, AttributeError): data = {} @@ -1029,6 +1036,10 @@ return requests + def project_status_final(self, status): + """Determine if staging project is both active and no longer pending.""" + return status['overall_state'] in ['acceptable', 'review', 'failed'] + def days_since_last_freeze(self, project): """ Checks the last update for the frozen links @@ -1206,9 +1217,7 @@ # Skip inner-project links for letter staging if not self.is_adi_project(project) and sub_prj == project: continue - if self._supersede: - disable_build = self._package_disabled.get('/'.join([sub_prj, sub_pkg]), False) - self.create_package_container(sub_prj, sub_pkg, disable_build=disable_build) + self.create_package_container(sub_prj, sub_pkg) root = ET.Element('link', package=tar_pkg, project=project) url = self.makeurl(['source', sub_prj, sub_pkg, '_link']) @@ -1606,10 +1615,10 @@ u = self.makeurl(['build', prj], query=query) try: - print "tried to trigger rebuild for project '%s' package '%s'" % (prj, pkg) + print("tried to trigger rebuild for project '%s' package '%s'" % (prj, pkg)) http_POST(u) except: - print "could not trigger rebuild for project '%s' package '%s'" % (prj, pkg) + print("could not trigger rebuild for project '%s' package '%s'" % (prj, pkg)) def _candidate_adi_project(self): """Decide a candidate name for an ADI project.""" diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/rabbit-openqa.py new/openSUSE-release-tools-20181119.1714c58/rabbit-openqa.py --- old/openSUSE-release-tools-20181113.f936dc2/rabbit-openqa.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/rabbit-openqa.py 2018-11-19 11:33:52.000000000 +0100 @@ -15,13 +15,15 @@ from lxml import etree as ET from openqa_client.client import OpenQA_Client from openqa_client.exceptions import ConnectionError -from urllib import quote_plus -import requests try: from urllib.error import HTTPError, URLError + from urllib.parse import quote_plus except ImportError: # python 2.x from urllib2 import HTTPError, URLError + from urllib import quote_plus + +import requests from PubSubConsumer import PubSubConsumer @@ -267,4 +269,3 @@ l.run() except KeyboardInterrupt: l.stop() - diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/staging-report.py new/openSUSE-release-tools-20181119.1714c58/staging-report.py --- old/openSUSE-release-tools-20181113.f936dc2/staging-report.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/staging-report.py 2018-11-19 11:33:52.000000000 +0100 @@ -97,17 +97,13 @@ def report(self, project, aggregate=True, force=False): info = self.api.project_status(project, aggregate) - # Some staging projects do not have info like - # openSUSE:Factory:Staging:Gcc49 - if not info: - return - - if info['overall_state'] == 'empty': - return - - # The 'unacceptable' status means that the project will be - # replaced soon. Better do not disturb with noise. - if info['overall_state'] == 'unacceptable': + # Do not attempt to process projects without staging info, or projects + # in a pending state that will change before settling. This avoids + # intermediate notifications that may end up being spammy and for + # long-lived stagings where checks may be re-triggered multiple times + # and thus enter pending state (not seen on first run) which is not + # useful to report. + if not info or not self.api.project_status_final(info): return report_broken_packages = self._report_broken_packages(info) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/suppkg_rebuild.py new/openSUSE-release-tools-20181119.1714c58/suppkg_rebuild.py --- old/openSUSE-release-tools-20181113.f936dc2/suppkg_rebuild.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/suppkg_rebuild.py 2018-11-19 11:33:52.000000000 +0100 @@ -3,7 +3,12 @@ import argparse import logging import sys -import urllib2 +try: + from urllib.error import HTTPError +except ImportError: + # python 2.x + from urllib2 import HTTPError + import re import yaml from xml.etree import cElementTree as ET @@ -80,7 +85,7 @@ url = makeurl(self.apiurl, ['source', project, pkg], query=query) try: root = ET.parse(http_GET(url)).getroot() - except urllib2.HTTPError as e: + except HTTPError as e: if e.code == 404: continue raise diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/tests/obs.py new/openSUSE-release-tools-20181119.1714c58/tests/obs.py --- old/openSUSE-release-tools-20181113.f936dc2/tests/obs.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/tests/obs.py 2018-11-19 11:33:52.000000000 +0100 @@ -6,7 +6,12 @@ import string import time import urllib -import urllib2 +try: + from urllib.parse import unquote +except ImportError: + # python 2.x + from urllib import unquote + import urlparse import xml.etree.cElementTree as ET @@ -852,7 +857,7 @@ @GET('/search/request') def search_request(self, request, uri, headers): """Return a search result for /search/request.""" - query = urllib2.unquote(urlparse.urlparse(uri).query) + query = unquote(urlparse.urlparse(uri).query) assert query in ( "match=state/@name='review'+and+review[@by_group='factory-staging'+and+@state='new']+and+(target[@project='openSUSE:Factory']+or+target[@project='openSUSE:Factory:NonFree'])", "match=state/@name='review'+and+review[@by_user='factory-repo-checker'+and+@state='new']+and+(target[@project='openSUSE:Factory']+or+target[@project='openSUSE:Factory:NonFree'])" diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/totest-manager.py new/openSUSE-release-tools-20181119.1714c58/totest-manager.py --- old/openSUSE-release-tools-20181113.f936dc2/totest-manager.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/totest-manager.py 2018-11-19 11:33:52.000000000 +0100 @@ -17,7 +17,13 @@ import os import re import sys -import urllib2 + +try: + from urllib.error import HTTPError +except ImportError: + # python 2.x + from urllib2 import HTTPError + import logging import signal import time @@ -45,6 +51,12 @@ pass +class ImageProduct(object): + def __init__(self, package, archs): + self.package = package + self.archs = archs + + class ToTestBase(object): """Base class to store the basic interface""" @@ -52,7 +64,13 @@ product_repo = 'images' product_arch = 'local' livecd_repo = 'images' - livecd_archs = ['i586', 'x86_64'] + totest_container_repo = 'containers' + + main_products = [] + ftp_products = [] + container_products = [] + livecd_products = [] + image_products = [] def __init__(self, project, dryrun=False, norelease=False, api_url=None, openqa_server='https://openqa.opensuse.org', test_subproject=None): self.project = project @@ -97,11 +115,16 @@ def current_version(self): return self.release_version() - def binaries_of_product(self, project, product): - url = self.api.makeurl(['build', project, self.product_repo, self.product_arch, product]) + def binaries_of_product(self, project, product, repo=None, arch=None): + if repo is None: + repo = self.product_repo + if arch is None: + arch = self.product_arch + + url = self.api.makeurl(['build', project, repo, arch, product]) try: f = self.api.retried_GET(url) - except urllib2.HTTPError: + except HTTPError: return [] ret = [] @@ -131,10 +154,10 @@ return result.group(1) raise NotFoundException("can't find %s ftp version" % project) - def iso_build_version(self, project, tree, base=None): + def iso_build_version(self, project, tree, base=None, repo=None, arch=None): if not base: base = self.project_base - for binary in self.binaries_of_product(project, tree): + for binary in self.binaries_of_product(project, tree, repo=repo, arch=arch): result = re.match(r'.*-(?:Build|Snapshot)([0-9.]+)(?:-Media.*\.iso|\.docker\.tar\.xz)', binary) if result: return result.group(1) @@ -378,7 +401,7 @@ return None # docker container has no size limit - if re.match(r'opensuse-leap-image.*', package): + if re.match(r'opensuse-.*-image.*', package): return None if '-Addon-NonOss-ftp-ftp' in package: @@ -442,46 +465,69 @@ if not self.package_ok(self.project, product, self.product_repo, self.product_arch): return False + for product in self.image_products + self.container_products: + for arch in product.archs: + if not self.package_ok(self.project, product.package, self.product_repo, arch): + return False + if len(self.livecd_products): if not self.all_repos_done('%s:Live' % self.project): return False - for arch in self.livecd_archs: - for product in self.livecd_products: - if not self.package_ok('%s:Live' % self.project, product, self.livecd_repo, arch): + for product in self.livecd_products: + for arch in product.archs: + if not self.package_ok('%s:Live' % self.project, product.package, + self.product_repo, arch): return False return True - def _release_package(self, project, package, set_release=None): + def _release_package(self, project, package, set_release=None, repository=None, + target_project=None, target_repository=None): query = {'cmd': 'release'} if set_release: query['setrelease'] = set_release - # FIXME: make configurable. openSUSE:Factory:ARM currently has multiple - # repos with release targets, so obs needs to know which one to release - if project == 'openSUSE:Factory:ARM': - query['repository'] = 'images' + if repository is not None: + query['repository'] = repository + + if target_project is not None: + # Both need to be set + query['target_project'] = target_project + query['target_repository'] = target_repository baseurl = ['source', project, package] url = self.api.makeurl(baseurl, query=query) if self.dryrun or self.norelease: - logger.info("release %s/%s (%s)" % (project, package, set_release)) + logger.info("release %s/%s (%s)" % (project, package, query)) else: self.api.retried_POST(url) def _release(self, set_release=None): for product in self.ftp_products: - self._release_package(self.project, product) + self._release_package(self.project, product, repository=self.product_repo) for cd in self.livecd_products: self._release_package('%s:Live' % - self.project, cd, set_release=set_release) + self.project, cd.package, set_release=set_release, + repository=self.livecd_repo) + + for image in self.image_products: + self._release_package(self.project, image.package, set_release=set_release, + repository=self.product_repo) for cd in self.main_products: - self._release_package(self.project, cd, set_release=set_release) + self._release_package(self.project, cd, set_release=set_release, + repository=self.product_repo) + + for container in self.container_products: + # Containers are built in the same repo as other image products, + # but released into a different repo in :ToTest + self._release_package(self.project, container.package, repository=self.product_repo, + target_project=self.test_project, + target_repository=self.totest_container_repo) def update_totest(self, snapshot=None): release = 'Snapshot%s' % snapshot if snapshot else None @@ -495,7 +541,13 @@ logger.info('Publish test project content') if not (self.dryrun or self.norelease): self.api.switch_flag_in_prj( - self.test_project, flag='publish', state='enable') + self.test_project, flag='publish', state='enable', + repository=self.product_repo) + if self.container_products: + logger.info('Releasing container products from ToTest') + for container in self.container_products: + self._release_package(self.test_project, container.package, + repository=self.totest_container_repo) def totest_is_publishing(self): """Find out if the publishing flag is set in totest's _meta""" @@ -508,7 +560,9 @@ return True for flag in root.find('publish'): - if flag.get('repository', None) or flag.get('arch', None): + if flag.get('repository', None) not in [None, self.product_repo]: + continue + if flag.get('arch', None): continue if flag.tag == 'enable': return True @@ -652,7 +706,7 @@ # XXX still legacy for cd in self.livecd_products: self._release_package('%s:Live' % - self.project, cd, set_release=set_release) + self.project, cd.package, set_release=set_release) def release_version(self): url = self.api.makeurl(['build', self.project, 'standard', self.arch(), @@ -680,8 +734,12 @@ # XXX: don't care about nonoss atm. continue builds.add(self.ftp_build_version(self.project, p)) - for p in self.main_products + self.livecd_products: + for p in self.main_products: builds.add(self.iso_build_version(self.project, p)) + for p in self.livecd_products + self.image_products: + for arch in p.archs: + builds.add(self.iso_build_version(self.project, p.package, + arch=p.arch)) ret = (len(builds) == 1) if ret is False: @@ -706,9 +764,9 @@ ftp_products = ['000product:openSUSE-ftp-ftp-i586_x86_64', '000product:openSUSE-Addon-NonOss-ftp-ftp-i586_x86_64'] - livecd_products = ['livecd-tumbleweed-kde', - 'livecd-tumbleweed-gnome', - 'livecd-tumbleweed-x11'] + livecd_products = [ImageProduct('livecd-tumbleweed-kde', ['i586', 'x86_64']), + ImageProduct('livecd-tumbleweed-gnome', ['i586', 'x86_64']), + ImageProduct('livecd-tumbleweed-x11', ['i586', 'x86_64'])] def __init__(self, *args, **kwargs): ToTestBase.__init__(self, *args, **kwargs) @@ -729,8 +787,6 @@ ftp_products = ['000product:openSUSE-ftp-ftp-ppc64_ppc64le'] - livecd_products = [] - def __init__(self, *args, **kwargs): ToTestBase.__init__(self, *args, **kwargs) @@ -753,8 +809,6 @@ ftp_products = ['000product:openSUSE-ftp-ftp-s390x'] - livecd_products = [] - def __init__(self, *args, **kwargs): ToTestBase.__init__(self, *args, **kwargs) @@ -780,8 +834,7 @@ '000product:openSUSE-ftp-ftp-armv7hl', '000product:openSUSE-ftp-ftp-armv6hl'] - livecd_products = ['JeOS'] - livecd_archs = ['armv7l'] + livecd_products = [ImageProduct('JeOS', ['armv7l'])] def __init__(self, *args, **kwargs): ToTestFactory.__init__(self, *args, **kwargs) @@ -806,8 +859,6 @@ '000product:openSUSE-Addon-NonOss-ftp-ftp-x86_64' ] - livecd_products = [] - def openqa_group(self): return 'openSUSE Leap 15' @@ -825,8 +876,7 @@ '000product:openSUSE-ftp-ftp-armv7hl', ] - livecd_products = ['JeOS'] - livecd_archs = ['armv7l'] + livecd_products = [ImageProduct('JeOS', ['armv7l'])] # Leap 15.1 ARM still need to update snapshot set_snapshot_number = True @@ -850,8 +900,6 @@ '000product:openSUSE-ftp-ftp-armv7hl', ] - livecd_products = [] - # Leap 15.0 Ports still need to update snapshot set_snapshot_number = True @@ -871,24 +919,19 @@ class ToTest150Images(ToTestBaseNew): - main_products = [ - 'livecd-leap-gnome', - 'livecd-leap-kde', - 'livecd-leap-x11', - 'opensuse-leap-image:docker', - 'opensuse-leap-image:lxc', - 'kiwi-templates-Leap15-JeOS:MS-HyperV', - 'kiwi-templates-Leap15-JeOS:OpenStack-Cloud', - 'kiwi-templates-Leap15-JeOS:VMware', - 'kiwi-templates-Leap15-JeOS:XEN', - 'kiwi-templates-Leap15-JeOS:kvm-and-xen', + image_products = [ + ImageProduct('livecd-leap-gnome', ['x86_64']), + ImageProduct('livecd-leap-kde', ['x86_64']), + ImageProduct('livecd-leap-x11', ['x86_64']), + ImageProduct('opensuse-leap-image:docker', ['x86_64']), + ImageProduct('opensuse-leap-image:lxc', ['x86_64']), + ImageProduct('kiwi-templates-Leap15-JeOS:MS-HyperV', ['x86_64']), + ImageProduct('kiwi-templates-Leap15-JeOS:OpenStack-Cloud', ['x86_64']), + ImageProduct('kiwi-templates-Leap15-JeOS:VMware', ['x86_64']), + ImageProduct('kiwi-templates-Leap15-JeOS:XEN', ['x86_64']), + ImageProduct('kiwi-templates-Leap15-JeOS:kvm-and-xen', ['x86_64']), ] - ftp_products = [] - - livecd_products = [] - product_arch = 'x86_64' - # docker image has a different number need_same_build_number = False set_snapshot_number = True @@ -902,8 +945,13 @@ def write_version_to_dashboard(self, target, version): super(ToTest150Images, self).write_version_to_dashboard('{}_images'.format(target), version) + def current_version(self): + return self.iso_build_version(self.project, self.image_products[0].package, + arch=self.image_products[0].archs[0]) + def get_current_snapshot(self): - return self.iso_build_version(self.project + ':ToTest', self.main_products[0]) + return self.iso_build_version(self.project + ':ToTest', self.image_products[0].package, + arch=self.image_products[0].archs[0]) def _release(self, set_release=None): ToTestBase._release(self, set_release) @@ -912,6 +960,7 @@ return 13 class ToTest151Images(ToTest150Images): + container_products = [ImageProduct('opensuse-leap-image:docker', ['x86_64'])] def openqa_group(self): return 'openSUSE Leap 15.1 Images' @@ -948,8 +997,6 @@ '_product:SLES-ftp-POOL-x86_64', ] - livecd_products = [] - class ToTestSLE15(ToTestSLE): main_products = [ '000product:SLES-cd-DVD-aarch64', @@ -965,8 +1012,6 @@ '000product:SLES-ftp-POOL-x86_64', ] - livecd_products = [] - class CommandlineInterface(cmdln.Cmdln): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/update_crawler.py new/openSUSE-release-tools-20181119.1714c58/update_crawler.py --- old/openSUSE-release-tools-20181113.f936dc2/update_crawler.py 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/update_crawler.py 2018-11-19 11:33:52.000000000 +0100 @@ -4,7 +4,13 @@ import itertools import logging import sys -import urllib2 + +try: + from urllib.error import HTTPError +except ImportError: + # python 2.x + from urllib2 import HTTPError + import time from xml.etree import cElementTree as ET @@ -79,7 +85,7 @@ def retried_GET(self, url): try: return http_GET(url) - except urllib2.HTTPError as e: + except HTTPError as e: if 500 <= e.code <= 599: print 'Retrying {}'.format(url) time.sleep(1) @@ -191,7 +197,7 @@ ))) if root.get('project') is None and root.get('cicount'): return True - except urllib2.HTTPError as err: + except HTTPError as err: # if there is no link, it can't be a link if err.code == 404: return False @@ -370,4 +376,3 @@ http_DELETE = dryrun('DELETE') sys.exit(main(args)) - diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openSUSE-release-tools-20181113.f936dc2/userscript/README.md new/openSUSE-release-tools-20181119.1714c58/userscript/README.md --- old/openSUSE-release-tools-20181113.f936dc2/userscript/README.md 2018-11-13 23:46:08.000000000 +0100 +++ new/openSUSE-release-tools-20181119.1714c58/userscript/README.md 2018-11-19 11:33:52.000000000 +0100 @@ -1,5 +1,17 @@ # User Scripts -The scripts may be installed in one's browser (using Greasemonkey or Tampermonkey) to provide additional features using OBS via the web. +The scripts may be installed in one's browser using the Tampermonkey extension to provide additional features using OBS via the web. After installing the extension simply click on the link for the desired script below to install it. Any scripts that provide an interface for making changes depend on the user being logged in to the OBS instance with a user with the appropriate permissions to complete the task. - [Staging Move Drag-n-Drop](https://github.com/openSUSE/openSUSE-release-tools/raw/master/userscript/staging-move-drag-n-drop.user.js) + + Provides a drag-n-drop interface for moving requests between stagings using the staging dashboard. The staging dashboard can be found by visiting `/project/staging_projects/$PROJECT` on the relevant OBS instance where `$PROJECT` is the target project for the stagings (ex. `openSUSE:Factory` or `SUSE:SLE-15-SP1:GA`). + + Once on the staging dashboard the option to `enter move mode` will be available in the legend on the right side. Either click the yellow box or press _ctrl + m_ as indicated when hovering over the box. After entering _move mode_ individual requests can be dragged between stagings or groups selected and moved together. Groups may be selected by either clicking in an open area and dragging a box around the desired requests to select them and/or by hold _ctrl_ and clicking on requests to add or remove them from the selections. + + Once all desired moves have been made the _Apply_ button in the bottom center of the window may be press to apply the changes to the staging. + + Note that the staging lock is still in effect and thus the moves will fail if someone else has acquired the staging lock. Also note that after a failure or decision to not go through with moves there is currently no way to leave/reset move mode, but reloading the page will clear any changes made in move mode. + +## Troubleshooting + +Additional information after a failed operation is available in the browser console which may be accessed by _right-clicking_ on the page and selecting _Inspect_ or _Inspect Element_ and clicking the _Console_ tab. ++++++ openSUSE-release-tools.obsinfo ++++++ --- /var/tmp/diff_new_pack.Z4tQWo/_old 2018-11-19 23:34:30.107016785 +0100 +++ /var/tmp/diff_new_pack.Z4tQWo/_new 2018-11-19 23:34:30.111016781 +0100 @@ -1,5 +1,5 @@ name: openSUSE-release-tools -version: 20181113.f936dc2 -mtime: 1542149168 -commit: f936dc2f47c066dfc9e2cad029cb66c8dd5505ec +version: 20181119.1714c58 +mtime: 1542623632 +commit: 1714c582f6543e92e2762d7d63da41d066df37ee