Gergő Tisza has uploaded a new change for review.
https://gerrit.wikimedia.org/r/225827
Change subject: [WIP] Version update: 7.4.3 -> 7.6.2
......................................................................
[WIP] Version update: 7.4.3 -> 7.6.2
Also add a script for easier generation of the list of packages
available via Debian.
Bug: T105374
Change-Id: Iac82ea202856315ba187c852070a4337bd7781a6
---
A =1.6.0,
M README.md
A list-debian-packages.py
A requirements.pyc
A requirements_list.py
A requirements_list.pyc
A requirements_map.py
A requirements_map.pyc
8 files changed, 168 insertions(+), 22 deletions(-)
git pull ssh://gerrit.wikimedia.org:29418/operations/software/sentry
refs/changes/27/225827/1
diff --git a/=1.6.0, b/=1.6.0,
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/=1.6.0,
diff --git a/README.md b/README.md
index 437445f..bc6dac8 100644
--- a/README.md
+++ b/README.md
@@ -3,34 +3,44 @@
### Steps to reproduce/update the contents of this venv ###
1. Create a new Debian Jessie VM on labs
-2. Install the dependencies listed under "System dependencies" below with apt
-3. Create the /srv/deployment/sentry/ folder
-4. Go to that folder and run "virtualenv --system-site-packages sentry" to
create the venv
-5. Run "source sentry/bin/activate" to activate the venv
-6. Run "pip install sentry[postgres]" to install the latest version of Sentry
and its dependencies into the venv with postgres support
-7. Set up git-review in the /srv/deployment/sentry/sentry folder and commit
the changes to this repository
+2. Install the virtualenv package
+3. Install the dependencies listed under "System dependencies" below with apt
+4. Create the /srv/deployment/sentry/ folder
+5. Go to that folder and run "virtualenv --system-site-packages sentry" to
create the venv
+6. Run "source sentry/bin/activate" to activate the venv
+7. Run "pip install sentry[postgres]" to install the latest version of Sentry
and its dependencies into the venv with postgres support
+8. Set up git-review in the /srv/deployment/sentry/sentry folder and commit
the changes to this repository
### System dependencies ###
-The following dependencies are currently (2015-04-01) available as Debian
packages in Jessie and meet Sentry 7.4.3's requirements:
+The following dependencies are currently (2015-07-20) available as Debian
packages in Jessie and meet Sentry 7.6.2's requirements:
-python-beautifulsoup
-python-cssutils
-python-django-crispy-forms
-python-django-jsonfield
-python-django-picklefield
-python-ipaddr
-python-mock
-python-progressbar
-python-pytest
-python-redis
-python-six
-python-setproctitle
-
-python-psycopg2
+* python-beautifulsoup
+* python-celery
+* python-cssutils
+* python-django-crispy-forms
+* python-django-jsonfield
+* python-django-picklefield
+* python-ipaddr
+* python-mock
+* python-progressbar
+* python-psycopg2
+* python-pytest
+* python-dateutil
+* python-redis
+* python-six
+* python-setproctitle
The machine where this venv is set up to run needs to have the above Devian
packages installed, with versions that
meet the requirements declared by Sentry.
+
+Additional packages that are not required for running Sentry but needed to
build some of the pip packages:
+* libxml2-dev
+* libxslt1-dev
+* libffi-dev
+* libssl-dev
+
+list-debian-packages.py can help with the updating of the dependency list.
### Possible future issues and reasoning ###
@@ -42,4 +52,4 @@
It's tempting to not use system packages at all and put all the dependencies
in the venv, and we might end up doing that.
However the venv has the major drawback of needing manual steps to be updated
when security updates are issued, which is
why we've started this by using as many system packages as we can. Long term,
the best of both worlds is probably
-to go full venv and set up a bot that will update our venvs automatically.
\ No newline at end of file
+to go full venv and set up a bot that will update our venvs automatically.
diff --git a/list-debian-packages.py b/list-debian-packages.py
new file mode 100755
index 0000000..6e9611e
--- /dev/null
+++ b/list-debian-packages.py
@@ -0,0 +1,56 @@
+#!/usr/bin/env python
+
+debug = False # or True
+
+import apt
+import operator
+from distutils.version import LooseVersion
+from pip.req import InstallRequirement
+
+from requirements_list import install_requires, postgres_requires
+from requirements_map import package_map
+
+ops_map = {
+ '<': operator.lt,
+ '>': operator.gt,
+ '<=': operator.le,
+ '>=': operator.ge,
+ '=': operator.eq,
+ '==': operator.eq,
+}
+cache = apt.Cache()
+in_debian = []
+failures = []
+
+for line in install_requires + postgres_requires:
+ req = InstallRequirement.from_line(line)
+ if req.req.extras:
+ failures.append('%s - has extras' % req)
+ continue
+ if not req.name in package_map:
+ failures.append('%s - no debian package' % req)
+ continue
+ pkg = cache[package_map[req.name]]
+ debian_version = LooseVersion(pkg.versions[0].version)
+ specs_match = True
+ for spec in req.req.specs:
+ spec_version = LooseVersion(spec[1])
+ if not ops_map[spec[0]](debian_version, spec_version):
+ specs_match = False
+ failures.append('%s - wrong version %s' % (req, spec[1]))
+ break
+ if specs_match:
+ in_debian.append((req, pkg))
+
+if len(in_debian):
+ print 'Packages which can be installed via apt:'
+ for item in in_debian:
+ if debug:
+ print " require_package('%s') # %s %s" % (item[1].name,
item[0], item[1].versions[0].version)
+ else:
+ print " require_package('%s')" % item[1].name
+
+if debug:
+ print '\nsudo apt-get install %s\n' % ' '.join([i[1].name for i in
in_debian])
+ print '\n'.join(failures)
+
diff --git a/requirements.pyc b/requirements.pyc
new file mode 100644
index 0000000..ce8cc95
--- /dev/null
+++ b/requirements.pyc
Binary files differ
diff --git a/requirements_list.py b/requirements_list.py
new file mode 100644
index 0000000..35d1bfa
--- /dev/null
+++ b/requirements_list.py
@@ -0,0 +1,51 @@
+# copied from Sentry's setup.py
+
+install_requires = [
+ 'BeautifulSoup>=3.2.1,<3.3.0',
+ 'celery>=3.1.8,<3.2.0',
+ 'cssutils>=0.9.9,<0.10.0',
+ 'Django>=1.6.0,<1.7',
+ 'django-bitfield>=1.7.0,<1.8.0',
+ 'django-crispy-forms>=1.4.0,<1.5.0',
+ 'django-paging>=0.2.5,<0.3.0',
+ 'django-jsonfield>=0.9.13,<0.10.0',
+ 'django-picklefield>=0.3.0,<0.4.0',
+ 'django-recaptcha>=1.0.0,<1.1.0',
+ 'django-social-auth>=0.7.28,<0.8.0',
+ 'django-statsd-mozilla>=0.3.14.0,<0.3.15.0',
+ 'django-sudo>=1.1.3,<1.2.0',
+ 'django-templatetag-sugar>=0.1.0',
+ 'djangorestframework>=2.3.8,<2.4.0',
+ 'email-reply-parser>=0.2.0,<0.3.0',
+ 'enum34>=0.9.18,<0.10.0',
+ 'exam>=0.5.1',
+ 'gunicorn>=19.2.1,<20.0.0',
+ 'ipaddr>=2.1.11,<2.2.0',
+ 'logan>=0.7.1,<0.8.0',
+ 'lxml>=3.4.1',
+ 'mock>=0.8.0',
+ 'nydus>=0.11.0,<0.12.0',
+ 'markdown>=2.4.1,<2.5.0',
+ 'petname>=1.7,<1.8',
+ 'progressbar>=2.2,<2.4',
+ 'pytest',
+ 'pytest-django',
+ 'python-dateutil>=2.0.0,<3.0.0',
+ 'python-memcached>=1.53,<2.0.0',
+ 'raven>=5.3.0',
+ 'redis>=2.7.0,<2.11.0',
+ 'requests[security]>=2.5.1,<2.6.0',
+ 'simplejson>=3.1.0,<3.4.0',
+ 'six>=1.6.0,<2.0.0',
+ 'setproctitle>=1.1.7,<1.2.0',
+ 'statsd>=3.1.0,<3.2.0',
+ 'South==1.0.1',
+ 'toronado>=0.0.4,<0.1.0',
+ 'ua-parser>=0.3.5',
+ 'urllib3>=1.7.1,<1.8.0',
+]
+
+postgres_requires = [
+ 'psycopg2>=2.5.0,<2.6.0',
+]
+
diff --git a/requirements_list.pyc b/requirements_list.pyc
new file mode 100644
index 0000000..20ec577
--- /dev/null
+++ b/requirements_list.pyc
Binary files differ
diff --git a/requirements_map.py b/requirements_map.py
new file mode 100644
index 0000000..bd25605
--- /dev/null
+++ b/requirements_map.py
@@ -0,0 +1,29 @@
+# map of Sentry python packages to apt packages, maintained manually
+package_map = {
+ 'BeautifulSoup': 'python-beautifulsoup',
+ 'celery': 'python-celery',
+ 'cssutils': 'python-cssutils',
+ 'Django': 'python-django',
+ 'django-bitfield': 'python-django-bitfield',
+ 'django-crispy-forms': 'python-django-crispy-forms',
+ 'django-jsonfield': 'python-django-jsonfield',
+ 'django-picklefield': 'python-django-picklefield',
+ 'djangorestframework': 'python-djangorestframework',
+ 'enum34': 'python-enum34',
+ 'gunicorn': 'gunicorn',
+ 'ipaddr': 'python-ipaddr',
+ 'lxml': 'python-lxml',
+ 'mock': 'python-mock',
+ 'markdown': 'python-markdown',
+ 'progressbar': 'python-progressbar',
+ 'pytest': 'python-pytest',
+ 'python-dateutil': 'python-dateutil',
+ 'redis': 'python-redis',
+ 'simplejson': 'python-simplejson',
+ 'six': 'python-six',
+ 'setproctitle': 'python-setproctitle',
+ 'statsd': 'python-statsd',
+ 'South': 'python-django-south',
+ 'urllib3': 'python-urllib3',
+ 'psycopg2': 'python-psycopg2',
+}
diff --git a/requirements_map.pyc b/requirements_map.pyc
new file mode 100644
index 0000000..67afd33
--- /dev/null
+++ b/requirements_map.pyc
Binary files differ
--
To view, visit https://gerrit.wikimedia.org/r/225827
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iac82ea202856315ba187c852070a4337bd7781a6
Gerrit-PatchSet: 1
Gerrit-Project: operations/software/sentry
Gerrit-Branch: master
Gerrit-Owner: Gergő Tisza <gti...@wikimedia.org>
_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
