-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
... >> I think there are a couple of factors: > >> 1) You're on a pretty impressive link. For me, the bzr+ssh handshake >> takes about 2.8s. (time echo hello | ssh bazaar....) So adding even 1s >> to that isn't terrible. > > Averages out to about 1.3s for me. > I'm curious what your ping time is. As here it is pretty consistently 2.5s+. And my ping is pretty good at 34ms. >> 2) debian was *much* slower than Ubuntu branches. This is because >> apparently Launchpad doesn't track "Published" for debian. So when >> making an api request for "ubuntu/bzr" we can ask for what the single >> most recent Published state is. For Debian we have to ask for the >> Pending list. Now, we limit the list to 1 entry, and Launchpad sorts in >> newest-first. However, I imagine at the app/DB level, Launchpad has to >> read in all the rows and sort them. Turns out they also have a SQUID cache in front of the app servers, so repeat-testing may or may not be valid. For the real use case, I think it actually isn't valid, since most of the requests are going to be cold, and aren't going to be repeated in within the cache lifetime. On devpad, with SQUID it takes 50ms to get the response. Explicitly skipping the cache (by setting an Auth header), I get cold 750ms, then 190-360ms. (for debian). 197-250ms for ubuntu. Though again, there is still some DB level caching when repeating these queries. So regardless of latency, the "best" worst-case time is still 750ms. So adding that to your 1.3s ssh handshake is certainly going to show up. I believe these queries haven't been particularly optimized on the LP side, either. I also ran into a very strange python bug. Which is that making the first SSL connection takes about 600ms on my machine, but making a second SSL connection (not shared, not even to the same host), takes only 200ms. I'm guessing something with openssl caching the certificate files or something. > >> 3) The API request is a lot slower today than when I was testing it >> during development. I don't have any idea why that would be. But pulling >> out the setup code into: > > Interesting. I'm wondering if it is the second-ssl connection thing. It shouldn't be, because there should have still only been a single https connection, but maybe I'm missing something. > >> Great. I think it would certainly be reasonable to have a knob that >> issues short form for people who know what everything means and what it >> to be as easy to parse as possible. > > +1, and thanks! > > Something else I've been wanting to do, but which probably isn't within > bzr/udd mission. I want an uber-rmadison which tells me all the versions of a > package in both Ubuntu and Debian, *and* in the packaging branches, without > having to download anything. Yeah, I should just go write the thing. > > -Barry I'm attaching the script that maxb wrote which I used to get bootstrapped. It lets you do a "madison" request against launchpad's data. It currently only gives you one archive, and if you want debian, you have to switch from Published to Pending, and watch out for series, etc. You can get a lot of data back from a single query, but it may not always have everything you want in it. It appears to purely sort the result by debian version if you don't specify an explicit distro_series. So as the top N results you get: bzr | 2.4.0~beta5-3 | sid | main bzr | 2.4.0~beta5-2 | sid | main bzr | 2.4.0~beta5-1 | sid | main bzr | 2.4.0~beta4-4 | sid | main bzr | 2.4.0~beta4-2 | sid | main bzr | 2.4.0~beta4-1 | sid | main bzr | 2.4.0~beta3-2 | sid | main bzr | 2.4.0~beta2-3 | sid | main bzr | 2.4.0~beta2-2 | experimental | main bzr | 2.4.0~beta2-1 | experimental | main bzr | 2.4.0~beta1-2 | experimental | main bzr | 2.4.0~beta1-1 | experimental | main bzr | 2.3.1-2 | wheezy | main bzr | 2.3.1-2 | sid | main which inter-mixes series. Its pretty easy to get the tags from the packaging branches. I'd be happy to work with you on something like that. Especially on working out ways to avoid round-trips to expand the info. It would be *really* nice if Launchpad tracked Published for debian the way they do for Ubuntu: bzr | 2.4.0~beta5-2ubuntu1 | oneiric | main bzr | 2.3.3-0ubuntu1 | natty-proposed | main bzr | 2.3.1-1ubuntu1 | natty | main bzr | 2.2.4-0ubuntu1 | maverick-updates | main bzr | 2.2.0-1 | maverick | main bzr | 2.1.4-0ubuntu1 | lucid-updates | main bzr | 2.1.1-1 | lucid | main bzr | 2.0.2-0ubuntu1 | karmic-updates | main bzr | 2.0.0-0ubuntu1 | karmic | main bzr | 1.13.1-1 | jaunty | main bzr | 1.6.1-1 | intrepid | main bzr | 1.3.1-1ubuntu0.1 | hardy-updates | main bzr | 1.3.1-1 | hardy | main bzr | 1.3.1~rc1-0ubuntu1~gutsy1 | gutsy-backports | main bzr | 0.90-1 | gutsy | main bzr | 0.8.2-1ubuntu3 | dapper | main Anyway, I'll start poking at the verbosity stuff next. John =:-> -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Cygwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk4llcMACgkQJdeBCYSNAAMopQCfdDAdBOX6ZnJbWtUP+Dr/rA6K zMsAn1iK6UxmxovtxlCyWU2U1GcI6+do =PWm/ -----END PGP SIGNATURE-----
#!/usr/bin/python
try:
import simplejson as json
except ImportError:
import json # should be present since 2.6
import sys
import urllib
import urllib2
API_ROOT = 'https://api.launchpad.net/1.0/'
def lpmadison(archive_id, source_name):
req = urllib2.Request(API_ROOT + archive_id + '?' +
urllib.urlencode({
'ws.op': 'getPublishedSources',
'exact_match': 'true',
'source_name': '"' + source_name + '"',
'status': 'Published', # Used for "ubuntu"
# 'status': 'Pending', # Used for 'debian'
'ws.size': '300',
#'distro_series': '/debian/sid',
#'distro_series': '/ubuntu/natty',
#'pocket': 'Proposed',
}),
# Used to be needed - no longer required:
# headers={ 'Authorization': 'OAuth realm="",
oauth_consumer_key="lpmadison", oauth_signature_method="PLAINTEXT",
oauth_version="1.0", oauth_token="", oauth_signature="%26"' }
)
try:
response = urllib2.urlopen(req)
json_txt = response.read()
except urllib2.HTTPError, e:
print e
print e.read()
return
# print json
o = json.loads(json_txt)
rows = []
for e in o['entries']:
name = e['source_package_name']
ver = e['source_package_version']
comp = e['component_name']
pocket = e['pocket'].lower()
series = e['distro_series_link'].split('/')[-1]
if pocket != 'release':
series += '-' + pocket
rows.append((name, ver, series, comp))
col_sizes = [0, 0, 0, 0]
for row in rows:
for i in xrange(len(row)):
col_sizes[i] = max(col_sizes[i], len(row[i]))
for row in rows:
print "%-*s | %-*s | %-*s | %-*s" % (
col_sizes[0], row[0],
col_sizes[1], row[1],
col_sizes[2], row[2],
col_sizes[3], row[3],
)
if __name__ == '__main__':
archive_id = sys.argv[1]
source_name = sys.argv[2]
lpmadison(archive_id, source_name)
up_to_date.py.sig
Description: Binary data
-- ubuntu-distributed-devel mailing list ubuntu-distributed-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel
