[Bug 828664] Re: importer doesn't cope with broken ssh pipe

2011-08-24 Thread Martin Pool 
ubuntu-docs seems to be ok, but eglibc and php5 failed with a similar
error.

so for eglibc it also seems to have run into a stale transport and not
to be handling it properly:

26225.211  refresh_possible_transports dropping stale transport 

 ([Errno 32] Broken pipe)
26225.211  refresh_possible_transports encountered a smart transport with 
outstanding request: 

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/828664

Title:
  importer doesn't cope with broken ssh pipe

To manage notifications about this bug go to:
https://bugs.launchpad.net/udd/+bug/828664/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

[Bug 828664] Re: importer doesn't cope with broken ssh pipe

2011-08-24 Thread Martin Pool 
eglibc php5 and ubuntu-docs all had similar errors so I've requeued them
too

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/828664

Title:
  importer doesn't cope with broken ssh pipe

To manage notifications about this bug go to:
https://bugs.launchpad.net/udd/+bug/828664/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

[Bug 828664] Re: Jenkins package failed to import - TooManyConcurrentRequests

2011-08-24 Thread Martin Pool 
@James ubuntu:jenkins is now up to date (checked by branching it which
implicitly checks against publication) - thanks for the report and sorry
for the disruption.

I'm attaching the log which shows the failure is

136.161  Committed revid 
james.wes...@ubuntu.com-2011072018-fes7o7ahfcevg11p as revno 2.
137.614  Marking version 1.409.1-0ubuntu1 in oneiric at revid 
james.wes...@ubuntu.com-2011072018-fes7o7ahfcevg11p
141.324  refresh_possible_transports dropping stale transport 

 ([Errno 32] Broken pipe)
141.324  refresh_possible_transports encountered a smart transport with 
outstanding request: 
... and on and on.

So I think the bug is: launchpad dropped the connection (or maybe the
network glitched or something else.)  What the udd importer  should have
done is probably stop at that point and treat it as a transient failure.
What did happen is that it kept trying to use the dead ssh connection
and tied itself in knots - there is possibly also a bzr bug there that
it didn't try to reconnect.   (cf bug 819604 and friends).

So, though the immediate thing is fixed by retrying I'll leave this open
for that bug.

** Summary changed:

- Jenkins package failed to import - TooManyConcurrentRequests
+ importer doesn't cope with broken ssh pipe

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/828664

Title:
  importer doesn't cope with broken ssh pipe

To manage notifications about this bug go to:
https://bugs.launchpad.net/udd/+bug/828664/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

[Bug 626960] Re: Collection dictionary access incorrectly folds all HTTP errors to KeyError

2011-08-24 Thread Martin Pool 
i'm closing the udd task because nothing needs to change in udd itself;
but it would be really good to finish and deploy a new
lazr.restfulclient package.

** Changed in: udd
   Status: In Progress => Invalid

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/626960

Title:
  Collection dictionary access incorrectly folds all HTTP errors to
  KeyError

To manage notifications about this bug go to:
https://bugs.launchpad.net/lazr.restfulclient/+bug/626960/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

[Bug 831707] [NEW] file_mp fails: target_branch: Required input is missing.

2011-08-22 Thread Martin Pool 
Public bug reported:

udd is failing to file merge proposals, I think because of a further lp
api bug following on from bug 819910.

for example, libraw:

0.642  Time (UTC): 2011-08-23 00:17:53.545969
1.284  ssh implementation is OpenSSH
7.853  sleeping to file merge proposal
134.806  Traceback (most recent call last):
  File "/srv/package-import.canonical.com/new/scripts/import_package.py", line 
1000, in main
revid_db.cleanup_last_run(cleanup, push_back)
  File "/srv/package-import.canonical.com/new/scripts/udd/icommon.py", line 
1110, in cleanup_last_run
push_cb()
  File "/srv/package-import.canonical.com/new/scripts/import_package.py", line 
994, in push_back
possible_transports, local_branches)
  File "/srv/package-import.canonical.com/new/scripts/import_package.py", line 
734, in push_branches_back
bstore, lp_side_branch_path)
  File "/srv/package-import.canonical.com/new/scripts/import_package.py", line 
513, in file_mp
lp_side_branch_path),
  File "/srv/package-import.canonical.com/new/scripts/udd/lpapi.py", line 60, 
in lp_call
return callable(*args, **kwargs)
  File "/usr/lib/pymodules/python2.6/lazr/restfulclient/resource.py", line 527, 
in __call__
url, in_representation, http_method, extra_headers=extra_headers)
  File "/usr/lib/pymodules/python2.6/lazr/restfulclient/_browser.py", line 306, 
in _request
raise HTTPError(response, content)
HTTPError: HTTP Error 400: Bad Request
Response headers:
---
connection: close
content-length: 41
content-type: text/plain;charset=utf-8
date: Tue, 23 Aug 2011 00:20:07 GMT
server: zope.server.http (HTTP)
status: 400
vary: Accept,Accept-Encoding
via: 1.1 api.launchpad.net
x-lazr-notifications: []
x-powered-by: Zope (www.zope.org), Python (www.python.org)
---
Response body:
---
target_branch: Required input is missing.
---

** Affects: udd
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/831707

Title:
  file_mp fails: target_branch: Required input is missing.

To manage notifications about this bug go to:
https://bugs.launchpad.net/udd/+bug/831707/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

[Bug 831699] [NEW] no report/log of successful packages

2011-08-22 Thread Martin Pool 
Public bug reported:

It would be useful if the udd importer status page and logs also showed
packages that have reported successfully, as well as those that failed.

See also bug 752078.

** Affects: udd
 Importance: High
 Status: Confirmed

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/831699

Title:
  no report/log of successful packages

To manage notifications about this bug go to:
https://bugs.launchpad.net/udd/+bug/831699/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

[Bug 828664] Re: Jenkins package failed to import - TooManyConcurrentRequests

2011-08-18 Thread Martin Pool 
** Changed in: udd
 Assignee: (unassigned) => Martin Pool (mbp)

** Changed in: udd
   Status: Confirmed => In Progress

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/828664

Title:
  Jenkins package failed to import - TooManyConcurrentRequests

To manage notifications about this bug go to:
https://bugs.launchpad.net/udd/+bug/828664/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

[Bug 828664] Re: Jenkins package failed to import - TooManyConcurrentRequests

2011-08-18 Thread Martin Pool 
** Changed in: udd
   Status: New => Confirmed

** Changed in: udd
   Importance: Undecided => High

** Description changed:

  http://package-import.ubuntu.com/status/jenkins.html#2011-08-01
  18:15:35.769737
  
  Please could this package be imported as I need to make a couple of
  fixes.
  
  Cheers
  
  James
+ 
+ 
+ Failed at 2011-08-01 18:15:35.769737
+ 
+ Connection to bazaar.launchpad.net closed by remote host.
+ Traceback (most recent call last):
+   File "/srv/package-import.canonical.com/new/scripts/import_package.py", 
line 1124, in 
+ only_before=options.only_before))
+   File "/srv/package-import.canonical.com/new/scripts/import_package.py", 
line 1058, in main
+ push_branches_back(package, temp_dir, bstore, possible_transports, 
local_branches)
+   File "/srv/package-import.canonical.com/new/scripts/import_package.py", 
line 712, in push_branches_back
+ overwrite=overwrite)
+   File "/srv/package-import.canonical.com/new/scripts/import_package.py", 
line 388, in push_branch
+ dir_to = bzrdir.BzrDir.open_from_transport(to_transport)
+   File "/usr/lib/pymodules/python2.6/bzrlib/bzrdir.py", line 750, in 
open_from_transport
+ return format.open(transport, _found=True)
+   File "/usr/lib/pymodules/python2.6/bzrlib/bzrdir.py", line 1697, in open
+ return self._open(transport)
+   File "/usr/lib/pymodules/python2.6/bzrlib/bzrdir.py", line 2884, in _open
+ return remote.RemoteBzrDir(transport, self)
+   File "/usr/lib/pymodules/python2.6/bzrlib/remote.py", line 120, in __init__
+ self._probe_bzrdir()
+   File "/usr/lib/pymodules/python2.6/bzrlib/remote.py", line 132, in 
_probe_bzrdir
+ self._rpc_open_2_1(path)
+   File "/usr/lib/pymodules/python2.6/bzrlib/remote.py", line 139, in 
_rpc_open_2_1
+ response = self._call('BzrDir.open_2.1', path)
+   File "/usr/lib/pymodules/python2.6/bzrlib/remote.py", line 57, in _call
+ return self._client.call(method, *args)
+   File "/usr/lib/pymodules/python2.6/bzrlib/smart/client.py", line 132, in 
call
+ result, protocol = self.call_expecting_body(method, *args)
+   File "/usr/lib/pymodules/python2.6/bzrlib/smart/client.py", line 145, in 
call_expecting_body
+ method, args, expect_response_body=True)
+   File "/usr/lib/pymodules/python2.6/bzrlib/smart/client.py", line 79, in 
_call_and_read_response
+ readv_body=readv_body, body_stream=body_stream)
+   File "/usr/lib/pymodules/python2.6/bzrlib/smart/client.py", line 45, in 
_send_request
+ protocol_version)
+   File "/usr/lib/pymodules/python2.6/bzrlib/smart/client.py", line 115, in 
_construct_protocol
+ request = self._medium.get_request()
+   File "/usr/lib/pymodules/python2.6/bzrlib/smart/medium.py", line 710, in 
get_request
+ return SmartClientStreamMediumRequest(self)
+   File "/usr/lib/pymodules/python2.6/bzrlib/smart/medium.py", line 968, in 
__init__
+ raise errors.TooManyConcurrentRequests(self._medium)
+ bzrlib.errors.TooManyConcurrentRequests: The medium 
'SmartSSHClientMedium(bzr+ssh://package-imp...@bazaar.launchpad.net/)' has 
reached its concurrent request limit. Be sure to finish_writing and 
finish_reading on the currently open request.

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/828664

Title:
  Jenkins package failed to import - TooManyConcurrentRequests

To manage notifications about this bug go to:
https://bugs.launchpad.net/udd/+bug/828664/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

[Bug 626960] Re: Collection dictionary access incorrectly folds all HTTP errors to KeyError

2011-08-16 Thread Martin Pool 
** Changed in: lazr.restfulclient
Milestone: None => 0.11.3

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/626960

Title:
  Collection dictionary access incorrectly folds all HTTP errors to
  KeyError

To manage notifications about this bug go to:
https://bugs.launchpad.net/lazr.restfulclient/+bug/626960/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

[Bug 626960] Re: Collection dictionary access incorrectly folds all HTTP errors to KeyError

2011-08-16 Thread Martin Pool 
this is merged to trunk of restfulclient; thanks for the speedy review
gmb.

I think this doesn't need any specific changes in udd, and it would not
be worth adding a workaround.  We should make a release of restfulclient
and then get that backported and deployed.

** Changed in: lazr.restfulclient
   Status: In Progress => Fix Committed

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/626960

Title:
  Collection dictionary access incorrectly folds all HTTP errors to
  KeyError

To manage notifications about this bug go to:
https://bugs.launchpad.net/lazr.restfulclient/+bug/626960/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

[Bug 827263] Re: KeyError: 'debian' in get_debian_versions

2011-08-16 Thread Martin Pool 
*** This bug is a duplicate of bug 626960 ***
https://bugs.launchpad.net/bugs/626960

This is bug 626960 and probably caused by a transient error, so retrying
exult ought to fix it.

** This bug has been marked a duplicate of bug 626960
   Collection dictionary access incorrectly folds all HTTP errors to KeyError

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/827263

Title:
  KeyError: 'debian' in get_debian_versions

To manage notifications about this bug go to:
https://bugs.launchpad.net/udd/+bug/827263/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

[Bug 827263] Re: KeyError: 'debian' in get_debian_versions

2011-08-16 Thread Martin Pool 
*** This bug is a duplicate of bug 626960 ***
https://bugs.launchpad.net/bugs/626960

On my local machine when debugging this I observed that some requests
failed with "Expired token", which is commented as not being handled by
get_lp.  I'm not sure if that's actually related to this bug though.

>From lp-shell, I can get this item, so I wonder if this is actually
buggy error handling in restfulclient:

>>> lp.distributions['debian']
https://api.launchpad.net/1.0/debian>

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/827263

Title:
  KeyError: 'debian' in get_debian_versions

To manage notifications about this bug go to:
https://bugs.launchpad.net/udd/+bug/827263/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

[Bug 626960] Re: Collection dictionary access incorrectly folds all HTTP errors to KeyError

2011-08-16 Thread Martin Pool 
This probably accounts for exult, and possibly other things that failed
in a restfulclient __getentry__.  Requeuing them will possibly fix it
(if the error was a transient error.)

** Changed in: lazr.restfulclient
 Assignee: (unassigned) => Martin Pool (mbp)

** Changed in: lazr.restfulclient
   Importance: Medium => High

** Changed in: lazr.restfulclient
   Status: Triaged => In Progress

** Also affects: udd
   Importance: Undecided
   Status: New

** Changed in: udd
   Status: New => In Progress

** Changed in: udd
   Importance: Undecided => High

** Changed in: udd
 Assignee: (unassigned) => Martin Pool (mbp)

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/626960

Title:
  Collection dictionary access incorrectly folds all HTTP errors to
  KeyError

To manage notifications about this bug go to:
https://bugs.launchpad.net/lazr.restfulclient/+bug/626960/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

[Bug 827263] [NEW] KeyError: 'debian' in get_debian_versions

2011-08-16 Thread Martin Pool 
Public bug reported:

exult and some other packages fail like this:

Traceback (most recent call last):
  File "./import_package.py", line 1124, in 
only_before=options.only_before))
  File "./import_package.py", line 1014, in main
versions = get_versions(package, extra_debian=extra_debian)
  File "./import_package.py", line 220, in get_versions
vlist = get_debian_versions(package, extra_debian=extra_debian)
  File "./import_package.py", line 152, in get_debian_versions
d_archive = lp.distributions['debian'].main_archive
  File "/usr/lib/python2.7/dist-packages/lazr/restfulclient/resource.py", line 
950, in __getitem__
raise KeyError(key)
KeyError: 'debian'

http://package-import.ubuntu.com/status/exult.html

I can also reproduce this locally trying to import ntrack.

** Affects: udd
 Importance: High
 Status: Confirmed

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/827263

Title:
  KeyError: 'debian' in get_debian_versions

To manage notifications about this bug go to:
https://bugs.launchpad.net/udd/+bug/827263/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

[Bug 827074] [NEW] could use python-oops to record errors

2011-08-15 Thread Martin Pool 
Public bug reported:

It might be useful if the udd importer unified its error tracking code
with https://launchpad.net/python-oops; that would be a good foundation
for capturing more information (like local variables or a timeline.)

12:40
lifeless: poolie: you should be able to configure and publish an oops from that 
system now, with lp:python-oops and lp:python-oops-datedir-repo
lifeless: config = oops.Config()
lifeless: repo = oops_datedir_repo.DateDirRepo('/where/the/oopses/go', 
'packageimporter')
poolie: nice
lifeless: report = config.create()
lifeless: # flesh it out
lifeless: config.publish(report)
lifeless: # bah I forgot a step - 
lifeless: config.publishers.append(repo.publish)

** Affects: udd
 Importance: Medium
 Status: Confirmed

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/827074

Title:
  could use python-oops to record errors

To manage notifications about this bug go to:
https://bugs.launchpad.net/udd/+bug/827074/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

[Bug 819910] Re: many packages fail to import due to HTTPError talking to Launchpad (eg ubuntu:transmission)

2011-08-11 Thread Martin Pool 
... and that was basically fallout from
http://bazaar.launchpad.net/+branch/launchpadlib/revision/82 which took
that component out of the predefined urls.

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/819910

Title:
  many packages fail to import due to HTTPError talking to Launchpad (eg
  ubuntu:transmission)

To manage notifications about this bug go to:
https://bugs.launchpad.net/udd/+bug/819910/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

[Bug 819910] Re: many packages fail to import due to HTTPError talking to Launchpad (eg ubuntu:transmission)

2011-08-11 Thread Martin Pool 
and I think the specific problem is that file_mp is making up a url by
string manipulation, and it's omitting the /1.0 component.

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/819910

Title:
  many packages fail to import due to HTTPError talking to Launchpad (eg
  ubuntu:transmission)

To manage notifications about this bug go to:
https://bugs.launchpad.net/udd/+bug/819910/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

[Bug 819910] Re: many packages fail to import due to HTTPError talking to Launchpad (eg ubuntu:transmission)

2011-08-11 Thread Martin Pool 
With some code inserted to log about the failure, it seems the importer is 
trying to read 

 and the problem is that it's missing the api version component (eg '/beta')

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/819910

Title:
  many packages fail to import due to HTTPError talking to Launchpad (eg
  ubuntu:transmission)

To manage notifications about this bug go to:
https://bugs.launchpad.net/udd/+bug/819910/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

[Bug 819910] Re: many packages fail to import due to HTTPError talking to Launchpad (eg ubuntu:transmission)

2011-08-10 Thread Martin Pool 
I rolled out my attempted fix, but that's apparently not fixed it:


2011-08-11 06:44:57,918 - __main__ - WARNING - Importing transmission failed:
Object: https://api.launchpad.net>, name: ''
Traceback (most recent call last):
  File "/srv/package-import.canonical.com/new/scripts/import_package.py", line 
1124, in 
only_before=options.only_before))
  File "/srv/package-import.canonical.com/new/scripts/import_package.py", line 
1000, in main
revid_db.cleanup_last_run(cleanup, push_back)
  File "/srv/package-import.canonical.com/new/scripts/udd/icommon.py", line 
1107, in cleanup_last_run
push_cb()
  File "/srv/package-import.canonical.com/new/scripts/import_package.py", line 
994, in push_back
possible_transports, local_branches)
  File "/srv/package-import.canonical.com/new/scripts/import_package.py", line 
734, in push_branches_back
bstore, lp_side_branch_path)
  File "/srv/package-import.canonical.com/new/scripts/import_package.py", line 
504, in file_mp
lp_branch = lpapi.lp_call(lp.load, lp_url)
  File "/srv/package-import.canonical.com/new/scripts/udd/lpapi.py", line 60, 
in lp_call
return callable(*args, **kwargs)
  File "/usr/lib/pymodules/python2.6/lazr/restfulclient/resource.py", line 459, 
in load
document = self._browser.get(url)
  File "/usr/lib/pymodules/python2.6/lazr/restfulclient/_browser.py", line 316, 
in get
response, content = self._request(url, extra_headers=headers)
  File "/usr/lib/pymodules/python2.6/lazr/restfulclient/_browser.py", line 306, 
in _request
raise HTTPError(response, content)
lazr.restfulclient.errors.HTTPError: HTTP Error 404: Not Found
Response headers:

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/819910

Title:
  many packages fail to import due to HTTPError talking to Launchpad (eg
  ubuntu:transmission)

To manage notifications about this bug go to:
https://bugs.launchpad.net/udd/+bug/819910/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

[Bug 819910] Re: many packages fail to import due to HTTPError talking to Launchpad (eg ubuntu:transmission)

2011-08-10 Thread Martin Pool 
** Branch linked: lp:~mbp/udd/819910-service-root

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/819910

Title:
  many packages fail to import due to HTTPError talking to Launchpad (eg
  ubuntu:transmission)

To manage notifications about this bug go to:
https://bugs.launchpad.net/udd/+bug/819910/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

[Bug 819910] Re: many packages fail to import due to HTTPError talking to Launchpad (eg ubuntu:transmission)

2011-08-10 Thread Martin Pool 
** Summary changed:

- VCS for ubuntu:transmission failed to auto-import
+ many packages fail to import due to HTTPError talking to Launchpad (eg 
ubuntu:transmission)

** Changed in: udd
 Assignee: (unassigned) => Martin Pool (mbp)

** Changed in: udd
   Status: Confirmed => In Progress

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/819910

Title:
  many packages fail to import due to HTTPError talking to Launchpad (eg
  ubuntu:transmission)

To manage notifications about this bug go to:
https://bugs.launchpad.net/udd/+bug/819910/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

[Bug 819910] Re: VCS for ubuntu:transmission failed to auto-import

2011-08-10 Thread Martin Pool 
Thanks for analyzing that James, it sounds like it's the same as
.

This probably broke recently when we upgraded jubany to lucid.

Max put a workaround for this into bzr in
 and perhaps the easiest thing is just to get the
correct version from bzr.

** Changed in: udd
   Status: New => Confirmed

** Changed in: udd
   Importance: Undecided => High

** Changed in: udd
   Importance: High => Critical

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/819910

Title:
  VCS for ubuntu:transmission failed to auto-import

To manage notifications about this bug go to:
https://bugs.launchpad.net/udd/+bug/819910/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

[Bug 806348] Re: BzrCheckError: Internal check failed: Cannot add revision(s) to repository: missing chk node(s) for id_to_entry maps

2011-08-09 Thread Martin Pool 
Moving jubany back to bzr 2.3 did seem to avoid these problems, so
fixing bug 771184 will probably largely fix this.

Not having versioned tags makes it very hard to get rid of tags;
therefore very hard to get rid of tags that point at irrelevant history;
so that keeps floating around.

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/806348

Title:
  BzrCheckError: Internal check failed: Cannot add revision(s) to
  repository: missing chk node(s) for id_to_entry maps

To manage notifications about this bug go to:
https://bugs.launchpad.net/bzr/+bug/806348/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

[Bug 524173] Re: package-import uses james_w credentials

2011-07-28 Thread Martin Pool 
I _think_ this is now done; if not please reopen again.

** Changed in: udd
   Status: Confirmed => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/524173

Title:
  package-import uses james_w credentials

To manage notifications about this bug go to:
https://bugs.launchpad.net/udd/+bug/524173/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

[Bug 609187] Re: users are not warned when branching ubuntu:foo (or lp:ubuntu/foo) and the package import of foo is out of date

2011-07-27 Thread Martin Pool 
after discussion in the udd meeting, i think we should backport this to
lucid at least (2.1) and it will be reasonable to put into srus

-- 
You received this bug notification because you are a member of Ubuntu
Distributed Development Developers, which is subscribed to Ubuntu
Distributed Development.
https://bugs.launchpad.net/bugs/609187

Title:
  users are not warned when branching ubuntu:foo (or lp:ubuntu/foo) and
  the package import of foo is out of date

To manage notifications about this bug go to:
https://bugs.launchpad.net/bzr/+bug/609187/+subscriptions

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

Re: Re-aligning the Ubuntu Developer Process

2011-07-12 Thread Martin Pool 
On 11 July 2011 03:14, Michael Bienia  wrote:
> On 2011-07-09 08:10:30 +1000, Martin Pool wrote:
>> On 9 July 2011 03:44, Michael Bienia  wrote:

> I guess this is because LP makes it easier to track development related
> contributions through sponsored uploads (just click on "Related software
> and packages" on the LP profile) than through (successfully) contributed
> patches or branches.

You can look at say https://code.launchpad.net/~mbp/+merges to see
merges someone has done in the past.  I hope lp will add a full
timeline view at some point, but that probably won't happen soon.  If
there are things that Launchpad could do to help Ubuntu, please file
bugs.

> I acknowledge that we failed repeatedly to reach quorum in the past.
> One problem is that it's hard to find a suitable meeting time for 7
> members located around the world (without considering the applicants).
>
> In the past the MC has tried to process applications by mail but that
> didn't work out too well either (so the MC switched to the current
> processing in meetings). Processing an application by mail took very
> long: first there was some time for questions which can take up to two
> weeks (or even longer depending if a discussion is going on). Then the
> voting period of one week started. In practice only a few members voted
> in that week, so the remaining members got pinged again for the vote.
> After two weeks you might have enough votes for a qurom (do you wait
> longer for the still open votes or not?). So in total processing an
> application took one month or even longer without the applicant knowing
> whats going on.
> IRC meetings have the benefit of realtime communications with the
> applicant and (in most cases) a result of the voting (of course only if
> the quorum is reached).
>
> If someone has a good solution how to improve the voting process, please
> let us know.

Perhaps it would be good to identify what information is gained by the
meeting and the voting.  If the board almost always votes in line with
the endorsements, it's not necessarily worth having a delay and a
meeting.

One possible issue with relying mostly on endorsements is that people
probably self-censor concerns or negative experiences from the wiki
page; perhaps they raise them in private with the membership board,
and there needs to be a space for this to happen?  On the other hand,
if the person really is a jerk and persists with this after they get
access, it is always possible to censure them or in the worst case
withdraw access.

> So I wouldn't object to focus the template more towards testimonials.
> I even prefer to have many testimonials instead of reviewing some
> contributions myself to judge the quality.

+1

> I see this problem (few testimonials) mostly for specialist applications
> (PPU or package sets) for "uncommon" areas without any interested
> developers in that area that could add testimonials. For well
> established developer "types" like core-dev or MOTU, I don't see that
> problem.
>
> Here is a example (not very representive):
> - https://wiki.ubuntu.com/MarcCluet/UbuntuContributingDeveloper
>  for membership; 4 endorsements
> - https://wiki.ubuntu.com/JamesPage/ServerDeveloperApplication
>  for server-dev package set; 6 endorsements
> - https://wiki.ubuntu.com/ChrisHalseRogers/CoreDevApplication
>  for core-dev; 5 endorsements
> compare that to
> - https://wiki.ubuntu.com/MarcinJuszkiewicz/DeveloperApplication-PPU
>  gcc-*-cross PPU; 2 endorsements

Still, those are fairly positive comments from two very well respected
people, and to me the word of Loïc and Steve would be conclusive to at
least give him a try.  We need to distinguish:

 * he could get more testimonials but because they're not the centre
of this process people don't realize it helps to get more
 * he doesn't know that many people in Ubuntu yet but is trusted by
those who do know him
 * he hasn't done enough work to judge (doesn't seem true if you
include work outside Ubuntu)
 * (purely hypothetically) he knows people but they don't like or trust him
 * ..?

Like in the patch pilot process I think it's really good to make a
decision and not leave people hanging.  For instance we could say "ok,
access granted, but please have someone mentor/review your work
first", or "please get one more endorsement", or whatever.

m

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

Re: Re-aligning the Ubuntu Developer Process

2011-07-08 Thread Martin Pool 
On 9 July 2011 03:44, Michael Bienia  wrote:
> On 2011-07-05 11:30:09 -0700, Jono Bacon wrote:
> Hello,
>
> first it would be good if this mail also went to the DMB even if it is
> just to collect initial input from the TB. Otherwise it looks like the
> DMB will only get notified about the outcome/decision without any real
> chance to comment.
>
>> I want to propose a few changes to our developer process. Recently there
>> has been concerns expressed that the process is difficult to get
>> through, even for people with extensive knowledge.
>
> Can you be more specific which "requirements" are seen to difficult?
> But I agree that some parts could be improved but I don't any ideas that
> would work out.

I can give my impression, from going through a PPU application myself
last year, and from seeing other people go through membership
applications.

 * There is [or was] a lot of emphasis on looking for contributions
specifically through sponsored uploads to Ubuntu.  People can make
developer-type contributions to Ubuntu in other ways, such as
attaching patches or branches to bugs that are then uploaded by
others.  Even if this isn't actually a requirement, the typical
process of counting uploads makes it look like it is.  Like Jono says,
"good people getting rejected because their body of work was not in
the specific form that a DMB member expects"

 * There seemed to be a thing that just having done extensive work on
one large area of technology was not enough, and you needed to
demonstrate work outside of this.  This is a bit artificial: most open
source developers specialize.

 * You have to attend an irc meeting to be grilled, possibly at an
inconvenient time, and that meeting in my case repeatedly failed to
meet quorum or was rescheduled.  The requirement for the meeting and
the kind of questions that would be asked were not very clear in the
documentation at the start of the process.  It seems a bit like
jumping through hoops just to prove you want access, when the point
should be for us to prove we want developers.

 * Going back to list specific bug numbers you fixed seems a bit
mechanical when you could instead have other experienced developers
testify you've done good work.

 * It ends up with "well, do all this other work of a type or on a
subject you don't really care about so that maybe we'll let you work
on the problems you do care about," and then there's no clear
commitment ticking those boxes will actually succeed.  Why?  I don't
insist people fix bzr gui bugs before I let them commit fixes for
unicode bugs.

I know people are working on this and some of it will have shifted
since last year, which is good.

So, I can see why people would say "why bother?" and either just keep
contributing as a non-member, or spend their time elsewhere.

Personally, I give access by considering

1. do they have a record of technical achievement?
2. will they use their powers responsibly?
3. and in particular are they mature enough to know what they don't
know and to ask for advice rather than rushing in?
4. can they work well with others?

>> Today I feel our process is too focused on showcasing work as opposed to
>> harnessing reputation - I have always felt that a +1 from a respected
>> community member holds more weight than a list of bugs fixed on a wiki
>> page.  I believe that adjusting our process to focus on reputation will
>> streamline it and reduce the risk of good people getting rejected
>> because their body of work was not in the specific form that a DMB
>> member expects.

+1

> I'd be willing to give good testimonials a heigher weight if there were
> more testimonials in the applications. And a lack of comments or
> testimonials can be seen as a bad integration into the development
> community.

Perhaps this is partly because the process (as it was late last year)
seems more interested in data than in testimonials?  If we simplified
it to say that they main thing is to get testimonials from 3+ people
already well respected in Ubuntu, people would probably do it.

>> With this in mind I would like to propose two modifications to the
>> current assess process for each of these areas:
>>
>>      1. Core-Dev Testimonials - for each candidate they should gather
>>         testimonials from core-devs. These testimonials should account
>>         for the vast majority of the assessment. I believe that if
>>         someone gets two core-dev +1s for approval, there should be a
>>         good reason if the DMB wants to reject the candidate.
>>      2. Brief Core-Devs - we should make it clear that getting a +1 from
>>         a core-dev is key part of the assessment, and core-devs should
>>         expect to provide honest and fair testimonials as part of they
>>         work as a core-dev. We could even provide some kind of
>>         application queue for those requesting testimonials.
>
> Doesn't this move the problem towards core-dev?
> Some applicants have a nice long list of testimonals which ma

Re: access for maxb to the udd package importer (jubany)

2011-07-06 Thread Martin Pool 
On 28 June 2011 23:14, Martin Pool  wrote:

> I have discussed this with elmo and cjwatson and they think this would
> be reasonable if the TB assents.  If you don't object, I will ask
> Canonical IS to create an account for maxb.

Since nobody did object, I'm going to proceed with this.

Martin

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

access for maxb to the udd package importer (jubany)

2011-06-28 Thread Martin Pool 
Max Bowsher  has been doing huge amounts
of great work on improving the UDD package importer, of which
 is one typical example.  This is the machine
that pulls in packages from Debian and Ubuntu and does a two-way synch
with the Ubuntu package branches.

After fixing bugs or operational problems, we typically need to update
the instance of lp:udd on the server, and then restart some imports.
(See again that bug for an example.)

At the moment maxb needs to block on a Canonical staff member to
basically be an intelligent sudo to requeue packages and to get
debugging information.  I think it would be better if we can let Max
directly log in to this machine and run these commands.

Security implications: This will give him control over a a token that
can write to Ubuntu core branches and to access Launchpad with similar
credentials to ~core-devs.  This machine can only access Launchpad
through the rest api and bzr+ssh, and it's outside the main Canonical
firewall.

I have discussed this with elmo and cjwatson and they think this would
be reasonable if the TB assents.  If you don't object, I will ask
Canonical IS to create an account for maxb.

Martin

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

Re: tb approval for package importer robot account (LP: #524173)

2011-06-08 Thread Martin Pool 
On 9 June 2011 15:30, Martin Pitt  wrote:
> Hello Martin,
>
> Martin Pool [2011-06-09 14:15 +1000]:
>>  * having the tb add this to the list of permitted uploaders for
>> /ubuntu (but not into ~core-dev)
>
> What does "/ubuntu" mean here? I. e. where should we add this bot?

Hi Martin,

I meant <https://launchpad.net/ubuntu/> and specifically the
'Uploaders' section of that page.  I am told there is an api client
script which can add people into it, when run by a member of
~tech-board.  If the tech board agrees with the proposal but doesn't
know about that script I will find or write one.

> Also, what's the role of https://launchpad.net/~ubuntu-branches here?
> I thought that was the team that owns all the auto-imports?

It was, but it will be going away within the next few days.  There was
a tech-board thread about 10 days ago where Francis discusses removing
them: it helps adding support for Ensemble formulas; it's crufty
inside Launchpad; and because Launchpad is trying to get away from
having hardcoded permissions and celebrities.

<https://lists.ubuntu.com/archives/technical-board/2011-May/000907.html>

>> Some reasonable concerns have been raised that this does not get as
>> much to a least-privilege setup as one could desire.   In particular:
>> the new account will be able to upload packages as well as write to
>> branches: Launchpad does not have separate ACLs for those actions at
>> present.
>
> It should be easy to ascertain that the bot doesn't actually do any
> dput or ftp'ing, so I'm not too concerned about this as long as it
> runs in a trusted environment in the DC.
>
> At some point we plan to do package builds from branches, so it seems
> to me that this separation will become smaller or nonexistent in the
> future.

Right.

> Once that works, how can we ensure that the bot doesn't
> "accidentally" create a branch which will cause a package build?

That's an interesting question.  Normally it should only be creating
revisions for already-published packages.  As an implementation detail
we could want some kind of safeguards to make sure it doesn't get into
a feedback loop of causing existing packages to be rebuild in either
the same archive or a different one.  One approach would be to
specifically blacklist revisions that come from this robot (for which
giving it its own identity can of course only help).  There might be
others.  Or, it may be that just the check that the package version
has not already been published and is targeted to the right pocket is
enough.

>> On both of these I think it's worth acknowledging that more should be
>> done in the future, but also that making the importer use its own
>> account and identity will be a step forward for security and not a
>> step back.
>
> I agree. James already has way more privileges, so it's not a
> regression.

Thanks.

Martin

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

tb approval for package importer robot account (LP: #524173)

2011-06-08 Thread Martin Pool 
There is a robot in  that
synchronizes source packages into Ubuntu package branches.

At the moment, this runs in james_w's personal account, since that's
how he originally set it up.  This is suboptimal for a bunch of
reasons, amongst which are that the imported revisions appear to have
been personally created by James; that it has credentials to
impersonate James in other ways (eg tweaking bugs); and that it will
break if for any reason James ceases involvement with Ubuntu.

This is .

I'd like to fix this now by:

 * creating a new bot account with an email address called say
package-imp...@canonical.com, with mail to that account going to (say)
canonical-bazaar
 * creating an API key and SSH key for that account
 * switching the package import to use that account
 * having the tb add this to the list of permitted uploaders for
/ubuntu (but not into ~core-dev)

Some reasonable concerns have been raised that this does not get as
much to a least-privilege setup as one could desire.   In particular:
the new account will be able to upload packages as well as write to
branches: Launchpad does not have separate ACLs for those actions at
present.  Secondly, the service is not yet fully LOSA maintained,
though there is a ticket asking for it to become so (canonical rt
39614).

On both of these I think it's worth acknowledging that more should be
done in the future, but also that making the importer use its own
account and identity will be a step forward for security and not a
step back.  It won't gain any extra privileges as a result of those
changes, it will lose access to other things James can do, and it will
be easier to track.

If you see any problems or object to this, please let me know.

Martin

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

Re: bzr maverick srus now qa'd

2011-05-18 Thread Martin Pool 
Thanks.  I've added text to <
https://wiki.ubuntu.com/StableReleaseUpdates/MicroReleaseExceptions<https://wiki.ubuntu.com/StableReleaseUpdates/MicroReleaseExceptions#preview>>
on the grounds it is already the intended policy.

Martin



On 18 May 2011 15:36, Mark Shuttleworth  wrote:

>
> +1 from me.
>
> On 18/05/11 13:26, Martin Pool wrote:
> > Elsewhere, on 26 April 2011 08:13, Martin Pitt 
> wrote:
> >> Hello Martin,
> >>
> >> Martin Pool [2011-04-21 18:48 +1000]:
> >>> On the whole I'm not sure [manually verifying all SRU bugs in bzr] was
> a good use of time: I think we tend
> >>> to have problems not so much when we fail to fix the bug but rather
> >>> when we break something else in doing so.  Manually testing the bug is
> >>> probably not going to catch that, and is probably redundant with
> >>> testing we did and the original reporter did when it was merged
> >>> upstream.
> >> I agree. Indeed I'm much more concerned about regression testing,
> >> which is of course hard to describe in general for all SRUs. So we
> >> usually resort to minimal patches and have reporters test the actual
> >> package in a real environment. Strictly speaking this is not true
> >> regression testing, but the next best thing to what we can reasonably
> >> achieve.
> >>
> >> In the bzr case however, we can do proper regression testing, because
> >> it already has a huge test suite. Indeed the MRE says:
> >>
> >>  conditions: test suite running during package build from Ubuntu
> >>  11.04 on; SRU verification should run test suite in installed sytem
> >>
> >> From my POV doing the latter and then reporting back to one of the
> >> bugs with "test suite showed no regressions for the package in
> >> foo-proposed" would suffice here. I think that was the original intent
> >> when we discussed the MRE.
> >>
> >> We handle things in a similar way for other MREs like postgresql or
> >> Firefox: We run standard tests (manual or automatic suites) for
> >> signing them off.
> > I would like to ask the tech board to approve an edit
> > <https://wiki.ubuntu.com/StableReleaseUpdates/MicroReleaseExceptions>
> > to say something along these lines, in the interests of people getting
> > changes without wasted effort or inconsistent handling.
> >
> > Thanks
> > Martin
> >
>
>
-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

Re: bzr maverick srus now qa'd

2011-05-18 Thread Martin Pool 
Elsewhere, on 26 April 2011 08:13, Martin Pitt  wrote:
> Hello Martin,
>
> Martin Pool [2011-04-21 18:48 +1000]:
>> On the whole I'm not sure [manually verifying all SRU bugs in bzr] was a 
>> good use of time: I think we tend
>> to have problems not so much when we fail to fix the bug but rather
>> when we break something else in doing so.  Manually testing the bug is
>> probably not going to catch that, and is probably redundant with
>> testing we did and the original reporter did when it was merged
>> upstream.
>
> I agree. Indeed I'm much more concerned about regression testing,
> which is of course hard to describe in general for all SRUs. So we
> usually resort to minimal patches and have reporters test the actual
> package in a real environment. Strictly speaking this is not true
> regression testing, but the next best thing to what we can reasonably
> achieve.
>
> In the bzr case however, we can do proper regression testing, because
> it already has a huge test suite. Indeed the MRE says:
>
>  conditions: test suite running during package build from Ubuntu
>  11.04 on; SRU verification should run test suite in installed sytem
>
> From my POV doing the latter and then reporting back to one of the
> bugs with "test suite showed no regressions for the package in
> foo-proposed" would suffice here. I think that was the original intent
> when we discussed the MRE.
>
> We handle things in a similar way for other MREs like postgresql or
> Firefox: We run standard tests (manual or automatic suites) for
> signing them off.

I would like to ask the tech board to approve an edit
<https://wiki.ubuntu.com/StableReleaseUpdates/MicroReleaseExceptions>
to say something along these lines, in the interests of people getting
changes without wasted effort or inconsistent handling.

Thanks
Martin

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

Re: micro release exception for bzr

2010-11-29 Thread Martin Pool 
(Let me know if I should take this to a more specific forum than the TB.)

On 22 September 2010 13:09, Martin Pool  wrote:
> On 22 September 2010 03:05, Martin Pitt  wrote:
>> Hello Martin,
>>
>> Martin Pool [2010-09-20 12:08 +1000]:
>>> Thanks, everyone.  Since there do not seem to be any objections, would
>>> one of you please add bzr to
>>> <https://wiki.ubuntu.com/StableReleaseUpdates/MicroReleaseExceptions>?
>>
>> The request was approved by the TB today, so I added it.
>>
>> The permission is coupled to those conditions:
>>
>>  * The test suite gets re-enabled during package builds.
>>  * SRU verification should include tunning the tests in an installed system.

We have just done our second bugfix release for the 2.2 series, which
shipped in Maverick[0], and requested an SRU for it as part of
<http://pad.lv/659590>.

The good news, with regard to our SRU MRE conditions, is that the test
suite now passes cleanly when run both during a build, and when run
from an installed package.  We've checked this for the specific source
packaged in 2.2.2.

However, the bad news is that we can't really meet this MRE condition
for Maverick.  To run its test suite, bzr needs the "python-testtools"
package[1], which is packaged in Universe while bzr itself is in
main[2].  This hasn't been a problem before because every other
command works without it, and attempting to run the tests just asks
you to install testtools.  I personally can't see any way to unlock
this: I presume we can't promote python-testtools into main in an SRU.

It would be pretty disappointing to stall bzr SRUs after what we've
done towards enabling them.

It seems to me the best course is:

 * grandfather the 2.2 series SRUs by running their tests in a
separate build, and manually after installation of the package
 * for Natty, pursue a main inclusion request
<https://wiki.ubuntu.com/MainInclusionProcess> for python-testtools

I'd very much appreciate either an amended MRE exception to allow us
to do this for 2.2, or an alternative suggestion.

[0] <https://launchpad.net/bzr/2.2/2.2.2>
[1] <http://mumak.net/testtools/docs/>
[2] https://lists.ubuntu.com/archives/bazaar/2010q4/071019.html

-- 
Martin

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

Re: micro release exception for bzr

2010-09-30 Thread Martin Pool 
On 22 September 2010 03:05, Martin Pitt  wrote:
> Hello Martin,
>
> Martin Pool [2010-09-20 12:08 +1000]:
>> Thanks, everyone.  Since there do not seem to be any objections, would
>> one of you please add bzr to
>> <https://wiki.ubuntu.com/StableReleaseUpdates/MicroReleaseExceptions>?
>
> The request was approved by the TB today, so I added it.
>
> The permission is coupled to those conditions:
>
>  * The test suite gets re-enabled during package builds.
>  * SRU verification should include tunning the tests in an installed system.
>
> For the latter, can you please give us some instructions how to do
> that? Is it enough to run "bzr selftest" or should this also add some
> options for plugins, etc.? It should be noted that this currently
> fails, so for lucid/maverick SRUs we should probably just compare the
> test result output.

'bzr selftest' should be enough.  If you have plugins installed other
than from the bzr package, you might 'bzr --no-plugins selftest'.  If
it ever fails, please file a bug against upstream bzr.  I just added
<https://bugs.edge.launchpad.net/bzr/+bug/644855> which is probably
pretty shallow: a check that we're not touching things out of the
source tree is wrong if we actually are specifically trying to test
the installed copy.

> However, it would be nice if the test would fully succeed. I assume
> you are developing bzr on Ubuntu, do you have any test failures
> locally?

In short, no.  There are some lurking very-intermittent
probably-gc-related failures, but keeping the suite clean is very
important to us.

> That'd be nice. It would also be great if we could enable it for
> lucid/maverick SRUs as well? Is it easy/possible to suppress tests
> which can't succeed in the buildd environment, such as the ones
> involving the network?

We should only need networking to localhost for testing.  Any test
that depends on something external that might not be present (eg a
library or a non-ascii locale) ought to cleanly skip if it's not
present.  If not, it's a bug, and again please report it.

-- 
Martin

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

Re: micro release exception for bzr

2010-09-21 Thread Martin Pool 
Thanks, everyone.  Since there do not seem to be any objections, would
one of you please add bzr to
?

I infer that even after we're added, we should keep nominating
figurehead bugs to kick off each individual SRU?

> IMO, we should generally be running test suites by default on package
> uploads unless there's a very good reason not to; this is even part of
> our main inclusion guidelines.
>
> The bzr test suite is perhaps larger than most, taking 47 minutes on my
> fairly heavily loaded laptop (so I'd expect it to be faster on most of
> the buildds, with the exception of armel).  But bzr is only uploaded to
> Ubuntu on the general order of once a month; this isn't a serious
> imposition on the buildds, and it would give us assurance we wouldn't
> get from manual runs: for instance, we're unlikely to get manual runs on
> all architectures.
>
> I'm not saying we have to run it on the nightly PPA builds; that would
> probably be asking too much.  We could also turn it off by default on
> particular architectures if it became a serious problem there, although
> I think that's something we should think hard about.
>
> The test suite was originally turned off in the package build not
> because it was taking too long, but:
>
>  * Flip DEB_BUILD_OPTIONS logic on the testsuite until someone figures out
>    why it intermittently hangs under fakeroot.
>
> The changelog entry dates from January 2007, and I don't know whether
> that has been resolved since then.

I think it's worth giving it a go, because we have fixed a good number
of timing or environment-dependency test suite bugs.  Perhaps I should
propose a change in Natty that turns it back on?

-- 
Martin

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board

Re: micro release exception for bzr

2010-09-21 Thread Martin Pool 
On 20 September 2010 12:08, Martin Pool  wrote:
>>  * Flip DEB_BUILD_OPTIONS logic on the testsuite until someone figures out
>>    why it intermittently hangs under fakeroot.
>>
>> The changelog entry dates from January 2007, and I don't know whether
>> that has been resolved since then.
>
> I think it's worth giving it a go, because we have fixed a good number
> of timing or environment-dependency test suite bugs.  Perhaps I should
> propose a change in Natty that turns it back on?

(for the benefit of the tape)
<https://bugs.launchpad.net/ubuntu/+source/bzr/+bug/644015>

-- 
Martin

-- 
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board