Re: Updates to the UPG for UDD

2012-01-31 Thread Martin Pool
On 1 February 2012 10:18, Barry Warsaw ba...@ubuntu.com wrote:
 Hi all,

 I have a merge proposal to update the UDD documentation part of the packaging
 guide.  I'm hoping this can be reviewed, merged, and published before my UDW
 session tomorrow (it's okay if it can't).

That was great, thankyou.

I think messages like yours might as well be sent to say
ubuntu-devel-discuss (too).


 The branch fixes some typos and formatting issues, but there are really two
 substantive changes.

 First, I've updated all the discussion of using `bzr merge-package` to just
 use `bzr merge` as is now the case in Precise.  I do give minimum bzr and
 bzr-builddeb version numbers for this, but don't dwell on the old way too
 much.

Thanks.


 Second, I've removed all the discussion of using looms to generate quilt
 patches.  In going through the described procedures, I realized they just
 don't work very well, and it makes things much more complicated.
 Specifically, the problem comes down to when you are in the thread that
 contains both the in-source change and the quilt patch, there is no way to
 `quilt push` the patch, because the source tree already has the changes.

 This is a problem in the non-loom approach (which I describe better now, and
 have tested), but it's easier because you can just shelve or revert the
 in-source change once you've imported the quilt patch.  I've actually been
 using this fairly successfully for a while now.  While I still like the idea
 of using looms instead of quilt, and am a little sad to let go of this, for
 now, I think it's clearer to avoid the whole thing.

 So, I hope these changes simplify the UDD workflow.  I've run through and
 tested them with various packages so I'm pretty sure the instructions are
 accurate.  Reviews are welcome of course!

I think this is a sensible choice for the understandability of the document.


 https://code.launchpad.net/~barry/ubuntu-packaging-guide/udd-update/+merge/90976

+1.

-- 
Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


xz tar imports failing?

2012-01-31 Thread Martin Pool
I haven't dug into whether these are new, or whether there is already
a bug for them, but they do seem to be new.  I see
https://bugs.launchpad.net/udd/+bug/553668 about supporting xz in
bzr-builddeb: does that perhaps need to be deployed on Jubany?


-- Forwarded message --
From: Cron Daemon r...@jubany.canonical.com
Date: 1 February 2012 14:30
Subject: Cron pkg_import@jubany /usr/bin/python ${SCRIPTS_DIR}/email-failures
To: pkg_imp...@jubany.canonical.com




evince failed at 2012-01-30 13:42:01.725202 due to:
Traceback (most recent call last):
 File /srv/package-import.canonical.com/new/scripts/bin/import-package,
line 7, in module
   main()
 File 
/srv/package-import.canonical.com/new/scripts/udd/scripts/import_package.py,
line 1177, in main
   only_before=options.only_before))
 File 
/srv/package-import.canonical.com/new/scripts/udd/scripts/import_package.py,
line 1078, in _import_package
   bstore, push, possible_transports=possible_transports)
 File 
/srv/package-import.canonical.com/new/scripts/udd/scripts/import_package.py,
line 638, in import_package
   use_time_from_changelog=True)
 File 
/srv/package-import.canonical.com/new/scripts/plugins/builddeb/import_dsc.py,
line 1243, in import_package
   pull_debian=pull_debian)
 File 
/srv/package-import.canonical.com/new/scripts/plugins/builddeb/import_dsc.py,
line 1146, in _import_normal_package
   file_ids_from=file_ids_from)
 File 
/srv/package-import.canonical.com/new/scripts/plugins/builddeb/import_dsc.py,
line 874, in import_upstream
   exclude=exclude)
 File 
/srv/package-import.canonical.com/new/scripts/plugins/builddeb/upstream/pristinetar.py,
line 199, in import_component_tarball
   exclude=exclude)
 File 
/srv/package-import.canonical.com/new/scripts/plugins/builddeb/upstream/pristinetar.py,
line 413, in make_pristine_tar_delta
   return make_pristine_tar_delta(dest, tarball_path)
 File 
/srv/package-import.canonical.com/new/scripts/plugins/builddeb/upstream/pristinetar.py,
line 115, in make_pristine_tar_delta
   raise PristineTarDeltaTooLarge(stderr)
bzrlib.plugins.builddeb.upstream.pristinetar.PristineTarDeltaTooLarge:
The delta generated was too large: tar: Record size = 16 blocks
xdelta: warning: no matches found in from file, patch will apply without it
error: excessively large binary delta for
/srv/package-import.canonical.com/new/updates/evince/evince_3.3.4.orig.tar.xz
(Probably the tarball is compressed with an unsupported form of compression.)
.



xscreensaver failed at 2012-01-30 23:37:31.010788 due to:
Traceback (most recent call last):
 File /srv/package-import.canonical.com/new/scripts/bin/import-package,
line 7, in module
   main()
 File 
/srv/package-import.canonical.com/new/scripts/udd/scripts/import_package.py,
line 1177, in main
   only_before=options.only_before))
 File 
/srv/package-import.canonical.com/new/scripts/udd/scripts/import_package.py,
line 1071, in _import_package
   possible_transports=possible_transports)
 File 
/srv/package-import.canonical.com/new/scripts/udd/scripts/import_package.py,
line 951, in handle_collisions
   if clean_collision(importp, suite, db, temp_dir, download_dir,
name, updates_branch):
 File 
/srv/package-import.canonical.com/new/scripts/udd/scripts/import_package.py,
line 885, in clean_collision
   return check_same(importp, db, revid, name, updates_branch,
temp_dir, download_dir)
 File 
/srv/package-import.canonical.com/new/scripts/udd/scripts/import_package.py,
line 852, in check_same
   importp, db, revid, name, updates_branch, temp_dir, download_dir)
 File /srv/package-import.canonical.com/new/bzr/bzrlib/cleanup.py,
line 132, in run
   self.cleanups, self.func, self, *args, **kwargs)
 File /srv/package-import.canonical.com/new/bzr/bzrlib/cleanup.py,
line 166, in _do_with_cleanups
   result = func(*args, **kwargs)
 File 
/srv/package-import.canonical.com/new/scripts/udd/scripts/import_package.py,
line 872, in _check_same
   pull_debian=False)
 File 
/srv/package-import.canonical.com/new/scripts/plugins/builddeb/import_dsc.py,
line 1243, in import_package
   pull_debian=pull_debian)
 File 
/srv/package-import.canonical.com/new/scripts/plugins/builddeb/import_dsc.py,
line 1140, in _import_normal_package
   version.upstream_version)
 File 
/srv/package-import.canonical.com/new/scripts/plugins/builddeb/import_dsc.py,
line 1061, in upstream_parents
   package, pull_version.upstream_version)
 File 
/srv/package-import.canonical.com/new/scripts/plugins/builddeb/upstream/pristinetar.py,
line 299, in version_as_revisions
   None: self.version_component_as_revision(package, version, component=None) }
 File 
/srv/package-import.canonical.com/new/scripts/plugins/builddeb/upstream/pristinetar.py,
line 320, in version_component_as_revision
   raise PackageVersionNotPresent(package, version, self)
bzrlib.plugins.builddeb.errors.PackageVersionNotPresent: xscreensaver
5.15 was not found in PristineTarSource at

Re: Improved quilt patch handling

2012-01-18 Thread Martin Pool
On 19 January 2012 11:30, Jelmer Vernooij jel...@samba.org wrote:

 bzr-builddeb 2.8.1 has just landed on Debian Sid and Ubuntu Precise. This 
 version contains some of my improvements from late last year for the handling 
 of quilt patches in packaging branches.

That's great.  I have taken the liberty of reposting it on the blog
http://blog.bazaar.canonical.com/?p=405.  I think you should forward
this to ubuntu-devel too.

--
Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: juju-izing udd

2011-12-21 Thread Martin Pool
I'm going to try writing a Juju charm for the udd importer.

(time passes)

Well, I did start, but I got bogged down in debugging why my lxc
provider won't boot.  Details and tiny patches on the juju list (which
you should join.)  If someone else wants to have a go, please do.

Plan:
 * it runs on Lucid, so we need a lucid charm
 * the importer ought to be checked out from the branch
 * what other packages does it need? bzr, bzr-builder, ...
 * where is it going to store its working state? on ebs?
 * how does it get the secret keys to access Launchpad?

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: juju-izing udd

2011-12-21 Thread Martin Pool
... I was working in lp:~canonical-bazaar/charm/oneiric/udd/trunk

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: Moving udd to django

2011-12-13 Thread Martin Pool
I think switching from batch html, handcoded sql and ssh access to
higher-level alternatives would be well worth while and Django seems
like a good choice. +1 from me, and also on the specific incremental
changes.

I agree with James, and disagree with Vincent, about bringing these
changes in to the running UDD importer.  It will benefit from eg
having a better web and admin ui, it will be less risky to deploy
changes gradually, and we are better off not forking if we can avoid
it.  Eventually we can split it into separate services/components.

Merging some of udd's failure tracking code with oopses would be good
but I think it's mostly complementary too this work, rather than an
alternative.

-- 
Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: Using lp:udd beyond its original purpose

2011-12-13 Thread Martin Pool
 There are two big changes that aren't necessary for us, but would help a lot:
  1. Decouple lp:udd from the package-import.canonical.com production 
 deployment
  2. Split the package scanning and the branch importing parts of
 lp:udd into separate projects

That sounds good to me, especially #1.  I would much rather have the
splitting merged back upstream than to have the tree forked.

-- 
Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: Using lp:udd beyond its original purpose

2011-12-07 Thread Martin Pool
(re-send)

On 8 December 2011 07:13, Martin Pool m...@sourcefrog.net wrote:
 There are two big changes that aren't necessary for us, but would help a lot:
  1. Decouple lp:udd from the package-import.canonical.com production 
 deployment
  2. Split the package scanning and the branch importing parts of
 lp:udd into separate projects

 That sounds good to me, especially #1.  I would much rather have the
 splitting merged back upstream than to have the tree forked.

 --
 Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: UDD up-to-date

2011-11-27 Thread Martin Pool
The buildd update saga is still continuing.  We're planning a new
deployment this week which will improve translation functions, and
then after that we will deploy a new launchpad-buildd which will fix
http://pad.lv/891892, building of packages where -sa is needed to
force the source tarball to be included.

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: Fwd: Thanks for fixing recipes...and Launchpad Recipe help please

2011-11-17 Thread Martin Pool
On 18 November 2011 11:23, Scott Ritchie scottritc...@ubuntu.com wrote:
 On 11/15/2011 07:07 PM, Martin Pool wrote:

 And now that it's deployed I'm getting a different error:

 bzr: ERROR: deb-version not fully expanded: {latest-tag}+daily-2018.
 Valid substitutions are: ['{time}', '{date}', '{revno:packaging}',
 '{revno}', '{svn-revno:packaging}', '{svn-revno}',
 '{git-commit:packaging}', '{git-commit}', '{latest-tag:packaging}',
 '{latest-tag}', '{debversion:packaging}', '{debversion}',
 '{debupstream-base:packaging}', '{debupstream-base}',
 '{debupstream:packaging}', '{debupstream}', '{revdate:packaging}',
 '{revdate}', '{revtime:packaging}', '{revtime}']

 https://launchpadlibrarian.net/85367867/buildlog.txt.gz

 # bzr-builder format 0.3 deb-version {latest-tag}+daily-{date}
 lp:wine
 nest-part packaging lp:~scottritchie/wine/natty-packaging debian debian

 Am I doing something wrong?

No, it's another bug, thanks for your persistence.

https://bugs.launchpad.net/launchpad-buildd/+bug/891892

This is now fixed in the source, I'm not sure if it's deployed yet.

-- 
Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: Fwd: Thanks for fixing recipes...and Launchpad Recipe help please

2011-11-17 Thread Martin Pool
 This is producing this launchpad error when I try to change the recipe:

 Error parsing recipe:1:22: Unknown format: '0.4'.

 Another bug? :)

i'm guessing that means it's not yet deployed rather than an actual
bug.  I will look in to it.

-- 
Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: Fwd: Thanks for fixing recipes...and Launchpad Recipe help please

2011-11-17 Thread Martin Pool
On 18 November 2011 14:16, Martin Pool m...@canonical.com wrote:
 This is producing this launchpad error when I try to change the recipe:

 Error parsing recipe:1:22: Unknown format: '0.4'.

 Another bug? :)

 i'm guessing that means it's not yet deployed rather than an actual
 bug.  I will look in to it.

ok, so uranium for example is running  0.7.2+bzr156-0 which ought to
have support for version 0.4 as far as i can see.  please file a bug,
including a link to the recipe and the failed build.

-- 
Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: Fwd: Thanks for fixing recipes...and Launchpad Recipe help please

2011-11-15 Thread Martin Pool
 I changed my recipe to use {latest-tag} instead of {debupstream} today,
 requested a rebuild, and got a new build failure:

 https://launchpadlibrarian.net/85208307/buildlog.txt.gz

 https://bugs.launchpad.net/launchpad/+bug/890834

 It seems to be a regression.

Thanks for reporting this.  It is now fixed in the source and should
be deployed in a day or two.


-- 
Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: UDD up-to-date

2011-11-10 Thread Martin Pool
One more recent improvement, thanks to Raphaël
https://bugs.launchpad.net/bugs/827935, you can now look at
https://code.launchpad.net/~ubuntu-branches to see the recently
changed official Ubuntu branches.

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Fwd: Thanks for fixing recipes...and Launchpad Recipe help please

2011-11-09 Thread Martin Pool
I don't know the answer to this off hand.  Does someone else?  Maybe
we can add this to the help.l.n documentation?

Martin





-- Forwarded message --
From: Scott Ritchie scottritc...@ubuntu.com
Date: 10 November 2011 06:05
Subject: Thanks for fixing recipes...and Launchpad Recipe help please
To: martin.p...@canonical.com


Hey Martin,

Thanks a ton for helping fix the daily build recipes bzr-out-of-memory
issue.  Now that my daily builds are building again, I'm hoping I can
bother you with a question...


How exactly do I use Git tags in a recipe?  It seems like there's
support for this, but I couldn't figure it out.

Currently my recipe looks like this:

# bzr-builder format 0.3 deb-version {debupstream}+daily-{date}
lp:wine
nest-part packaging lp:~scottritchie/wine/natty-packaging debian debian

The {debupstream} pulls the upstream version from that natty-packaging
branch I set up, but what I'd like it to use is the GIT tag from lp:wine
since I don't really have any reason to update this branch other than to
increment the version number.

1) Are git tags preserved at all in LP's BZR imports?
2) If yes, what's the equivalent of {debupstream} that I can use to
access them?


Thanks,
Scott Ritchie

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: Fwd: Thanks for fixing recipes...and Launchpad Recipe help please

2011-11-09 Thread Martin Pool
 Is there a place where I can track this?  I'd like to know when it
 becomes available.  Thanks for the full answer :)

That's a good question, and maybe something that's a bit missing in
lp's communication at the moment.

We will reply on this thread and also put something on the Launchpad
blog when it's live.

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: UDD up-to-date

2011-11-08 Thread Martin Pool
We're in the middle of a fairly epic series of roll-outs to the
Launchpad buildds.  Done today, both on staging and production, is an
update to bzr 2.4, which should let large trees be assembled in recipe
builds without running out of memory.

Coming up soon are upgrades to the buildd client mechanism to make it
more supportable, and to allow building non-native packages
(https://bugs.launchpad.net/launchpad/+bug/885497) and some other
things.

Thanks to Lamont, Jelmer, wgrant, and everyone else involved.

m

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: UDD up-to-date

2011-10-03 Thread Martin Pool
On 27 September 2011 19:34, Martin Pool m...@canonical.com wrote:
 progress in recent weeks:

  * John did great work to give much faster branching of large linear
 histories, like linaro-gcc: this is now 3x faster (down from 3h to 1h
 to branch to me in Australia; flatlined pipe would be about 15m so
 there's some room for more)

  * package importer is getting smarter at dealing with lp downtime,
 and at the same time lp is doing great work on not having any downtime

... and I'm happy to say this is now done, and during Launchpad's
recent rollout things were indeed much smoother:
http://pad.lv/795321

I was wondering when I prepared this if there was some better place to
send or post it that might reach more people who use this service but
don't necessarily want all the details of the discussion.

m

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


UDD up-to-date

2011-09-27 Thread Martin Pool
progress in recent weeks:

 * John did great work to give much faster branching of large linear
histories, like linaro-gcc: this is now 3x faster (down from 3h to 1h
to branch to me in Australia; flatlined pipe would be about 15m so
there's some room for more)

 * package importer is getting smarter at dealing with lp downtime,
and at the same time lp is doing great work on not having any downtime

 * bzr 2.4.1 is in Oneiric: many speedups compared to Natty - I am
happy to hear at least some developers say speed is generally no
longer an issue

 * jelmer is working on multi-tarball imports with good progress

 * mgz joined the canonical bazaar team, hooray

 * jam's working on high-availability for the launchpad bzr server, so
that when launchpad do an update clients will smoothly disconnect and
reconnect

mgz, jam and vila will be at UDS-P: we need to propose some session
topics: at the moment I'm thinking of one about
BuildFromBranchIntoArchive, one about quilt merging, one about
bzr/lp/udd in general.  Any other ideas?


Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


ubuntu: branches lacking history with upstream branches

2011-09-22 Thread Martin Pool
pitti mentioned to me today the issue of ubuntu-namespace branches
that ought to be related to an upstream bzr branch, but that actually
aren't.

lp:indicator-power is one example, and the desktop team actually
maintain an unofficial packaging branch that does share history:
lp:~ubuntu-desktop/indicator-power/ubuntu

It seems clear it will save confusion to fix this up.  Making the
branch correct for people actively working on it is probably the most
important thing.

 * I think in general we just rebind the branch alias to the branch
that should be official?  I guess we need a losa to do that?
 * The package importer might need to be told to reset those branches.
 * Perhaps we should document when/how we do this.
 * Perhaps we can get something closer to self service for people who
do hit it, or systematically prevent it in the importer?

Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: Build service thoughts

2011-09-14 Thread Martin Pool
On 14 September 2011 00:42, Jonathan Riddell jridd...@ubuntu.com wrote:

 I packaged bzr for openSUSE recently and it was suggested I send some
 notes on how their build service compares to Launchpad.

Thanks for writing that up.  jml and I (at least) have looked at it
before but there's nothing like really using a tool to get an idea of
it.

 One difference that surprised me is packages are built instantly as
 soon as you commit.  This can probably be seen as quite a waste of
 build daemon time since it will be trying to make packages even if
 what you are working on isn't complete but it does remove a step from
 the user (in our case quite a hassle filled step since it requires
 using different tools to upload to the archive until build from branch
 appears).

Right, so part of that could be covered by BFBIA[1] removing the other
step.  I guess if it's really 'instant' then that also implies they
have more hardware headroom than Launchpad does, where there is
typically(?) quite a queue before a package is built.

 Build numbers automatically increment for each build which saves the
 minor hassle of having to do so yourself in the changelog.

That could be nice.  Perhaps we can handle that - perhaps just as
simply as making bzr commit in the branch suggest or generate a
changelog entry, or something.

 One source package is built for many distros and distro versions.
 This would be nice to have in PPAs, again removing a minor hassle from
 packagers of having to build and upload source packages multiple
 times.

Right, this is somewhat harder to individually script around, and I
think one of the bigger annoyances of PPAs today.  From the
implementation point of view you do need different version numbers for
each rebuild, but for the user it seems unreasonable.

We could do either of a couple of kludges:

 * some kind of easy flow around copying the binary between
distroseries, for cases where it's appropriate
 * a purely-automated tool to copy the package, add a changelog entry,
reupload - like a subset of autoppa, but without needing any
configuration aside from the choice of destination series.

I kind of like the latter: it would cope ok in the fairly common
(cite?) case of one source package building adequately well across all
active series.

 Packages are stored with packaging only and source in a tar file.
 This simplifies the problem compared to our UDD process of having full
 source branches in revision control.  Given the hassle full source
 branches seem to make for UDD (still  500 import failures, quilt on
 top to bzr double revision control, notable new workflows for
 packagers, definitive source RCS is upstream and ours don't generally
 match) doing packaging only branches would seem to me to have been an
 easy win but too late to change now.

right.

 They have OSC, a command line tool for checkout and commit (similar to
 subversion in its revision control functions so much more limited than
 bzr there).  It also allows for easy searching of packages, checkouts
 include personal archives, and it submits merge requests which could
 be done as bzr plugins with support from Launchpad.

I would love to have a single commandline tool, integrated with bzr,
that reaches everything in Launchpad.  Perhaps lptools and/or lbox
will be that thing.

 They have external users, notably the Linux Foundation which means
 MeeGo is closely aligned to SUSE.  Maybe if Launchpad had been easier
 for external users to set up MeeGo would be aligned to Ubuntu?

I suspect (with no special knowledge) there are business
considerations that were more important than anything about the web
site, but making the tools really good can only help in encouraging
people to work with Ubuntu.

m

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: Collision branches

2011-09-07 Thread Martin Pool
On 8 September 2011 07:48, James Westby jw+deb...@jameswestby.net wrote:
 Hi,

 Thanks for fixing the bugs that were preventing merge proposals for
 getting filed for collisions.

 This had led to a surge in the number of such merge proposals. This is
 mainly due to a backlog, but there have been 10 or so in the two days
 since.

Yes, I'm also glad it got unblocked but also a bit alarmed at how many
mps were created with no apparent real meaning.


 You can see the extent of this by searching for ubuntu-branches at
 http://reports.qa.ubuntu.com/reports/sponsoring/index.html

 Colin has valiantly reviewed some of them (maybe half, thanks Colin,)
 and has found that in none of the cases so far were the collisions
 real in the sense that someone pushed and someone else uploaded
 something different.

 There was one case at

  https://code.launchpad.net/~ubuntu-branches/ubuntu/oneiric/ibus/oneiric-201108121834/+merge/73916

 which seems to indicate a bug though.

 From my previous experience going through these merge proposals the
 majority of issues will be caused by the representation of quilt
 in the branch.

 Can the Bazaar team do something to stop this influx of merge proposals
 that must be sorted, leaving just real ones? Does this have to involve
 work on looms?

I looked into a few of them and they weren't all clearly due to quilt
problems, but perhaps most of them are (or I didn't understand the
cause from a glance.)

I think we can handle this without blocking on looms by doing a
smarter merge that unapplies and reapplies the patches.  There is some
work towards this in eg https://bugs.launchpad.net/bugs/608675 which
Jelmer is working on - we may need extra work to hook it up into the
udd importer.

What we should probably do next is look at the merge proposals that
were filed and work out whether each one

- is a real conflict in a sensible form
- is not a real conflict and shouldn't be generated at all (some have zero diff)
- could be either avoided or better presented by smarter quilt
handling or something else

Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: Terminating the append_revisions_only experiment, for now

2011-08-23 Thread Martin Pool
On 23 August 2011 17:41, vila v.ladeuil...@free.fr wrote:
 Max Bowsher _...@maxb.eu writes:

     We currently have 97 packages failing with AppendRevisionsOnlyViolation
     - that's around a sixth of all failures.

     I think we should roll back the change to the importer to set
     append_revisions_only on UDD branches, and make the importer clear the
     flag when doing its own pushes to Launchpad.

 Very close to my own thoughts, thanks for bringing this up.

+1 from me too.


     It feels clear to me that what seemed like an obvious win at the
     time of introduction has revealed lots of interesting and
     unexpected edge cases.

 This is also very close to my feelings and I like to add that we should
 *really* write tests to capture them and make sure we don't regress
 again in the future.

 Overall, I feel we have far too much failures when landing *good* stuff
 *because* we lack a proper way to test. It's a long known issue that we
 miss a launchpad test server but *not* having tests to guard against
 regressions is clearly a path we don't want to pursue as demonstrated
 here (IMNSHO).

 Whether we address that by using staging or any sort of local launchpad
 test server is open to discussion but the sooner we start this
 discussion the better.

I think the easiest way to write and run the tests is probably to make
them run against the staging Launchpad, and then perhaps to have them
skip if Launchpad is unreachable.  At any rate I was going to try
doing that for the file_mp related bugs.  We'll see how it works out.

Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: critical: 806348 BzrCheckError

2011-07-21 Thread Martin Pool
On 22 July 2011 11:31, Max Bowsher _...@maxb.eu wrote:
 On 22/07/11 00:54, Martin Pool wrote:
 https://bugs.launchpad.net/bzr/+bug/806348 seems to be causing a
 steadily increasing number of package import failures - if you look at
 the failure count graph it is ramping steadily up to the right.  This
 could have happened either because of the upgrade of jubany to lucid
 or the upgrade of Launchpad to bzr 2.3.3.

 How about we switch jubany back to 2.3.x and see what happens?

That makes sense to me.  I will talk to the sysadmins.

m

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: critical: 806348 BzrCheckError

2011-07-21 Thread Martin Pool
We're back on 2.3.4 on Jubany and the importer is at least running.
So far it has not actually found any work to do.  I'm going to see if
I can get it to re-run some of these packages and see what effect that
may have, and perhaps try to improve the reporting within the
importer.

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: Please RT: apt-get install python-bzrlib.tests

2011-07-17 Thread Martin Pool
On 16 July 2011 23:05, John Arbash Meinel j...@arbash-meinel.com wrote:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

 On 7/16/2011 11:27 AM, Max Bowsher wrote:
 We've lost the ability to run ./selftest.py on jubany, because we've
 upgraded to a bzr packaging which splits bzrlib.tests out into a
 separate .deb, and this has not been installed.

 Therefore please RT for: apt-get install python-bzrlib.tests

 Thanks,
 Max.



 rt #46911

When you file tickets, please put me on the cc line of the original
mail, so that I can see them before they're triaged.

Thanks,
Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: Please file a RT for installation of python-debian on jubany

2011-07-15 Thread Martin Pool
It's done now.

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: RFC: Minor collisions handling, both manual and automatic

2011-07-13 Thread Martin Pool
That makes sense to me too.

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: jubany. lucid?

2011-07-13 Thread Martin Pool
I think we're now up and running again; if you see any continuing
problems please let me know (and/or fix it ;-).

m

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: Ubuntu Packaging Guide

2011-06-05 Thread Martin Pool
On 4 June 2011 01:28, Mackenzie Morgan maco...@gmail.com wrote:
 On Fri, Jun 3, 2011 at 10:50 AM, Barry Warsaw ba...@ubuntu.com wrote:
 On Jun 03, 2011, at 10:07 AM, Mackenzie Morgan wrote:

 From what I understand, there are people doing things all sorts of ways with
quilt, and I really don't want to have to learn all the ways people are using
quilt with bzr and try to figure out *which* way any particular package is
using that combination.  I'll stick to apt-get source for those.

 I've successfully used the guidelines on this page for several quilt 
 packages:

 http://people.canonical.com/~dholbach/packaging-guide/html/udd-patchsys.html

 By no means is it perfect, which everyone acknowledges.  Depending on your
 level of pain tolerance, you don't necessarily have to punt on UDD right away
 when working on a quilt3 package.

 What if you just want to do quilt import ../mychanges.patch (my
 usual use-case for quilt)?  Right now, I'm thinking the old cheater
 way (cp ../mychanges.patch debian/patches  echo mychanges.patch 
 debian/patches/series) seems a lot easier.

 Also, the text between the code-boxes on that page are not so helpful
 if you don't know what a loom or a thread are. Well, I mean, I know
 what real looms and real threads are (and goodness are real looms ever
 *expensive*!), but I don't think my textile interests are much help
 here.  I'm guessing that a thread is a branch of a branch, but hiding
 inside the meta-branch like how git branches all live in one dir, but
 really this is my confusion talking.

Your guess is correct.  A loom also records (when you 'bzr record')
which version of each of the threads goes together at any point in
time, as a kind of meta-versioning.

There is some more documentation here:
http://wiki.bazaar.canonical.com/Documentation/LoomAsSmarterQuilt.

Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: UDD steering committee meeting minutes 2011-06-01

2011-06-01 Thread Martin Pool
Sorry I missed it, I had a personal interrupt last night.
-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


UDS branches plenary

2011-04-20 Thread Martin Pool
I think it would be great to have a plenary at UDS-O about what people
are doing with Ubuntu branches.  I have put a proposal in
https://wiki.ubuntu.com/UDS-O.  We have to make a plan for something
to say in 15m that will be exciting, relevant, not waffly, and not a
vastly-truncated tutorial.

[this mail's a bit of a placeholder; more tomorrow; or say what you
think we should cover]

Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: Meeting minutes and possible meeting time change

2011-04-11 Thread Martin Pool
Thanks, Barry, that's very neat.

spiv mentioned the other day he found it a bit unclear at the moment
what was on the bzr team's path towards udd was at the moment.  In a
nutshell, what we're going to do is get package-import.ubuntu.com
failures down to 0, and work on bugs that are affecting Ubuntu or
Linaro developers, including fetch speed.  The specific things we
choose towards these are going to be shortlisted into
https://bugs.launchpad.net/~canonical-bazaar/+assignedbugs, and will
also show up in
http://people.canonical.com/~mbp/kanban/canonical-bazaar-kanban.html.
 The next major features coming up would probably be quilt imports and
colocated branches, but I'd like to get more existing failures fixed
first.  Perhaps we should draw a line for when to cut over.

Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


discussion of ubuntu-desktop packaging branches

2011-04-10 Thread Martin Pool
Robert pointed out
https://lists.ubuntu.com/archives/ubuntu-desktop/2011-April/002940.html,
about moving to 'normal mode' of bzr-builddeb, in which the entire
tree is versioned.

 - It is possible to screw up the branches so that bzr merge-package throws a 
 confusing error (I keep doing it).  Perhaps we need some hooks in bzr to stop 
 this from occurring.  If it can be broken, it will be broken (repeatedly).

Robert, could you tell us more about this or file bugs?

Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: Summary from UDD meeting 2011-03-23

2011-03-31 Thread Martin Pool
Just to have a bit of closure on this: we're going to work on imports
and other bugs (performance, warning about out-of-date branches) for a
bit more, until either we feel the import failures are running dry, or
until a Launchpad squad is free to do this.

Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: missed meeting

2011-03-02 Thread Martin Pool
Sorry, I'm going to be about 15m late.

- Martin
On 03/03/2011 4:04 AM, Barry Warsaw ba...@ubuntu.com wrote:
 On Mar 02, 2011, at 06:02 PM, Martin Pool wrote:

Those dates work well for me. (I'm going to be offline again on the
10th of March, but here this week and also the 17th.)

I could have a reasonable chance of being awake at 7am Sydney ==
20:00UTC, if that is any better. (Personally I think 10pm is actually
a bit less disruptive than 9pm.)

 Thanks for updating the Calendar Martin. See you all at 2100 UTC.

 -Barry
-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: rfc: permissions on package branches

2011-03-01 Thread Martin Pool
On 18 February 2011 17:27, Robert Collins robe...@robertcollins.net wrote:
 On Fri, Feb 18, 2011 at 6:30 PM, Martin Pool m...@canonical.com wrote:
 On 18 February 2011 16:27, Robert Collins robe...@robertcollins.net wrote:
 What about 3 - have no owner at all: there is a unique path for each
 package branch, so we could just use that, and only that, for the
 branch path for official package branches.

 That's fine with me.  It seemed like it might be harder to implement
 inside Launchpad?  We can either go there directly, or actually do 1
 (celebrity owner) but make it look like 3.

 We probably want an owner in the same sense that a team has an owner:
 someone that has administrative privilege over the thing but no direct
 access to the content of the thing. (For instance, the owner of a team
 can set an administrator, but can't join the teams mailing list).
 [modulo bugs :P]. I'd make *that* owner for these branches the owner
 of the distro series the branch is for, not a celebrity.

 Making package upload rights supercede 'owners' rights should be very
 straight forward.
 The nominal owner - the distro series owner - would
 provide a regular namespace today, and it should be pretty straight
 forward to coerce the official package branches into just having their
 official namespace as default.

(In passing, there doesn't seem to be any way on
https://launchpad.net/ubuntu/maverick to see who the owner is, but
the api tells me it is ~techboard. http://pad.lv/727632)

OK, so it seems like this doesn't _necessarily_ require any changes to
Launchpad: only the distro series owner (ie ~techboard) can mark a
branch as official.  They could simply refrain from doing that except
on branches they own themselves, in which case only ~techboard plus
the package uploaders would have access to it.

However, there is a bit of a trap there that they might not realize
the security implications of making a branch owned by someone else be
official.

From the thread so far, it seems like the simplest thing might be to
allow branches to only be made official when they are also directly
owned by the distro series owner?

Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: rfc: permissions on package branches

2011-02-17 Thread Martin Pool
On 18 February 2011 12:07, Scott Kitterman ubu...@kitterman.com wrote:
 On Tuesday, February 15, 2011 12:34:28 am Martin Pool wrote:

 There are a few options here and we'd appreciate hearing from Ubuntu
 people how they think it should work:

 0- No change: the nominal owner keeps write access.

 1- Don't allow branches owned by non-celebrities to become the
 official branch for a package.  Instead, you need to push from that
 branch into the real official branch.

 2- When the branch becomes an official package branch, the owner loses
 write access (unless they're also an uploader.)  That's what
 https://code.launchpad.net/~jml/launchpad/owner-cannot-write-to-official-b
 ranch-516709/+merge/29446 would do.  It seems potentially confusing.

 3- Something else?

 Let us know what you think either here or on that bug.  Also if you
 think we ought to ask eg the TB, please tell me.



 Now that I know what a celebrity is in this context, I think #2 is reasonable.
 #1 would be achieved by not turning a particular branch into a packaging
 branch and using the default official branch (we have that now).

 Official Ubuntu branches are supposed, in some way, to relate to the code in
 the distro, so I think it makes sense to keep the upload and branch
 permissions aligned.  If someone doesn't like losing write permissions to a
 branch for a package they can't upload, then they shouldn't have it made the
 official branch.

People are funny things.  I can safely bet that if we support changing
person branches into official packaging branches, some people will do
it and then be confused by the consequences.  Also I think it makes
the access control model a bit more special-cased than it would
otherwise be.

So after this discussion, I think we should do 1, and then move
towards perhaps having the celebrity owner be mostly hidden.

I can't see any really positive reasons for allowing someone to change
an existing branch into an official source package branch.  Are there
any?

Thanks
--
Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: Import layout of Quilt v3 packages

2011-02-08 Thread Martin Pool
On 8 February 2011 19:00, Max Bowsher m...@f2s.com wrote:
 On 08/02/11 07:00, Martin Pool wrote:
 At the moment it seems to me we need to either: import to looms and
 mandate using looms; or check in things with everything expanded and
 provide glue that will keep the quilt data up to date with the wt.
 (Perhaps they should be considered derived data and updated from a
 hook.)

 I don't think we want to be checking in .pc at all.

 Personally I quite like the simplicity of just checking in the package
 with the patches NOT applied, but I gather people like being able to
 navigate the actual history of the patched package.

 Therefore, what about checking in the patched code, without any quilt
 metadata (.pc dir) but with a flag file that triggers bzr-builddeb to
 write out the appropriate metadata whenever a working tree is built for
 such a branch?

 (Writing out the metadata would consist of copying the series file to
 .pc/applied-patches, and reverse-applying each patch in reverse order,
 stashing the resultant modified file in .pc/patchname/filename for
 each patch)

Right, that's what I meant by my (somewhat unclear) second option.

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: Making bzr commit more like debcommit

2011-01-24 Thread Martin Pool
On 25 January 2011 03:32, Robert Collins robe...@robertcollins.net wrote:
 Should be easy enough to add a hook to get fixes from plugins; the
 tricky bit is allowing users to confirm that the detected things are
 correct (or perhaps thats overkill?)

 Anyhow the bzr-builddeb, commitfromnews, launchpad plugins would all
 probably like this.

Yes, let's add a hook.  https://bugs.launchpad.net/bzr/+bug/707274

There is already commit_message_template, and start_commit.

I guess what we want here is a hook that can say:

 * here's the actual message to use; you don't need to edit it
 * i've modified the revision properties as follows

-- 
Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: udd at uds-n

2010-11-30 Thread Martin Pool
On 1 December 2010 09:27, Barry Warsaw ba...@ubuntu.com wrote:
 Hi Martin, thanks for posting this update, and apologies for taking so long to
 get around to reading it.

 On Nov 17, 2010, at 08:06 PM, Martin Pool wrote:

At the end of that discussion we picked two specific items for the bzr team:
 * speed
 * loom support, on lp and within bzr, and connecting them to packaging 
 patches

and for Launchpad
 * build from branch into the main archive
 * actually execute a merge from a merge proposal
 * through launchpad. merge from a debian branch into an ubuntu udd branch

 I think we also have to address the package import failure issues.  I see two
 parts to that.  First, we need to make sure that if someone branches
 lp:ubuntu/foo (or ubuntu:foo wink) on a branch that has had import failures,
 that some very prominent warning is displayed.  Perhaps the branch fails
 unless a --force flag or something is given.  I'm not sure exactly, but I'm
 fairly confident that silently producing a branch that's out-of-date is *not*
 a good thing. :)

 Second would be to address the issues actually causing the failures, but
 that's a longer term project.

jml asked a similar question but I just realized it was off the list.
That was, how do I reconcile what was discussed at UDS with the
feedback we got in the UDD survey results
https://lists.ubuntu.com/archives/ubuntu-distributed-devel/2010-November/000645.html
which did emphasize import reliability.

The short story is that for now, we're going to work on holistic
network performance to/from Launchpad, on getting the package importer
working better, and on general bugs/reactive work.

-- 
Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Fwd: Proposed bzr-builddeb SRU

2010-11-23 Thread Martin Pool
-- Forwarded message --
From: Bilal Akhtar bilalakh...@ubuntu.com
Date: 24 November 2010 02:36
Subject: Proposed bzr-builddeb SRU
To: ubuntu-distributed-devel@lists.ubuntu.com


-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hello all!
       Version 2.6ubuntu0.1 of package bzr-builddeb is in maverick-proposed
from 2 weeks now, and it would definitely be helpful for every (new and
old) UDD user (or developer) if this package went into -updates.
https://bugs.launchpad.net/bzr-builddeb/+bug/668764 is the bug it is
fixing. I request someone to verify the upload whenever they get time.

Thanks again!

Bilal Akhtar.


- --
Bilal Akhtar - Ubuntu Developer, yet still 14 years old.
More information about him can be found on
https://edge.launchpad.net/~bilalakhtar
IRC nick: bilalakhtar
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJM6997AAoJENsMxwzgYeH255gIAJtaJAn7woUWC3v/G0w12JIY
zkpwO+7xmFhW8+IA0ktd67cAq3NOwMboEFMglQe6Vz2KeIdf5S3Vy7wW76tssmTW
m/0NbeUFS2gJAkI9Xgd6qM4uspMAiFSpINJUbkqXn1Vn3VCFKhzObGd9H8x+IGZI
gPFf3vFuk8jLl8moNBy+xmnWnk0fksm73I4orjqawHBX2TXYLH1YV2d2zUAwbpQP
4xRIvruuBCHK8U0jROLGHolAQULr3gPcj8qU7C1vgvLRYU6lwXZuA5W00pY0YwGi
t34LKpHGYLJTfDpGnPPieMFr6BbmYiTdMwlPwu2uge/DMgdsS7AdhdwqQdussh4=
=e3f8
-END PGP SIGNATURE-

--
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel




-- 
Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: UDD survey results

2010-11-18 Thread Martin Pool
On 19 November 2010 06:34, Francis J. Lacoste
francis.laco...@canonical.com wrote:

 On November 18, 2010, John Arbash Meinel wrote:

 Anyway, I don't know the net promoter stuff. I certainly don't see how
 you get 1-in-35 being -35.

 Doh,

 My mistake, you are right. I completely missed the 9 people who gave it a 10.

 So indeed, the net promoter score is (12/46)-(13/46) = -0.02 which is a lot
 less negative.

Obviously there is some question here about whether respondents see a
'8' or '9' as indicating the same degree of enthusiasm as whoever
decided it should be 9-10.  Perhaps a more objective question, next
time round, would be _have you actually_ suggested that other people
should (or shouldn't) use this.

Anyhow, this does seem to mesh with the other feedback: there are some
people who don't like the idea at all; there are some who are pretty
positive; most are willing to like it but hitting various problems.
Which we will fix.

-- 
Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


udd at uds-n

2010-11-17 Thread Martin Pool
Just a few brief notes about UDD discussions at UDS-N.  A lot of this
is captured to various degrees in gobby documents and IRC logs at
http://ubottu.com/uds-logs/, but they're pretty long:

http://ubottu.com/uds-logs/%23ubuntu-uds-bonaire2.log
* There was a lot of interest in adopting something like the bzr patch
pilot process  in Ubuntu; it's worked well for us and I'm pleased Rick
is willing to give it a go in the different situation of Ubuntu.  I
hope it goes well.

http://ubottu.com/uds-logs/%23ubuntu-uds-curacao1+2.log
The main discussion about UDD is here, starting about
2010-10-27T16:08, and there are a lot of good things for us to work
on, including:

 * speed of getting source through UDD, considered holistically
(including outright speed, needing to fetch from the UK rather than a
closer mirror, shallow checkouts, proactive mirroring)
 * various merge cases that could be better
 * problems with packages with complex history already in bzr,
creating multiple histories

Some other observations:
 * currently used by very much early-adopters
 * build from recipe quite successful, though still quite young
 * focus on eliminating tedium
 * check out grab-udd-merge

At the end of that discussion we picked two specific items for the bzr team:
 * speed
 * loom support, on lp and within bzr, and connecting them to packaging patches

and for Launchpad
 * build from branch into the main archive
 * actually execute a merge from a merge proposal
 * through launchpad. merge from a debian branch into an ubuntu udd branch

We did not get around to doing interactive user testing; I'm trying to
reschedule that with mrevell.

We did get survey responses, and I will send a summary of them soon.

-- 
Martin

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


UDD survey results

2010-11-17 Thread Martin Pool
TLDR? Mixed but generally positive feedback.  Top issues to fix are
speed of branching/merging from Launchpad; keeping import branches
reliably up to date; getting branches where possible to current
formats; removing various small-medium roadblocks; supporting v3
packages and being smarter about merging.

We did a survey before UDS asking people if they had tried using
Bazaar to deal with packaging branches for Ubuntu.   The results are
quite interesting.  Obviously like any opt-in survey there is going to
be some sampling bias, but nevertheless this gives us something to
steer our future efforts.

Most were quite positive, some with constructive criticism; some do
not like the idea at all.

I've put a detailed view survey results up here:

  http://people.canonical.com/~mbp/udd-survey-201011/SurveySummary.html or
  http://people.canonical.com/~mbp/udd-survey-201011/SurveySummary.pdf
(all on one page but unfortunately a bit mangled)

This doesn't have individual sheets, personally identifiable
information and I don't think it has any sensitive information.  It
does have the email addresses for people who chose to leave them, but
not connected to their responses.  Since all of those people have used
their addresses on public lists, I'm going to assume they are ok with
having them on the web.  (If you hear of any problems please let me
know and we'll take it down.)  If anyone wants to drill into
particular responses I will give you access.

85 people answered in all.  I'm grateful to all of them for giving their time.

75 have already used bzr branches for Ubuntu development to some extent.

There is a good even mix of core devs, motus, contributing developers,
casual developers, and new developers.

Net promoter score: 22 would recommend overall Ubuntu development
using Bazaar at least fairly strongly (net promoter score 7..10); 6
would recommend avoiding it (0..3).  However, 50 people skipped this
question, perhaps suggesting they have mixed feelings, or the question
was poorly stated (eg they don't see it as a thing they recommend to
others.)

Overall speed: 32 fast enough, 12 not fast enough, 41 no answer.
Despite those numbers I think many of the no answers actually do see
speed as a major issue, taking the survey as a whole.  The great
majority of comments about speed are to do with fetching from
Launchpad.

Reliability:
Similarly 34 reliable, 10 unreliable, 41 no comment, but here my
interpretation is that improving reliability is important, but it is
not at the moment a showstopper the way speed can be.  Some people see
bzr as reliable and lp as flaky; others just the opposite.  lp
downtime is hated, so I'm happy to know this is moving very fast
towards being reduced.  Getting bugfixes in bzr rolled out into the
distro packages is important.  The biggest issue is making sure that
branches are reliably up to date.

The strongest cluster of criticism was essentially it's not git,
with those respondents saying: they and their potential contributors
are already familiar with git; upstreams and Debian use git for the
package; there are features they like in it; they like git-style
collocated branches; it is faster.  Some of these people would
generally rather we switch to using git, some want Launchpad to
support git alongside bzr, and some want to continue along the line of
interoperation but to do it better.  Some specific features they miss
are mentioned below.  People only made this argument about git: nobody
said we should use support hg or svn alongside or instead of bzr.

Contra this, some people reported bzr easier to set up, learn and use
than git; better error messages.  There were mixed opinions on the
quality of git imports.

Positive things (for at least some people):
* Getting the code, branching and committing reported to be easy.
* bzr itself is nice.
* Having bzr-level log for changes.
* Easily being able to get old release sources.
* Pulling commit messages from changelogs.
* merge-upstream is nice.
* Familiarity of commands from cvs, svn or darcs.
* When Launchpad is obvious, it works well.
* Pipelines are cool.
* Bugs are generally fixed fast.
* Supportive community.

Criticisms mostly of Launchpad:
* Slow, in various ways, especially from APAC.
* Hard to navigate: finding specific branches; understanding what to
do; hard to find the cool features.
* Confusion/annoyance from old branch formats.
* Bad that the launchpad-side setup of stacked branches is
inconsistent with the client side repository directory.
* When something goes wrong, errors can be confusing.
* My biggest gripes are with the lp:ubuntu/pkgname full source
branches. They have some nice features and traits, but a lot of
problems still: - They are way too easy to break for new upstream
versions from maintainers who aren't used to it yet. - It's impossible
(at least for me) to convert an already existing branch (derived from
upstream bzr) into the official lp:ubuntu/pkgname branch (ask me
(pitti) for details) - Because of 

Re: import failures

2010-02-15 Thread Martin Pool
On 13 February 2010 04:09, James Westby jw+deb...@jameswestby.net wrote:
 On Thu, 11 Feb 2010 12:49:22 +1100, Robert Collins 
 robert.coll...@canonical.com wrote:
 On Wed, 2010-02-10 at 21:46 +, James Westby wrote:
 
  Some of them have been upgraded. If it's easier for me to do an info
  against all of them and filter out those not in 2a then I can do so.

 I think thats easiest.

 For some value of easy that involves writing a python script and running
 it for four hours to extract information from the LP database :-)

 Attached is a raw list of branch API url and repository format string,
 it can be massaged in to a list of branches to upgrade using your
 preferred text manipulation tools.

 3336 branches by my reckoning.

So, just to be sure, we want to upgrade all the non-2a ones to 2a, on
Launchpad?  Probably by running the upgrade either on the lp machine
itself or at least on a nearby machine?

-- 
Martin http://launchpad.net/~mbp/

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: UDD as a product?

2010-02-11 Thread Martin Pool
On 11 February 2010 18:59, Ian Clatworthy ian.clatwor...@canonical.com wrote:
 UDD now has an active mailing list, a Launchpad project and a bug/task
 list. Does it make sense to begin thinking about UDD as a product? Would
 it be valuable to talk about UDD x.y vs x.z?

 Code wise, I guess the product is a mix of LP features, Bazaar
 features and Bazaar plugins. OTOH, those things come together to form a
 system.

 There's a huge amount of wisdom being imparted each week on this list.
 Perhaps we should turn some of the threads into tutorials (either in a
 wiki or bzr branch) or a UDD Hackers Guide say. Is it too early for that?

Interesting question.  My feeling is that if there was a technical
product, it would be primarily a document describing how to do it, and
including by reference the bzr etc versions that you can use to do it.

I don't know if specifically reifying it as a Product helps.

Right now we're clearly at 0.something; perhaps after we get to 1.0 we
will think of doing a large step to UDD 2.0.

If you would like to lead/write a document about it that could be a
valuable contribution.  Obviously there is some stuff already written
but I don't think someone can say so what's UDD all about and how to
I do it quite as easily as we would like.

-- 
Martin http://launchpad.net/~mbp/

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: UDD @ Portland

2010-02-10 Thread Martin Pool
On 11 February 2010 13:18, Robert Collins robert.coll...@canonical.com wrote:
 James Westby and I had some time together in Portland to talk about UDD
 stuff.

 We talked about a few things:

 * Looms, their use today and where they should go
 * The operational issues with the package importer and how the bzr team
 can help
 * analysed a few specific bugs and tried to come up with solutions.

 Firstly though, a couple of overview points:
  - the udd project has 200 bugs on it. While many of these are
 'collision' reports many are not. The collision reports are currently
 overly noisy, so please ignore them for now. However, the other bugs are
 open season for people to fix - and every bug fixed there will
 streamline things for people using bzr to package in Ubuntu.

Can you explain what a collision is?

 Finally we looked at Looms with mathiaz who is hoping to get the MySQL
 packages in Looms for both Debian and Ubuntu. We identified some rough
 spots and a missing command (import-upstream) but it seems doable, if
 not /nice/ today. After that we talked about a sparser loom merge graph.

I'd like to let looms progress, but not (unless james or others feel
differently) add them into the dependency chain for getting UDD going.
 iow people should be able to try them on particular branches, without
mandating them for all package branches, and (perhaps?) without
requiring everyone working on that package to use them.

-- 
Martin http://launchpad.net/~mbp/

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: hottest100

2010-02-04 Thread Martin Pool
On 3 February 2010 22:14, Reinhard Tartler siret...@ubuntu.com wrote:



 Your list includes this:

 mplayer
   categorized package-bug:513282
   categorized ok_upstream
   lp:mplayer                                0    ok_upstream

 this cannot be true. mplayer makes heavy use of svn:externals which
 don't appear in the bazaar branch. This means that the bzr import is
 unusable.

Thanks for pointing that out.
https://code.edge.launchpad.net/~vcs-imports/mplayer/trunk does seem
to be importing it happily so it's an interesting question.

 Moreover, I wonder why mplayer is in that list, but ffmpeg
 isn't.

At the moment mplayer has something over 140 bugs in Ubuntu and ffmpeg
has 24.  The hotness ranking was by bug activity.  So it's plausible
to me that mplayer would make the cut and ffmpeg didn't.

-- 
Martin http://launchpad.net/~mbp/

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: hottest100 (was Re: Bazaar focus for 2.1 and 2.2)

2010-02-03 Thread Martin Pool
We had a big push on hottest100 last week, and it was good.  A fairly
recent copy of the hottest100 results are below.

To summarize where we got to: most of the upstream branches are now
working; there are a few not correctly registered but that could
probably be fairly easily fixed.  In package branches a bit over half
of the ones we sampled are working, and there are specific bugs for
the failures.

Jelmer set up to run the check-hottest.py script across everything in
Ubuntu main; it shows things much less complete there, mostly in terms
of making upstream links.

We could go through and get them all to pass but that seems like it
would be kind of missing the point: we want to get people motivated to
be using this and for the tool to make that easy.  The point of doing
hottest100 is to shake out any problems and get some measurement of
what's going wrong.  I see people are now registering imports and
apparently using them.

So where do we go from here?

Specific actions from here at least for the Canonical Bazaar people are:

 * help james_w with some of the bugs opened against the package
importer (assuming he wants it)
 * talk to the Launchpad developers (through bugs or otherwise) about
making the story of adding a new import, package-product link, etc
easier
 * keep running this script at intervals so that we can track what
kind of snags things seem to hit
 * work on udd-related bugs like the issues raised here before, and
the more general stuff about getting multiple branches or UDD-like
merge scenarios

-- 
Martin http://launchpad.net/~mbp/


status
Description: Binary data
-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: hottest100 (was Re: Bazaar focus for 2.1 and 2.2)

2010-01-26 Thread Martin Pool
We're hacking a bit more on this script (in
http://code.launchpad.net/~canonical-bazaar/udd/hottest100) to make it
do things including

* check both the package and upstream branch for freshness and existence
* cross check the package branch against Madison
* understand some of the branches that are special cases of various
kinds (metapackages, obsolete packages etc)
* more...

and in passing we're fixing some misregistration.  At the moment the
biggest problem we can't fix is that some Launchpad projects have
branches but no development focus (ie default branch) and we don't
have permission to change it.  But we are collating that data and
presumably can ask a registry admin to change them.

-- 
Martin http://launchpad.net/~mbp/

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: hottest100 (was Re: Bazaar focus for 2.1 and 2.2)

2010-01-26 Thread Martin Pool
2010/1/26 Martin Pool m...@canonical.com:

 lp:accerciser - lp:~vcs-imports/accerciser/main
 lp:at-spi  - lp:~vcs-imports/at-spi/git-trunk
 lp:ekiga - lp:~vcs-imports/ekiga/git-trunk
 lp:gconf  - lp:~vcs-imports/gconf/trunk
 lp:glib - lp:~jjardon/glib/trunk

all done

 lp:gnome-common - lp:~vcs-imports/gnome-common/trunk

Fixed, but still reported stale because it rarely changes.  So we
might add a tag saying that some things just are expected to move
slowly.

 lp:gnome-cups-manager -  lp:~vcs-imports/gnome-cups-manager/trunk

One interesting thing that check-hottest.py found is that
gnome-cups-manager is not a live package in Ubuntu after Hardy, but
I've updated the branch anyhow.

-- 
Martin http://launchpad.net/~mbp/

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: Feedback on merging via bzr

2010-01-17 Thread Martin Pool
2010/1/18 Scott Kitterman ubu...@kitterman.com:
 I feel like I've gone through a process that is far more complex than the old
 one for no real benefit.  I have some recommendations for improving and
 simplifying this process.  I think simplification is an essential element
 because the learning curve for new contributors is very steep already and
 raising the barrier to entry is not something that will benefit Ubuntu.

Thanks for the feedback, and yes, a major point of this is to make
contribution easier not harder.  So we need to smooth off some fairly
rough edges.

 1.  The most important change would be to have some kind of a wrapper for
 getting the source.  It would be nice to have a script that would download
 branches for the common ancestor, current Ubuntu, and current Debian branches,
 and do the proposed merge.  Without local access to the different versions of
 the package, it is very difficult to know if you've got a correct merge.

 Additionally, if there were checkouts for the previous Debian and Ubuntu
 packages locally, then it should be easy enough to diff the debian directories
 to check for packaging changes when a new upstream version is involved.

That sounds good, and not too hard to do.

 2.  As you no doubt know, the changelog merging could be better and would
 reduce repetitive, boring work potential contributors have to do.

Andrew has a framework for this up for review, so we should be able to
integrate the existing code for debian/changelog merging.

We will probably also cut out the explicit and I think unnecessary
'bzr resolve' step.

-- 
Martin http://launchpad.net/~mbp/

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: hottest100 (was Re: Bazaar focus for 2.1 and 2.2)

2010-01-15 Thread Martin Pool
2010/1/16 Francis J. Lacoste francis.laco...@canonical.com:
 On January 15, 2010, Martin Pool wrote:
 In case people are wondering how far this has come.

 When we started focussing on the hottest100 a month ago we had about
 90 of the hottest100 packages linked to products, and about 52 of them
 had working branches.   Now we have 94 of them linked to products,
 which must be just about all that aren't special cases.  Of those, 64
 now have working branches.  That's pretty good, though I was hoping
 we'd be  a bit higher now, and I'm a bit surprised the second number
 hasn't shifted since Christmas.

 From where do you see this?

The lpstats graph.

 Is
 http://spreadsheets.google.com/ccc?key=0Ag3S65cphSMHdG1VckNSRXI4OHBmVmxGaklGVW4tcWchl=en_GB

 still the place to track this?

 There I do see 94 linked to products, but only 60 branches linked.

 The spreadsheet doesn't have any bug link yet either. Is there another report,
 I should be watching?

I think the spreadsheet should now be obsoleted in favour of having
the hottest100 script track that, either calculating the data it can,
or with things recorded by humans entered into its data file.  That
was my foreshadowed intention but I didn't get around to actually
announcing it on Friday.


 Given that the end goal for this project is to help with daily build but also
 UDD, it would be nice to also see if there is a package branch available for
 each of those. That doesn't change anything for this particular goal, but it
 makes the report more useful.

I think that's what my script checks?

-- 
Martin http://launchpad.net/~mbp/

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: hottest100 (was Re: Bazaar focus for 2.1 and 2.2)

2010-01-14 Thread Martin Pool
In case people are wondering how far this has come.

When we started focussing on the hottest100 a month ago we had about
90 of the hottest100 packages linked to products, and about 52 of them
had working branches.   Now we have 94 of them linked to products,
which must be just about all that aren't special cases.  Of those, 64
now have working branches.  That's pretty good, though I was hoping
we'd be  a bit higher now, and I'm a bit surprised the second number
hasn't shifted since Christmas.

The definition of 'working' here may be a bit loose; I'm working on a
script to scan them and report those which are stale.  This will also
give a better way to record the specific problems with any branches
that should exempt them from this experiment, or the bugs we have to
fix to get them working.

I plan that over the next few weeks we get branches registered for the
rest of them, and then be fixing some more of the import bugs.

-- 
Martin http://launchpad.net/~mbp/

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: hottest100 (was Re: Bazaar focus for 2.1 and 2.2)

2010-01-14 Thread Martin Pool
2010/1/15 Andrew Bennetts andrew.benne...@canonical.com:
 Martin Pool wrote:
 [...]
 The definition of 'working' here may be a bit loose; I'm working on a
 script to scan them and report those which are stale.  This will also

 I suspect that most of the gnome ones are currently stale (hopefully my
 mail from yesterday is a good step towards correcting that...), so I
 think definitely worth checking for staleness in our monitoring of
 hottest100 progress.

My script in https://code.edge.launchpad.net/~canonical-bazaar/udd/hottest100/
claims

   75 ok
   25 unregistered -- meaning Launchpad said there was no default
branch for that package
0 otherwise broken

This may say more about its stupidity than the actual state of these
package branches though; the next step is to make it check freshness.

-- 
Martin http://launchpad.net/~mbp/

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: hottest100 (was Re: Bazaar focus for 2.1 and 2.2)

2010-01-07 Thread Martin Pool
Here are some specific things people can do to help with hottest100:

 * work out how to make package-product links (explain that here :-)
and create them when they're missing
 * update the branches pointing to obsolete imports (gnome etc)
 * write a script that checks the date of the last revision on these
branches and complains if it's more than say 14 days old - it probably
indicates the import is stalled or points to an obsolete branch
 * progress the bugs mentioned in this thread and/or tag them hottest100
 * scan the failing branches, determine which bug causes the failure,
mention the branch in the bug and put the bug number in the
spreadsheet against that branch
 * work out if there is an api to get the import failure messages, and
use it to automatically match up the tracebacks against the bugs

-- 
Martin http://launchpad.net/~mbp/

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: hottest100 (was Re: Bazaar focus for 2.1 and 2.2)

2010-01-07 Thread Martin Pool
2010/1/8 James Westby jw+deb...@jameswestby.net:
 On Fri, 8 Jan 2010 10:58:04 +1100, Martin Pool m...@canonical.com wrote:
 Here are some specific things people can do to help with hottest100:

  * work out how to make package-product links (explain that here :-)
 and create them when they're missing

 Just a note that

  https://edge.launchpad.net/ubuntu/+upstreamreport

 has live data to help with this.

 Missing corresponding project means that a package-product link needs
 to be created (and sometimes a product too). The lack of a Bazaar icon
 means there is no default branch.

 Note that I went over this list a few weeks back registered a bunch of
 imports, and created the LP question to have them set as the development
 focus. That's still not done, and that's why I keep bringing it up,
 because you will often be duplicating effort if you look in to them.

Thanks for pointing that out.  Other things on my list may be
redundant with other posts too, so my list was more of threads for
people to pull on.

 This says nothing about how up-to-date all that information is though. A
 (preferably semi-automatic) check that everything is up-to-date would be
 good.

 This does raise other questions in my mind. Are we excluding some
 packages before we really start?

I don't know.  I guess we only would be if the top 100 list was not
actually the 100 most important packages.  I'm not actually sure where
that list comes from.

Ultimately there are more than 100 packages that matter, and starting
with 100 fairly important packages is reasonable.  We can go on to
more later.

 Is implementing bzr-monotone going to
 be something that falls under the hottest100 project?

If some of them are in mt that will be useful data.  I doubt it will
be so many of them that it represents a good cost/benefit tradeoff,
though it might be worth doing a fastimport from mt.  I think it's
reasonable for us to conclude this project with some number of
packages essentially classed 'wontfix (for now)', if they would
require a lot of work that would not help many packages.  But there
should be few like that, and it should be a specific justified
decision.

 There are also a
 few cases where we have two packages for one upstream repo, and some
 where there isn't really an upstream (aside from things like
 update-manager, linux-restricted-modules for instance). What will be
 done about the KDE packages, their upstream is one mega-repo.

I don't know yet.  I'd like to start by getting those packages marked
with a bug number that describes the situation.  The larger point of
hottest100 is to allow daily builds of those packages.  If there is no
upstream I don't know if the question even means anything.  If there
is an upstream but it's quirky again perhaps it's reasonable to say we
just don't support it yet.

-- 
Martin http://launchpad.net/~mbp/

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: Bazaar focus for 2.1 and 2.2

2010-01-04 Thread Martin Pool
2010/1/5 Ian Clatworthy ian.clatwor...@canonical.com:
 Martin Pool wrote:
 I think after the break we should focus on the vcs-imports of the top
 100 Ubuntu packages until they're all working well.  jml and spm
 helped with some scripts to query their current state, and we can map
 that into a spreadsheet showing the root cause for each failure.

OK, so let's do it!

Following Kiko's maximum about naming, I dub this Hottest 100.

It will help make daily builds useful, it should shake out some useful
bugs, and we may be able to get good visible progress quite soon.

I'd ask people on the Bazaar team to be putting most of their forward
development time into this, either analysis or actually fixing the
bugs that come out.

So what do we do now?

 * start tagging bugs hottest100 if fixing them will let us get more
branches imported
 * fix those bugs
 * jml gave me a data dump of the package states; I'll put that up on
the web in an accessible form
 * let's mark bugs against each of those failing imports
 * there are some items posted earlier in this thread that we should
chase up as far as registering branches, development focuses, etc,
especially https://answers.edge.launchpad.net/launchpad-code/+question/81994
(remove gnome svn imports) and
https://answers.edge.launchpad.net/launchpad/+question/94298 (put
some git branches into production)

-- 
Martin http://launchpad.net/~mbp/

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


hottest100 (was Re: Bazaar focus for 2.1 and 2.2)

2010-01-04 Thread Martin Pool
I put jml's query output into a Google spreadsheet, so that we can
annotate lines with the relevant bug etc.
http://spreadsheets.google.com/ccc?key=0Ag3S65cphSMHdG1VckNSRXI4OHBmVmxGaklGVW4tcWchl=en_GB.

Some observations:

Some aren't linked to products; that's probably easily fixed.

In some cases multiple packages are linked to the same product, like
various versions of grub, firefox, and linux.  That's probably ok.
However, some are linked to the same branch, which is probably wrong.

To be useful, this query needs some measurement of whether the branch is fresh.

By matching up the branch names against the previously posted failure
report, we can either identify some things thought to be working but
actually not, or put specific bugs against things that are failing.

So that we don't shift the goalposts we could keep the list of
packages the same across this project.  Maybe we should put it into a
file in a branch in the udd project, then we can use it in scripts
that prod Launchpad to find out the freshness of the branches etc.
Does someone want to try this?

-- 
Martin http://launchpad.net/~mbp/

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: Bazaar focus for 2.1 and 2.2

2009-12-21 Thread Martin Pool
I think after the break we should focus on the vcs-imports of the top
100 Ubuntu packages until they're all working well.  jml and spm
helped with some scripts to query their current state, and we can map
that into a spreadsheet showing the root cause for each failure.

I'll ask the Bazaar team to put most of their forward-development time
into this until they're either all working or we've specifically
chosen something else.  We won't ignore other things people bring up
but we won't look for trouble elsewhere til we're done.

We can focus on this at our mini-sprint in Strasbourg and I hope pair
with Jelmer on debugging some of the failures.

-- 
Martin http://launchpad.net/~mbp/

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: Bazaar focus for 2.1 and 2.2

2009-12-17 Thread Martin Pool
2009/12/18 John Arbash Meinel j...@arbash-meinel.com:
 James Westby wrote:
 On Thu, 17 Dec 2009 14:19:32 +1100, Martin Pool m...@canonical.com wrote:
 My hypothesis is that we (Canonical's Bazaar team) will get to grips
 with UDD better if there is a tighter medium-term focus.

 The key question is whether vcs-imports is that thing, or not.

 If we do make imports that one important thing then I'll be asking
 people not to work on looms, UDD merge support, or nested trees until
 they're done.

 What would you consider done. Do you think that with 4 months effort
 you could get to 0 outstanding failures, or would there be some other
 stop point?

 I would guess that needs to be evaluated on the fly. If we are getting
 to the point of diminishing returns, then it is reasonable to
 re-evaluate what our priority should be.

+1

The input data is messy enough that we may never get it to zero
failures, so maybe done is a poor word compared to enough: defined
as the point where other things are becoming clearly more important,
or the remaining bugs have a poor cost:benefit.  We may actually
already be at that point now.

Since it's a long tail problem, we don't want to set up our goal as
having zero failures or doing vcs-imports as an end in itself, but
rather to relate it to larger needs.  Thus this thread.

There's some up front cost to getting into the analysis of import
failures, so if we do focus on that we would want to persist with it
for a few months.  It's hard to estimate until we get further into it
but it's possible that four months would get many of them cleared.
It's a fairly large commitment and opportunity cost.

I do think Elliot has a good point though, that fixing imports is not
going to really teach us important things about the structure of the
problem.  If we had UDD or build-from-branch totally working on some
packages, but people were blocked from using it widely because of
failing imports, then it would be clearly the right time to do it.

-- 
Martin http://launchpad.net/~mbp/

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: Rewriting Ubuntu branches

2009-12-13 Thread Martin Pool
2009/12/14 James Westby jw+deb...@jameswestby.net:
 Hi,

 I wanted to split this out of the large mail so that we could complete
 a design of how this would work.

 Here's my initial proposal based on the feedback from that thread.

  1) bzr-builddeb decorates pull, merge and perhaps a couple of other
     commands to catch the diverged error, and check whether the branch
     has been rewritten so that we can prompt the user with the correct
     way to proceed.

In general whenever we're decorating commands we should ask whether
there is a more appropriate place to do the extension.  I think here
perhaps it would be better to add a hook called when an operation is
going to fail due to divergence, giving it the chance to clean it up.
That could be useful to other people.

  2) We ship map files in some known location, package-import.ubuntu.com say.
     These contain revision id pairs and file id pairs corresponding to
     the old branches and the new branches. Is just listing plain file ids
     safe, or do we need (revision id, file id) pairs to do this correctly.

I'm not sure I understand, perhaps due to reading this out of order.
If you want to express: 'revision id R with inventory I is equivalent
to R' with I'' then just listing the filed ids should be enough?


  3) Either bzr-rewrite itself, of something based on the code in there
     will be used to rewrite a given branch using the information in the
     map files.

 Open questions:

  * Should we bother with file id maps, or just match based on path?
  * Should we put the old revision ids in to revision properties of
    the new revisions? This could be the marker for the decorated
    commands, and make the map files smaller. It does inflate the
    size of the branches for ever though. If we don't, what should
    the marker be?

It sounds like this is going towards a more general 'replaces'
relationship between revisions, which ought to be treated as part of
the merge graph but generally not shown in logs etc?

-- 
Martin http://launchpad.net/~mbp/

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: your thoughts wanted on bzr team UDD focus

2009-12-03 Thread Martin Pool
2009/12/4 Francis J. Lacoste francis.laco...@canonical.com:
 On December 3, 2009, Martin Pool wrote:
 If there are existing bugs relevant to udd, or you know of
 appropriately concrete and self-contained things that can be filed as
 bugs, then tagging them and/or mentioning them here would be helpful.
 It would give us something to be getting on with.  But I agree the
 larger issues are too broad to make useful bugs now.  (One could have
 placeholder bugs like work out what to do about X but I doubt that
 helps.)


 Actually, given the workflow you guys seem to favour, I think it might be
 sense. Otherwise, how to do you track and make sure that somebody drives the
 requirements process on these larger issues?

Momentum on the udd list, plus James's specs?  Or maybe we should do
it, at least as bugs against the udd project.

So I'll refine that position a bit to: those bugs are ok as long as
they're things we've agreed are reasonably in scope and things we're
actively working on.  If they're not being worked on, they just seems
like clutter that causes confusion later on.  (It's not quite the same
thing but Brian's confusion to do with finding old Launchpad
blueprints about communication which are half-implemented or obsolete
or generally detached from reality is the kind of thing I'd like to
avoid.)  Bugs which are not clearly falsifiable and not moving towards
being clear are a drag.

-- 
Martin http://launchpad.net/~mbp/

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: your thoughts wanted on bzr team UDD focus

2009-12-02 Thread Martin Pool
2009/12/3 James Westby james.wes...@canonical.com:
 On Mon Nov 30 20:19:13 -0500 2009 Martin Pool wrote:
 I'd like to get a sanity check from UDD people on what the Bazaar team
 is going to do for our 2.1 release, which will have a feature freeze
 in February and go into Karmic.

 From the previous threads, it seems that the main large things we need
 to do are:

 Hi,

 Thanks for requesting feedback.

 There were a bunch of other things discussed, and scenarios considered,
 and these two items are the only things that came out the other end. You
 are obviously the one that should decide how much work to take on given
 the developer time available, so I'm not complaining that the list is
 short. I do however wonder about the other features that were discussed.
 Did you decide that these gave the best return on investment, or was
 it just that these were the items most discussed?

Well, what I posted was more of a strawman of where we want to end up,
limiting it to the things that seemed concrete and short-term
feasible, and it's true somewhat impatiently shortcircuiting the
discussion.  But I would be very very happy to hear from you, or from
others, different nominations for that list and something about why.
I suppose this will come out of your UDS specs.

  * get more code imports working -- investigation of failure causes is
 continuing

 Are you setting any sort of targets, or is it too early to know
 how varied the causes are and so know what a sensible target would
 be?

Let me point you to the thread Yo code team -- how do you want your
stats? on canonical-launchpad (for no very good reason; I think
generally it should be public.)  There is some triage and conversion
into bugs.  We haven't yet targeted them.

-- 
Martin http://launchpad.net/~mbp/

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: udd team on launchpad

2009-11-22 Thread Martin Pool
If by udd you mean ~ubuntu-distributed-devel, then it was an abandoned
attempt to create a team list.  (Apparently for unclear reasons this needs
to be on lists.ubuntu.com.)  I'll delete it.

- Martin
-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel