Re: hottest100 (was Re: Bazaar focus for 2.1 and 2.2)

2010-01-15 Thread John Arbash Meinel
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Martin Pool wrote:
 2010/1/15 Andrew Bennetts andrew.benne...@canonical.com:
 Martin Pool wrote:
 [...]
 The definition of 'working' here may be a bit loose; I'm working on a
 script to scan them and report those which are stale.  This will also
 I suspect that most of the gnome ones are currently stale (hopefully my
 mail from yesterday is a good step towards correcting that...), so I
 think definitely worth checking for staleness in our monitoring of
 hottest100 progress.
 
 My script in 
 https://code.edge.launchpad.net/~canonical-bazaar/udd/hottest100/
 claims
 
75 ok
25 unregistered -- meaning Launchpad said there was no default
 branch for that package
 0 otherwise broken
 
 This may say more about its stupidity than the actual state of these
 package branches though; the next step is to make it check freshness.
 

I went ahead and cleaned it up a little bit. (It now uses OptionParser,
etc.) I also changed the formatting a bit and added a 14-day stale
counter. (Check the last rev, if it is  14 days old, count it as stale.)

We can easily set the number of days.

Anyway, with that flag, I get:

Also note that we have some odd bits in there. Like we have *both*
'mozilla-thunderbird' and 'thunderbird' listed (former doesn't have a
branch, latter does, but it is 150 days old.)

Also, we have firefox, firefox-3.0 and firefox-3.5. The first doesn't
have a branch, but the last two do.

With a 14 day threshold we end up with:
TOTALS:
   26 ok
   49 stale
   25 unregistered
0 otherwise broken


14 days is probably a little tight. A few entries are in the 14-20 day
range. However we have a bunch of ones that are closer to 100+ days old,
and I'm pretty sure those are genuinely out of date.

John
=:-
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAktQl1MACgkQJdeBCYSNAAPz2gCcCK8RFXGyIBghqX/ilXBfM5rr
GSIAn0EHzRH2RSX6RUnBWC0CFvXgeBRN
=9r2B
-END PGP SIGNATURE-

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: New overview page for the Bazaar importer

2010-01-15 Thread Francis J. Lacoste
On January 7, 2010, James Westby wrote:
 This of course is designing for failure, which is vital. However, it's
 the scale of the issue that has taken some getting used to. It may be a
 matter of magnitude, with a lot of API calls being made, so even a small
 failure rate translates in to a lot of issues, but it still seems like a
 lot. I have of course been filing bugs on LP about issues that I can
 identify, and spent some time today provoking bad responses and digging
 in to the reasons to file some more bugs. It seems that a lot of the
 problem now is the appservers refusing to communicate though, and I'm
 not sure there's a lot I can do on my end to debug that.

We are currently experiencing an issue on the servers which causes timeout for 
non-obvious reason. (In some loaded conditions, getting the lag in the cluster 
is taking way too much time. Normally, this operation is done in a blink. We 
are working on a fix.)

 
 If you look at the list of the last 100 failures you will probably see
 some of this with clusters of the same signature, usually pointing to
 network communication in some manner.
 

Looking at the signature today, I only see the first one as network related:
2 packages failed to many times to retry with key 
launchpadlib.errors.HTTPError:module:main:get_versions:iterate_collection:get_collection_slice:get:_request
 

Unless some of the root cause behind the other signatures is network related, 
but from the signature itself, it's not obvious.

-- 
Francis J. Lacoste
francis.laco...@canonical.com


signature.asc
Description: This is a digitally signed message part.
-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel


Re: hottest100 (was Re: Bazaar focus for 2.1 and 2.2)

2010-01-15 Thread Martin Pool
2010/1/16 Francis J. Lacoste francis.laco...@canonical.com:
 On January 15, 2010, Martin Pool wrote:
 In case people are wondering how far this has come.

 When we started focussing on the hottest100 a month ago we had about
 90 of the hottest100 packages linked to products, and about 52 of them
 had working branches.   Now we have 94 of them linked to products,
 which must be just about all that aren't special cases.  Of those, 64
 now have working branches.  That's pretty good, though I was hoping
 we'd be  a bit higher now, and I'm a bit surprised the second number
 hasn't shifted since Christmas.

 From where do you see this?

The lpstats graph.

 Is
 http://spreadsheets.google.com/ccc?key=0Ag3S65cphSMHdG1VckNSRXI4OHBmVmxGaklGVW4tcWchl=en_GB

 still the place to track this?

 There I do see 94 linked to products, but only 60 branches linked.

 The spreadsheet doesn't have any bug link yet either. Is there another report,
 I should be watching?

I think the spreadsheet should now be obsoleted in favour of having
the hottest100 script track that, either calculating the data it can,
or with things recorded by humans entered into its data file.  That
was my foreshadowed intention but I didn't get around to actually
announcing it on Friday.


 Given that the end goal for this project is to help with daily build but also
 UDD, it would be nice to also see if there is a package branch available for
 each of those. That doesn't change anything for this particular goal, but it
 makes the report more useful.

I think that's what my script checks?

-- 
Martin http://launchpad.net/~mbp/

-- 
ubuntu-distributed-devel mailing list
ubuntu-distributed-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel