On 2014-02-28 17:24 , Hal Wine wrote: > tl;dr: what is the balance point between pushes to try taking too long > and loosing repository history of recent try pushes? Based on the responses to this specific question, we'll go back to waiting for developers to notify IT when there is enough performance impact to warrant a reset of the try repository. I've added the reporting instructions to the wiki page about try: https://wiki.mozilla.org/ReleaseEngineering/TryServer <https://wiki.mozilla.org/ReleaseEngineering/TryServer#Pushes_to_try_take_a_very_long_time>
Thanks to everyone else for showing interest in the underlying problems. Suggestions for that are best added to the bugs cited below. Thanks! --Hal > > Summary: > -------- > > As most developers have experienced, pushing to try can sometimes take > a long time. Once it takes "too long" (as measured by screams of pain > in #releng) <https://etherpad.mozilla.org/ep/search?query=releng%29>, > a "try [repository] reset" is scheduled. This hurts productivity and > increases frustration for everyone involved (devs, IT, RelEng). We > don't want to do this anymore. > > A reset of the try repository deletes the existing contents, and > replaces with a fresh clone from mozilla-central. While the tbpl > information will remain valid for any completed build, any attempt to > view the diffs for a try build will fail (unless you already had them > in your local repository). > > Progress on resolution of the root cause: > ----------------------------------------- > > IT has made tremendous progress in reducing the occurrence of "long > push times", but they still are not predictable. Various attempts at > monitoring[1] and auto correction[2] have not been successful in > improving the situation. Work continues on additional changes that > should improve the situation[3]. > > The most recent mitigation strategy is to trade the "unknown timing" > disruption of the push times increasing to a pain threshold with a > "known timing" of reseting the try repository every TCW (tree closing > window - every 6 wks currently). However, we heard from some folks > that this is too often. > > The most recent try-reset-triggered-by-pain was a duration of 6 > months[4]. There was at least one report just 3 months after reset of > problems[5]. > > So, the question is - what say developers -- what's the balance point > between: > - too often, making collaborating on try pushes hard > - too infrequent, introducing increasing push times > > --Hal > > Prior Work: > ----------- > [1] bug https://bugzil.la/691459 > [2] bugs https://bugzil.la/554656https://bugzil.la/734225 > <https://bugzil.la/734225#c24>#c24 > <https://bugzil.la/734225#c24>https://bugzil.la/633161https://bugzil.la/529179 > [3] bugs https://bugzil.la/770811https://bugzil.la/937732others > [4] bugs https://bugzil.la/894429&; https://bugzil.la/962275 > [5] bug https://bugzil.la/925354 > _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
