Hi, I may be alone here, but I actually found build/test(s) failure emails to be super useful. The part that I did not find useful was that each time a build/test(s) would fail on a particular platform, there would be two comments/emails, which furthermore split the useful information into two pieces. IIRC, the first comment/email would contain information about what build/test(s) failed, and the second comment/email was just an archive upload of the layout test results (if applicable).
I would find it very useful to get an email each time one of the bots had something go "wrong", and ideally include a link directly to the 'errors' log that Aakash mention in the beginning of this thread (or to the layout test results in the case of a test(s) failure). Right now, if something goes "wrong", I have to be the one to discover it and then click through a few pages until I find the data that will actually help me solve the problem. As a patch author, this is useful in letting me not have to remember to check my patch once a day in order to see if there are any red bubbles (something which I've forgotten to do, leaving a broken patch up for review thinking all was fine). I can upload a patch and ideally leave it be, as I will be notified if the build/test(s) fails. As a patch reviewer, I'm not sure this would be as useful, as I have a saved query that I use for finding all patches that need to be reviewed, and on a search result page there are no EWS bubbles anyways, so I can't see the info there. I have to click into the bug regardless. To Ryosuke's point, it may still be useful to get an email as a reviewer simply so I know that something failed and can potentially remember that fact before I click on a bug in the search results page, but I know personally that my mind doesn't really work that way :P Thanks, Devin > On Nov 5, 2019, at 09:04, Alexey Proskuryakov <a...@webkit.org> wrote: > > > >> 4 нояб. 2019 г., в 1:37 PM, Ryosuke Niwa <rn...@webkit.org >> <mailto:rn...@webkit.org>> написал(а): >> >> >> On Mon, Nov 4, 2019 at 9:40 AM Alexey Proskuryakov <a...@webkit.org >> <mailto:a...@webkit.org>> wrote: >> >> Can you elaborate on that, how exactly is e-mailing on first failure useful >> to reviewers? >> >> Getting rid of Bugzilla comments was one of the goals of EWS rewrite, based >> on engineering feedback about noise in bugs and in e-mail, and I >> wholeheartedly agree with this feedback. So I think that comments are >> generally undesirable. >> >> Since I don't understand what your precise scenario is, I may be make straw >> man arguments below, but here are some things that I think make the proposed >> behavior unhelpful (add a comment on first failure, or when all EWSes pass). >> >> 1. EWS comments in Bugzilla are so annoying that some people take the >> radical step of manually hiding them. EWS history is archived anyway, there >> is no need to look into comments for it. >> >> 2. There are often many people CC'ed on the bug to whom EWS data is >> irrelevant or even mysterious (e.g. reporters, web developers or >> non-reviewers). The noise is a slight annoyance, discouraging further >> participation in the project. >> >> 3. I believe that for most reviewers, the mode of operation is one of the >> two: (1) do it when pinged directly, or (2) go over the review queue when >> one has the time. Getting EWS comments helps neither. >> >> 4. Commenting when all EWSes pass is not very practical - it's too often >> that we have some stragglers that take days (or forever). I don't think that >> we can make it reliable even if we start actively policing EWS >> responsiveness. >> >> 5. The reviewer likely wants to know the state of multiple EWSes if they are >> going to wait for EWS at all. What exactly are they going to do after >> getting an e-mail that one EWS failed? >> >> I often use a EWS failure as a signal to wait reviewing a patch. Otherwise, >> a bug mail will stay in my inbox as one of items to get to. >> >> I can see the usefulness in the somewhat unusual case of a super urgent >> patch. We may want multiple people to watch it, so that members of CC list >> would go and ask the patch author to update it with more urgency than e-mail >> allows for. I think that opt-in is a better mechanism for that, so that >> people who opted in would receive information about each EWS data point. >> >> I think there is a value in knowing that a patch isn't ready instead of >> having to open the bug to realize that. > > So just to clarify, > > - a major part of how you get to review bugs is by being CC'ed, and you > review them when you have the time to read bugmail; > - and you don't open the bug in Bugzilla if there is already an EWS failure > by the time you read the e-mail where review is requested? > > That's clearly a valid benefit. In my mind, it probably doesn't outweigh the > downsides. On the other hand, yours is a voice of someone who reviews way > more patches than Maciej and me combined these days, so maybe more e-mail is > an overall benefit to many of the reviewers. > > - Alexey > > > >> - R. Niwa >>> 3 нояб. 2019 г., в 6:58 PM, Maciej Stachowiak <m...@apple.com >>> <mailto:m...@apple.com>> написал(а): >>> >>> >>> I think they are useful to actual and potential reviewers. Direct email to >>> the patch author is not something anyone can Cc themselves on, and is not >>> archived, so seems like a strictly worse form of communication. >>> >>>> On Nov 2, 2019, at 9:34 AM, Alexey Proskuryakov <a...@apple.com >>>> <mailto:a...@apple.com>> wrote: >>>> >>>> >>>> My preference is still e-mailing the patch author directly (possibly, also >>>> having an option to opt in for anyone). Bugzilla comments will always be >>>> irrelevant for most people CC'ed on the bug, and they are almost always >>>> undesirable to keep within the discussion flow. >>>> >>>> - Alexey >>>> >>>>> 1 нояб. 2019 г., в 18:28, Aakash Jain <aakash_j...@apple.com >>>>> <mailto:aakash_j...@apple.com>> написал(а): >>>>> >>>>> Sounds good. I prefer the single comment when the first failure occur. >>>>> That way notification would be sent as soon as the first failure happens. >>>>> >>>>> I'll implement that (assuming it's acceptable to everyone). >>>>> >>>>> Thanks >>>>> Aakash >>>>> >>>>>> On Nov 1, 2019, at 8:35 PM, Maciej Stachowiak <m...@apple.com >>>>>> <mailto:m...@apple.com>> wrote: >>>>>> >>>>>> >>>>>> How about only a single comment when the first failure occurs? (Or else >>>>>> when all bots pass, if there is never a failure.) >>>>>> >>>>>> This should help the author, the reviewer, and anyone else cc’d, without >>>>>> being too spammy. >>>>>> >>>>>>> On Nov 1, 2019, at 5:20 PM, Aakash Jain <aakash_j...@apple.com >>>>>>> <mailto:aakash_j...@apple.com>> wrote: >>>>>>> >>>>>>> Hi Ryosuke, >>>>>>> >>>>>>> Many people didn't like the noise by the EWS comments, and we removed >>>>>>> the comments as per previous discussion in: >>>>>>> https://lists.webkit.org/pipermail/webkit-dev/2019-June/030683.html >>>>>>> <https://lists.webkit.org/pipermail/webkit-dev/2019-June/030683.html>. >>>>>>> >>>>>>> I agree with your point that having some kind of notification might be >>>>>>> useful. >>>>>>> >>>>>>> I proposed some ideas in >>>>>>> https://lists.webkit.org/pipermail/webkit-dev/2019-September/030798.html >>>>>>> >>>>>>> <https://lists.webkit.org/pipermail/webkit-dev/2019-September/030798.html>, >>>>>>> but didn't get much feedback. If we can all agree on a solution, I can >>>>>>> look into implementing it. >>>>>>> >>>>>>> Thanks >>>>>>> Aakash >>>>>>> >>>>>>>> On Oct 30, 2019, at 1:03 AM, >>>>>>>> - R. Niwa >>>>>>>> <rn...@webkit.org <mailto:rn...@webkit.org>> wrote: >>>>>>>> >>>>>>>> These enhancements are great. There is one feature of the old EWS that >>>>>>>> I really miss, which is that I used to get emails when some EWS >>>>>>>> failed. With new EWS, I have to keep checking back the bugzilla to see >>>>>>>> if any of them have failed periodically. >>>>>>>> >>>>>>>> Can we add a feature to opt into such an email notification? Maybe a >>>>>>>> flag on a patch or JSON configuration file somewhere. >>>>>>>> >>>>>>>> - R. Niwa >>>>>>>> >>>>>>>> On Tue, Oct 29, 2019 at 4:05 PM Aakash Jain <aakash_j...@apple.com >>>>>>>> <mailto:aakash_j...@apple.com>> wrote: >>>>>>>> Hi Everyone, >>>>>>>> >>>>>>>> I am happy to announce another EWS feature. >>>>>>>> >>>>>>>> From now on, in case of build failure, EWS will parse the errors and >>>>>>>> display them in a separate 'errors' log. You wouldn't have to search >>>>>>>> through thousands of lines of logs to find the error message. >>>>>>>> >>>>>>>> For example, in https://ews-build.webkit.org/#/builders/16/builds/6054 >>>>>>>> <https://ews-build.webkit.org/#/builders/16/builds/6054>, in step #7 >>>>>>>> WebKit failed to compile. Complete logs (stdio) are 38,000+ lines, and >>>>>>>> the error is not at the end of the logs. Normally, it requires some >>>>>>>> searching through the logs to find the relevant errors. But now, there >>>>>>>> is another 'errors' log, which contains just the relevant 11 lines >>>>>>>> (containing error and few related lines to provide additional context). >>>>>>>> >>>>>>>> Hopefully this would save some time and efforts previously spent on >>>>>>>> searching through the large logs. >>>>>>>> >>>>>>>> Note that this information is not displayed in status-bubble tool-tip, >>>>>>>> since this might be lot of text to display in the tooltip. My further >>>>>>>> plan is to make this information more readily available, by adding it >>>>>>>> to a custom designed page which will open on clicking the status >>>>>>>> bubble https://webkit.org/b/197522 <https://webkit.org/b/197522> >>>>>>>> >>>>>>>> Please let me know if you notice any issues or have any feedback. >>>>>>>> >>>>>>>> Thanks >>>>>>>> Aakash >>>>>>>> >>>>>>>> Reference: https://webkit.org/b/203418 <https://webkit.org/b/203418> >>>>>>>> _______________________________________________ >>>>>>>> webkit-dev mailing list >>>>>>>> webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org> >>>>>>>> https://lists.webkit.org/mailman/listinfo/webkit-dev >>>>>>>> <https://lists.webkit.org/mailman/listinfo/webkit-dev> >>>>>>>> -- >>>>>>>> - R. Niwa >>>>>>> >>>>>>> _______________________________________________ >>>>>>> webkit-dev mailing list >>>>>>> webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org> >>>>>>> https://lists.webkit.org/mailman/listinfo/webkit-dev >>>>>>> <https://lists.webkit.org/mailman/listinfo/webkit-dev> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> webkit-dev mailing list >>>>> webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org> >>>>> https://lists.webkit.org/mailman/listinfo/webkit-dev >>>>> <https://lists.webkit.org/mailman/listinfo/webkit-dev> >>>> >>>> - Alexey >>>> >>> >> >> >> _______________________________________________ >> webkit-dev mailing list >> webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org> >> https://lists.webkit.org/mailman/listinfo/webkit-dev >> <https://lists.webkit.org/mailman/listinfo/webkit-dev> > > > _______________________________________________ > webkit-dev mailing list > webkit-dev@lists.webkit.org > https://lists.webkit.org/mailman/listinfo/webkit-dev
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
