Re: [webkit-dev] EWS Comments on Bugzilla (Was: EWS now parses error logs in case of build failure)

[Ryosuke Niwa]

Great. I've completed the survey.

- R. Niwa

On Mon, Dec 2, 2019 at 5:19 PM Aakash Jain  wrote:

> There were multiple ideas discussed in this thread. I would like to gather
> more data about what do most people prefer. I have sent out a short survey
> in https://lists.webkit.org/pipermail/webkit-dev/2019-December/030980.html
>
> Thanks
> Aakash
>
> On Nov 5, 2019, at 12:04 PM, Alexey Proskuryakov  wrote:
>
>
>
> 4 нояб. 2019 г., в 1:37 PM, Ryosuke Niwa  написал(а):
>
>
> On Mon, Nov 4, 2019 at 9:40 AM Alexey Proskuryakov  wrote:
>
>>
>> Can you elaborate on that, how exactly is e-mailing on first failure
>> useful to reviewers?
>>
>> Getting rid of Bugzilla comments was one of the goals of EWS rewrite,
>> based on engineering feedback about noise in bugs and in e-mail, and I
>> wholeheartedly agree with this feedback. So I think that comments are
>> generally undesirable.
>>
>> Since I don't understand what your precise scenario is, I may be make
>> straw man arguments below, but here are some things that I think make the
>> proposed behavior unhelpful (add a comment on first failure, or when all
>> EWSes pass).
>>
>> 1. EWS comments in Bugzilla are so annoying that some people take the
>> radical step of manually hiding them. EWS history is archived anyway, there
>> is no need to look into comments for it.
>>
>> 2. There are often many people CC'ed on the bug to whom EWS data is
>> irrelevant or even mysterious (e.g. reporters, web developers or
>> non-reviewers). The noise is a slight annoyance, discouraging further
>> participation in the project.
>>
>> 3. I believe that for most reviewers, the mode of operation is one of the
>> two: (1) do it when pinged directly, or (2) go over the review queue when
>> one has the time. Getting EWS comments helps neither.
>>
>> 4. Commenting when all EWSes pass is not very practical - it's too often
>> that we have some stragglers that take days (or forever). I don't think
>> that we can make it reliable even if we start actively policing EWS
>> responsiveness.
>>
>> 5. The reviewer likely wants to know the state of multiple EWSes if they
>> are going to wait for EWS at all. What exactly are they going to do after
>> getting an e-mail that one EWS failed?
>>
>
> I often use a EWS failure as a signal to wait reviewing a patch.
> Otherwise, a bug mail will stay in my inbox as one of items to get to.
>
> I can see the usefulness in the somewhat unusual case of a super urgent
>> patch. We may want multiple people to watch it, so that members of CC list
>> would go and ask the patch author to update it with more urgency than
>> e-mail allows for. I think that opt-in is a better mechanism for that, so
>> that people who opted in would receive information about each EWS data
>> point.
>>
>
> I think there is a value in knowing that a patch isn't ready instead of
> having to open the bug to realize that.
>
>
> So just to clarify,
>
> - a major part of how you get to review bugs is by being CC'ed, and you
> review them when you have the time to read bugmail;
> - and you don't open the bug in Bugzilla if there is already an EWS
> failure by the time you read the e-mail where review is requested?
>
> That's clearly a valid benefit. In my mind, it probably doesn't outweigh
> the downsides. On the other hand, yours is a voice of someone who reviews
> way more patches than Maciej and me combined these days, so maybe more
> e-mail is an overall benefit to many of the reviewers.
>
> - Alexey
>
>
>
> - R. Niwa
>
>> 3 нояб. 2019 г., в 6:58 PM, Maciej Stachowiak  написал(а):
>>
>>
>> I think they are useful to actual and potential reviewers. Direct email
>> to the patch author is not something anyone can Cc themselves on, and is
>> not archived, so seems like a strictly worse form of communication.
>>
>> On Nov 2, 2019, at 9:34 AM, Alexey Proskuryakov  wrote:
>>
>>
>> My preference is still e-mailing the patch author directly (possibly,
>> also having an option to opt in for anyone). Bugzilla comments will always
>> be irrelevant for most people CC'ed on the bug, and they are almost always
>> undesirable to keep within the discussion flow.
>>
>> - Alexey
>>
>> 1 нояб. 2019 г., в 18:28, Aakash Jain  написал(а):
>>
>> Sounds good. I prefer the single comment when the first failure occur.
>> That way notification would be sent as soon as the first failure happens.
>>
>> I'll implement that (assuming it's acceptable to everyone).
>>
>> Thanks
>> Aakash
>>
>> On Nov 1, 2019, at 8:35 PM, Maciej Stachowiak  wrote:
>>
>>
>> How about only a single comment when the first failure occurs? (Or else
>> when all bots pass, if there is never a failure.)
>>
>> This should help the author, the reviewer, and anyone else cc’d, without
>> being too spammy.
>>
>> On Nov 1, 2019, at 5:20 PM, Aakash Jain  wrote:
>>
>> Hi Ryosuke,
>>
>> Many people didn't like the noise by the EWS comments, and we removed the
>> comments as per previous discussion in:
>> https://lists.webkit.org/pipermail/webkit-dev/2019-Jun

Re: [webkit-dev] EWS Comments on Bugzilla (Was: EWS now parses error logs in case of build failure)

[Aakash Jain]

There were multiple ideas discussed in this thread. I would like to gather more 
data about what do most people prefer. I have sent out a short survey in 
https://lists.webkit.org/pipermail/webkit-dev/2019-December/030980.html 


Thanks
Aakash

> On Nov 5, 2019, at 12:04 PM, Alexey Proskuryakov  wrote:
> 
> 
> 
>> 4 нояб. 2019 г., в 1:37 PM, Ryosuke Niwa > > написал(а):
>> 
>> 
>> On Mon, Nov 4, 2019 at 9:40 AM Alexey Proskuryakov > > wrote:
>> 
>> Can you elaborate on that, how exactly is e-mailing on first failure useful 
>> to reviewers?
>> 
>> Getting rid of Bugzilla comments was one of the goals of EWS rewrite, based 
>> on engineering feedback about noise in bugs and in e-mail, and I 
>> wholeheartedly agree with this feedback. So I think that comments are 
>> generally undesirable.
>> 
>> Since I don't understand what your precise scenario is, I may be make straw 
>> man arguments below, but here are some things that I think make the proposed 
>> behavior unhelpful (add a comment on first failure, or when all EWSes pass).
>> 
>> 1. EWS comments in Bugzilla are so annoying that some people take the 
>> radical step of manually hiding them. EWS history is archived anyway, there 
>> is no need to look into comments for it.
>> 
>> 2. There are often many people CC'ed on the bug to whom EWS data is 
>> irrelevant or even mysterious (e.g. reporters, web developers or 
>> non-reviewers). The noise is a slight annoyance, discouraging further 
>> participation in the project.
>> 
>> 3. I believe that for most reviewers, the mode of operation is one of the 
>> two: (1) do it when pinged directly, or (2) go over the review queue when 
>> one has the time. Getting EWS comments helps neither.
>> 
>> 4. Commenting when all EWSes pass is not very practical - it's too often 
>> that we have some stragglers that take days (or forever). I don't think that 
>> we can make it reliable even if we start actively policing EWS 
>> responsiveness.
>> 
>> 5. The reviewer likely wants to know the state of multiple EWSes if they are 
>> going to wait for EWS at all. What exactly are they going to do after 
>> getting an e-mail that one EWS failed?
>> 
>> I often use a EWS failure as a signal to wait reviewing a patch. Otherwise, 
>> a bug mail will stay in my inbox as one of items to get to.
>> 
>> I can see the usefulness in the somewhat unusual case of a super urgent 
>> patch. We may want multiple people to watch it, so that members of CC list 
>> would go and ask the patch author to update it with more urgency than e-mail 
>> allows for. I think that opt-in is a better mechanism for that, so that 
>> people who opted in would receive information about each EWS data point.
>> 
>> I think there is a value in knowing that a patch isn't ready instead of 
>> having to open the bug to realize that.
> 
> So just to clarify, 
> 
> - a major part of how you get to review bugs is by being CC'ed, and you 
> review them when you have the time to read bugmail;
> - and you don't open the bug in Bugzilla if there is already an EWS failure 
> by the time you read the e-mail where review is requested?
> 
> That's clearly a valid benefit. In my mind, it probably doesn't outweigh the 
> downsides. On the other hand, yours is a voice of someone who reviews way 
> more patches than Maciej and me combined these days, so maybe more e-mail is 
> an overall benefit to many of the reviewers.
> 
> - Alexey
> 
> 
> 
>> - R. Niwa
>>> 3 нояб. 2019 г., в 6:58 PM, Maciej Stachowiak >> > написал(а):
>>> 
>>> 
>>> I think they are useful to actual and potential reviewers. Direct email to 
>>> the patch author is not something anyone can Cc themselves on, and is not 
>>> archived, so seems like a strictly worse form of communication.
>>> 
 On Nov 2, 2019, at 9:34 AM, Alexey Proskuryakov >>> > wrote:
 
 
 My preference is still e-mailing the patch author directly (possibly, also 
 having an option to opt in for anyone). Bugzilla comments will always be 
 irrelevant for most people CC'ed on the bug, and they are almost always 
 undesirable to keep within the discussion flow.
 
 - Alexey
 
> 1 нояб. 2019 г., в 18:28, Aakash Jain  > написал(а):
> 
> Sounds good. I prefer the single comment when the first failure occur. 
> That way notification would be sent as soon as the first failure happens.
> 
> I'll implement that (assuming it's acceptable to everyone).
> 
> Thanks
> Aakash
> 
>> On Nov 1, 2019, at 8:35 PM, Maciej Stachowiak > > wrote:
>> 
>> 
>> How about only a single comment when the first failure occurs? (Or else 
>> when all bots pass, if there is never a failure.)
>> 
>> This should help the author, the reviewer, and an

Re: [webkit-dev] EWS Comments on Bugzilla (Was: EWS now parses error logs in case of build failure)

[Devin Rousso]

Hi,

I may be alone here, but I actually found build/test(s) failure emails to be 
super useful.  The part that I did not find useful was that each time a 
build/test(s) would fail on a particular platform, there would be two 
comments/emails, which furthermore split the useful information into two 
pieces.  IIRC, the first comment/email would contain information about what 
build/test(s) failed, and the second comment/email was just an archive upload 
of the layout test results (if applicable).

I would find it very useful to get an email each time one of the bots had 
something go "wrong", and ideally include a link directly to the 'errors' log 
that Aakash mention in the beginning of this thread (or to the layout test 
results in the case of a test(s) failure).  Right now, if something goes 
"wrong", I have to be the one to discover it and then click through a few pages 
until I find the data that will actually help me solve the problem.

As a patch author, this is useful in letting me not have to remember to check 
my patch once a day in order to see if there are any red bubbles (something 
which I've forgotten to do, leaving a broken patch up for review thinking all 
was fine).  I can upload a patch and ideally leave it be, as I will be notified 
if the build/test(s) fails.

As a patch reviewer, I'm not sure this would be as useful, as I have a saved 
query that I use for finding all patches that need to be reviewed, and on a 
search result page there are no EWS bubbles anyways, so I can't see the info 
there.  I have to click into the bug regardless.  To Ryosuke's point, it may 
still be useful to get an email as a reviewer simply so I know that something 
failed and can potentially remember that fact before I click on a bug in the 
search results page, but I know personally that my mind doesn't really work 
that way :P

Thanks,
Devin

> On Nov 5, 2019, at 09:04, Alexey Proskuryakov  wrote:
> 
> 
> 
>> 4 нояб. 2019 г., в 1:37 PM, Ryosuke Niwa > > написал(а):
>> 
>> 
>> On Mon, Nov 4, 2019 at 9:40 AM Alexey Proskuryakov > > wrote:
>> 
>> Can you elaborate on that, how exactly is e-mailing on first failure useful 
>> to reviewers?
>> 
>> Getting rid of Bugzilla comments was one of the goals of EWS rewrite, based 
>> on engineering feedback about noise in bugs and in e-mail, and I 
>> wholeheartedly agree with this feedback. So I think that comments are 
>> generally undesirable.
>> 
>> Since I don't understand what your precise scenario is, I may be make straw 
>> man arguments below, but here are some things that I think make the proposed 
>> behavior unhelpful (add a comment on first failure, or when all EWSes pass).
>> 
>> 1. EWS comments in Bugzilla are so annoying that some people take the 
>> radical step of manually hiding them. EWS history is archived anyway, there 
>> is no need to look into comments for it.
>> 
>> 2. There are often many people CC'ed on the bug to whom EWS data is 
>> irrelevant or even mysterious (e.g. reporters, web developers or 
>> non-reviewers). The noise is a slight annoyance, discouraging further 
>> participation in the project.
>> 
>> 3. I believe that for most reviewers, the mode of operation is one of the 
>> two: (1) do it when pinged directly, or (2) go over the review queue when 
>> one has the time. Getting EWS comments helps neither.
>> 
>> 4. Commenting when all EWSes pass is not very practical - it's too often 
>> that we have some stragglers that take days (or forever). I don't think that 
>> we can make it reliable even if we start actively policing EWS 
>> responsiveness.
>> 
>> 5. The reviewer likely wants to know the state of multiple EWSes if they are 
>> going to wait for EWS at all. What exactly are they going to do after 
>> getting an e-mail that one EWS failed?
>> 
>> I often use a EWS failure as a signal to wait reviewing a patch. Otherwise, 
>> a bug mail will stay in my inbox as one of items to get to.
>> 
>> I can see the usefulness in the somewhat unusual case of a super urgent 
>> patch. We may want multiple people to watch it, so that members of CC list 
>> would go and ask the patch author to update it with more urgency than e-mail 
>> allows for. I think that opt-in is a better mechanism for that, so that 
>> people who opted in would receive information about each EWS data point.
>> 
>> I think there is a value in knowing that a patch isn't ready instead of 
>> having to open the bug to realize that.
> 
> So just to clarify, 
> 
> - a major part of how you get to review bugs is by being CC'ed, and you 
> review them when you have the time to read bugmail;
> - and you don't open the bug in Bugzilla if there is already an EWS failure 
> by the time you read the e-mail where review is requested?
> 
> That's clearly a valid benefit. In my mind, it probably doesn't outweigh the 
> downsides. On the other hand, yours is a voice of someone who reviews way 
> more patches than Maciej and me combi

Re: [webkit-dev] EWS Comments on Bugzilla (Was: EWS now parses error logs in case of build failure)

[Alexey Proskuryakov]

> 4 нояб. 2019 г., в 1:37 PM, Ryosuke Niwa  написал(а):
> 
> 
> On Mon, Nov 4, 2019 at 9:40 AM Alexey Proskuryakov  > wrote:
> 
> Can you elaborate on that, how exactly is e-mailing on first failure useful 
> to reviewers?
> 
> Getting rid of Bugzilla comments was one of the goals of EWS rewrite, based 
> on engineering feedback about noise in bugs and in e-mail, and I 
> wholeheartedly agree with this feedback. So I think that comments are 
> generally undesirable.
> 
> Since I don't understand what your precise scenario is, I may be make straw 
> man arguments below, but here are some things that I think make the proposed 
> behavior unhelpful (add a comment on first failure, or when all EWSes pass).
> 
> 1. EWS comments in Bugzilla are so annoying that some people take the radical 
> step of manually hiding them. EWS history is archived anyway, there is no 
> need to look into comments for it.
> 
> 2. There are often many people CC'ed on the bug to whom EWS data is 
> irrelevant or even mysterious (e.g. reporters, web developers or 
> non-reviewers). The noise is a slight annoyance, discouraging further 
> participation in the project.
> 
> 3. I believe that for most reviewers, the mode of operation is one of the 
> two: (1) do it when pinged directly, or (2) go over the review queue when one 
> has the time. Getting EWS comments helps neither.
> 
> 4. Commenting when all EWSes pass is not very practical - it's too often that 
> we have some stragglers that take days (or forever). I don't think that we 
> can make it reliable even if we start actively policing EWS responsiveness.
> 
> 5. The reviewer likely wants to know the state of multiple EWSes if they are 
> going to wait for EWS at all. What exactly are they going to do after getting 
> an e-mail that one EWS failed?
> 
> I often use a EWS failure as a signal to wait reviewing a patch. Otherwise, a 
> bug mail will stay in my inbox as one of items to get to.
> 
> I can see the usefulness in the somewhat unusual case of a super urgent 
> patch. We may want multiple people to watch it, so that members of CC list 
> would go and ask the patch author to update it with more urgency than e-mail 
> allows for. I think that opt-in is a better mechanism for that, so that 
> people who opted in would receive information about each EWS data point.
> 
> I think there is a value in knowing that a patch isn't ready instead of 
> having to open the bug to realize that.

So just to clarify, 

- a major part of how you get to review bugs is by being CC'ed, and you review 
them when you have the time to read bugmail;
- and you don't open the bug in Bugzilla if there is already an EWS failure by 
the time you read the e-mail where review is requested?

That's clearly a valid benefit. In my mind, it probably doesn't outweigh the 
downsides. On the other hand, yours is a voice of someone who reviews way more 
patches than Maciej and me combined these days, so maybe more e-mail is an 
overall benefit to many of the reviewers.

- Alexey



> - R. Niwa
>> 3 нояб. 2019 г., в 6:58 PM, Maciej Stachowiak > > написал(а):
>> 
>> 
>> I think they are useful to actual and potential reviewers. Direct email to 
>> the patch author is not something anyone can Cc themselves on, and is not 
>> archived, so seems like a strictly worse form of communication.
>> 
>>> On Nov 2, 2019, at 9:34 AM, Alexey Proskuryakov >> > wrote:
>>> 
>>> 
>>> My preference is still e-mailing the patch author directly (possibly, also 
>>> having an option to opt in for anyone). Bugzilla comments will always be 
>>> irrelevant for most people CC'ed on the bug, and they are almost always 
>>> undesirable to keep within the discussion flow.
>>> 
>>> - Alexey
>>> 
 1 нояб. 2019 г., в 18:28, Aakash Jain >>> > написал(а):
 
 Sounds good. I prefer the single comment when the first failure occur. 
 That way notification would be sent as soon as the first failure happens.
 
 I'll implement that (assuming it's acceptable to everyone).
 
 Thanks
 Aakash
 
> On Nov 1, 2019, at 8:35 PM, Maciej Stachowiak  > wrote:
> 
> 
> How about only a single comment when the first failure occurs? (Or else 
> when all bots pass, if there is never a failure.)
> 
> This should help the author, the reviewer, and anyone else cc’d, without 
> being too spammy.
> 
>> On Nov 1, 2019, at 5:20 PM, Aakash Jain > > wrote:
>> 
>> Hi Ryosuke,
>> 
>> Many people didn't like the noise by the EWS comments, and we removed 
>> the comments as per previous discussion in: 
>> https://lists.webkit.org/pipermail/webkit-dev/2019-June/030683.html 
>> .
>> 
>> I agree with your point that having some kind of notifica

Re: [webkit-dev] EWS Comments on Bugzilla (Was: EWS now parses error logs in case of build failure)

[Ryosuke Niwa]

On Mon, Nov 4, 2019 at 9:40 AM Alexey Proskuryakov  wrote:

>
> Can you elaborate on that, how exactly is e-mailing on first failure
> useful to reviewers?
>
> Getting rid of Bugzilla comments was one of the goals of EWS rewrite,
> based on engineering feedback about noise in bugs and in e-mail, and I
> wholeheartedly agree with this feedback. So I think that comments are
> generally undesirable.
>
> Since I don't understand what your precise scenario is, I may be make
> straw man arguments below, but here are some things that I think make the
> proposed behavior unhelpful (add a comment on first failure, or when all
> EWSes pass).
>
> 1. EWS comments in Bugzilla are so annoying that some people take the
> radical step of manually hiding them. EWS history is archived anyway, there
> is no need to look into comments for it.
>
> 2. There are often many people CC'ed on the bug to whom EWS data is
> irrelevant or even mysterious (e.g. reporters, web developers or
> non-reviewers). The noise is a slight annoyance, discouraging further
> participation in the project.
>
> 3. I believe that for most reviewers, the mode of operation is one of the
> two: (1) do it when pinged directly, or (2) go over the review queue when
> one has the time. Getting EWS comments helps neither.
>
> 4. Commenting when all EWSes pass is not very practical - it's too often
> that we have some stragglers that take days (or forever). I don't think
> that we can make it reliable even if we start actively policing EWS
> responsiveness.
>
> 5. The reviewer likely wants to know the state of multiple EWSes if they
> are going to wait for EWS at all. What exactly are they going to do after
> getting an e-mail that one EWS failed?
>

I often use a EWS failure as a signal to wait reviewing a patch. Otherwise,
a bug mail will stay in my inbox as one of items to get to.

I can see the usefulness in the somewhat unusual case of a super urgent
> patch. We may want multiple people to watch it, so that members of CC list
> would go and ask the patch author to update it with more urgency than
> e-mail allows for. I think that opt-in is a better mechanism for that, so
> that people who opted in would receive information about each EWS data
> point.
>

I think there is a value in knowing that a patch isn't ready instead of
having to open the bug to realize that.

- R. Niwa

> 3 нояб. 2019 г., в 6:58 PM, Maciej Stachowiak  написал(а):
>
>
> I think they are useful to actual and potential reviewers. Direct email to
> the patch author is not something anyone can Cc themselves on, and is not
> archived, so seems like a strictly worse form of communication.
>
> On Nov 2, 2019, at 9:34 AM, Alexey Proskuryakov  wrote:
>
>
> My preference is still e-mailing the patch author directly (possibly, also
> having an option to opt in for anyone). Bugzilla comments will always be
> irrelevant for most people CC'ed on the bug, and they are almost always
> undesirable to keep within the discussion flow.
>
> - Alexey
>
> 1 нояб. 2019 г., в 18:28, Aakash Jain  написал(а):
>
> Sounds good. I prefer the single comment when the first failure occur.
> That way notification would be sent as soon as the first failure happens.
>
> I'll implement that (assuming it's acceptable to everyone).
>
> Thanks
> Aakash
>
> On Nov 1, 2019, at 8:35 PM, Maciej Stachowiak  wrote:
>
>
> How about only a single comment when the first failure occurs? (Or else
> when all bots pass, if there is never a failure.)
>
> This should help the author, the reviewer, and anyone else cc’d, without
> being too spammy.
>
> On Nov 1, 2019, at 5:20 PM, Aakash Jain  wrote:
>
> Hi Ryosuke,
>
> Many people didn't like the noise by the EWS comments, and we removed the
> comments as per previous discussion in:
> https://lists.webkit.org/pipermail/webkit-dev/2019-June/030683.html.
>
> I agree with your point that having some kind of notification might be
> useful.
>
> I proposed some ideas in
> https://lists.webkit.org/pipermail/webkit-dev/2019-September/030798.html,
> but didn't get much feedback. If we can all agree on a solution, I can look
> into implementing it.
>
> Thanks
> Aakash
>
> On Oct 30, 2019, at 1:03 AM,
> - R. Niwa
>  wrote:
>
> These enhancements are great. There is one feature of the old EWS that I
> really miss, which is that I used to get emails when some EWS failed. With
> new EWS, I have to keep checking back the bugzilla to see if any of them
> have failed periodically.
>
> Can we add a feature to opt into such an email notification? Maybe a flag
> on a patch or JSON configuration file somewhere.
>
> - R. Niwa
>
> On Tue, Oct 29, 2019 at 4:05 PM Aakash Jain  wrote:
> Hi Everyone,
>
> I am happy to announce another EWS feature.
>
> From now on, in case of build failure, EWS will parse the errors and
> display them in a separate 'errors' log. You wouldn't have to search
> through thousands of lines of logs to find the error message.
>
> For example, in https://ews-build.webkit.org/#/builders/16/buil

Re: [webkit-dev] EWS Comments on Bugzilla (Was: EWS now parses error logs in case of build failure)

[Alexey Proskuryakov]

Can you elaborate on that, how exactly is e-mailing on first failure useful to 
reviewers?

Getting rid of Bugzilla comments was one of the goals of EWS rewrite, based on 
engineering feedback about noise in bugs and in e-mail, and I wholeheartedly 
agree with this feedback. So I think that comments are generally undesirable.

Since I don't understand what your precise scenario is, I may be make straw man 
arguments below, but here are some things that I think make the proposed 
behavior unhelpful (add a comment on first failure, or when all EWSes pass).

1. EWS comments in Bugzilla are so annoying that some people take the radical 
step of manually hiding them. EWS history is archived anyway, there is no need 
to look into comments for it.

2. There are often many people CC'ed on the bug to whom EWS data is irrelevant 
or even mysterious (e.g. reporters, web developers or non-reviewers). The noise 
is a slight annoyance, discouraging further participation in the project.

3. I believe that for most reviewers, the mode of operation is one of the two: 
(1) do it when pinged directly, or (2) go over the review queue when one has 
the time. Getting EWS comments helps neither.

4. Commenting when all EWSes pass is not very practical - it's too often that 
we have some stragglers that take days (or forever). I don't think that we can 
make it reliable even if we start actively policing EWS responsiveness.

5. The reviewer likely wants to know the state of multiple EWSes if they are 
going to wait for EWS at all. What exactly are they going to do after getting 
an e-mail that one EWS failed?

6. More bugmail delays response, especially for active project members who are 
CC'ed on a lot of bugs. I personally started reading bugmail more frequently 
now, knowing that there is more signal and less noise.

I can see the usefulness in the somewhat unusual case of a super urgent patch. 
We may want multiple people to watch it, so that members of CC list would go 
and ask the patch author to update it with more urgency than e-mail allows for. 
I think that opt-in is a better mechanism for that, so that people who opted in 
would receive information about each EWS data point.

- Alexey


> 3 нояб. 2019 г., в 6:58 PM, Maciej Stachowiak  написал(а):
> 
> 
> I think they are useful to actual and potential reviewers. Direct email to 
> the patch author is not something anyone can Cc themselves on, and is not 
> archived, so seems like a strictly worse form of communication.
> 
>> On Nov 2, 2019, at 9:34 AM, Alexey Proskuryakov  wrote:
>> 
>> 
>> My preference is still e-mailing the patch author directly (possibly, also 
>> having an option to opt in for anyone). Bugzilla comments will always be 
>> irrelevant for most people CC'ed on the bug, and they are almost always 
>> undesirable to keep within the discussion flow.
>> 
>> - Alexey
>> 
>>> 1 нояб. 2019 г., в 18:28, Aakash Jain  написал(а):
>>> 
>>> Sounds good. I prefer the single comment when the first failure occur. That 
>>> way notification would be sent as soon as the first failure happens.
>>> 
>>> I'll implement that (assuming it's acceptable to everyone).
>>> 
>>> Thanks
>>> Aakash
>>> 
 On Nov 1, 2019, at 8:35 PM, Maciej Stachowiak  wrote:
 
 
 How about only a single comment when the first failure occurs? (Or else 
 when all bots pass, if there is never a failure.)
 
 This should help the author, the reviewer, and anyone else cc’d, without 
 being too spammy.
 
> On Nov 1, 2019, at 5:20 PM, Aakash Jain  wrote:
> 
> Hi Ryosuke,
> 
> Many people didn't like the noise by the EWS comments, and we removed the 
> comments as per previous discussion in: 
> https://lists.webkit.org/pipermail/webkit-dev/2019-June/030683.html.
> 
> I agree with your point that having some kind of notification might be 
> useful.
> 
> I proposed some ideas in 
> https://lists.webkit.org/pipermail/webkit-dev/2019-September/030798.html, 
> but didn't get much feedback. If we can all agree on a solution, I can 
> look into implementing it.
> 
> Thanks
> Aakash
> 
>> On Oct 30, 2019, at 1:03 AM, Ryosuke Niwa  wrote:
>> 
>> These enhancements are great. There is one feature of the old EWS that I 
>> really miss, which is that I used to get emails when some EWS failed. 
>> With new EWS, I have to keep checking back the bugzilla to see if any of 
>> them have failed periodically.
>> 
>> Can we add a feature to opt into such an email notification? Maybe a 
>> flag on a patch or JSON configuration file somewhere.
>> 
>> - R. Niwa
>> 
>> On Tue, Oct 29, 2019 at 4:05 PM Aakash Jain  
>> wrote:
>> Hi Everyone,
>> 
>> I am happy to announce another EWS feature.
>> 
>> From now on, in case of build failure, EWS will parse the errors and 
>> display them in a separate 'errors' log. You wouldn't have to search 
>>

Re: [webkit-dev] EWS Comments on Bugzilla (Was: EWS now parses error logs in case of build failure)

[Maciej Stachowiak]

I think they are useful to actual and potential reviewers. Direct email to the 
patch author is not something anyone can Cc themselves on, and is not archived, 
so seems like a strictly worse form of communication.

> On Nov 2, 2019, at 9:34 AM, Alexey Proskuryakov  wrote:
> 
> 
> My preference is still e-mailing the patch author directly (possibly, also 
> having an option to opt in for anyone). Bugzilla comments will always be 
> irrelevant for most people CC'ed on the bug, and they are almost always 
> undesirable to keep within the discussion flow.
> 
> - Alexey
> 
>> 1 нояб. 2019 г., в 18:28, Aakash Jain  написал(а):
>> 
>> Sounds good. I prefer the single comment when the first failure occur. That 
>> way notification would be sent as soon as the first failure happens.
>> 
>> I'll implement that (assuming it's acceptable to everyone).
>> 
>> Thanks
>> Aakash
>> 
>>> On Nov 1, 2019, at 8:35 PM, Maciej Stachowiak  wrote:
>>> 
>>> 
>>> How about only a single comment when the first failure occurs? (Or else 
>>> when all bots pass, if there is never a failure.)
>>> 
>>> This should help the author, the reviewer, and anyone else cc’d, without 
>>> being too spammy.
>>> 
 On Nov 1, 2019, at 5:20 PM, Aakash Jain  wrote:
 
 Hi Ryosuke,
 
 Many people didn't like the noise by the EWS comments, and we removed the 
 comments as per previous discussion in: 
 https://lists.webkit.org/pipermail/webkit-dev/2019-June/030683.html.
 
 I agree with your point that having some kind of notification might be 
 useful.
 
 I proposed some ideas in 
 https://lists.webkit.org/pipermail/webkit-dev/2019-September/030798.html, 
 but didn't get much feedback. If we can all agree on a solution, I can 
 look into implementing it.
 
 Thanks
 Aakash
 
> On Oct 30, 2019, at 1:03 AM, Ryosuke Niwa  wrote:
> 
> These enhancements are great. There is one feature of the old EWS that I 
> really miss, which is that I used to get emails when some EWS failed. 
> With new EWS, I have to keep checking back the bugzilla to see if any of 
> them have failed periodically.
> 
> Can we add a feature to opt into such an email notification? Maybe a flag 
> on a patch or JSON configuration file somewhere.
> 
> - R. Niwa
> 
> On Tue, Oct 29, 2019 at 4:05 PM Aakash Jain  wrote:
> Hi Everyone,
> 
> I am happy to announce another EWS feature.
> 
> From now on, in case of build failure, EWS will parse the errors and 
> display them in a separate 'errors' log. You wouldn't have to search 
> through thousands of lines of logs to find the error message.
> 
> For example, in https://ews-build.webkit.org/#/builders/16/builds/6054, 
> in step #7 WebKit failed to compile. Complete logs (stdio) are 38,000+ 
> lines, and the error is not at the end of the logs. Normally, it requires 
> some searching through the logs to find the relevant errors. But now, 
> there is another 'errors' log, which contains just the relevant 11 lines 
> (containing error and few related lines to provide additional context).
> 
> Hopefully this would save some time and efforts previously spent on 
> searching through the large logs.
> 
> Note that this information is not displayed in status-bubble tool-tip, 
> since this might be lot of text to display in the tooltip. My further 
> plan is to make this information more readily available, by adding it to 
> a custom designed page which will open on clicking the status bubble 
> https://webkit.org/b/197522
> 
> Please let me know if you notice any issues or have any feedback.
> 
> Thanks
> Aakash
> 
> Reference: https://webkit.org/b/203418
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
> -- 
> - R. Niwa
 
 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 https://lists.webkit.org/mailman/listinfo/webkit-dev
>>> 
>> 
>> ___
>> webkit-dev mailing list
>> webkit-dev@lists.webkit.org
>> https://lists.webkit.org/mailman/listinfo/webkit-dev
> 
> - Alexey
> 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] EWS Comments on Bugzilla (Was: EWS now parses error logs in case of build failure)

[Alexey Proskuryakov]

My preference is still e-mailing the patch author directly (possibly, also 
having an option to opt in for anyone). Bugzilla comments will always be 
irrelevant for most people CC'ed on the bug, and they are almost always 
undesirable to keep within the discussion flow.

- Alexey

> 1 нояб. 2019 г., в 18:28, Aakash Jain  написал(а):
> 
> Sounds good. I prefer the single comment when the first failure occur. That 
> way notification would be sent as soon as the first failure happens.
> 
> I'll implement that (assuming it's acceptable to everyone).
> 
> Thanks
> Aakash
> 
>> On Nov 1, 2019, at 8:35 PM, Maciej Stachowiak  wrote:
>> 
>> 
>> How about only a single comment when the first failure occurs? (Or else when 
>> all bots pass, if there is never a failure.)
>> 
>> This should help the author, the reviewer, and anyone else cc’d, without 
>> being too spammy.
>> 
>>> On Nov 1, 2019, at 5:20 PM, Aakash Jain  wrote:
>>> 
>>> Hi Ryosuke,
>>> 
>>> Many people didn't like the noise by the EWS comments, and we removed the 
>>> comments as per previous discussion in: 
>>> https://lists.webkit.org/pipermail/webkit-dev/2019-June/030683.html.
>>> 
>>> I agree with your point that having some kind of notification might be 
>>> useful.
>>> 
>>> I proposed some ideas in 
>>> https://lists.webkit.org/pipermail/webkit-dev/2019-September/030798.html, 
>>> but didn't get much feedback. If we can all agree on a solution, I can look 
>>> into implementing it.
>>> 
>>> Thanks
>>> Aakash
>>> 
 On Oct 30, 2019, at 1:03 AM, Ryosuke Niwa  wrote:
 
 These enhancements are great. There is one feature of the old EWS that I 
 really miss, which is that I used to get emails when some EWS failed. With 
 new EWS, I have to keep checking back the bugzilla to see if any of them 
 have failed periodically.
 
 Can we add a feature to opt into such an email notification? Maybe a flag 
 on a patch or JSON configuration file somewhere.
 
 - R. Niwa
 
 On Tue, Oct 29, 2019 at 4:05 PM Aakash Jain  wrote:
 Hi Everyone,
 
 I am happy to announce another EWS feature.
 
 From now on, in case of build failure, EWS will parse the errors and 
 display them in a separate 'errors' log. You wouldn't have to search 
 through thousands of lines of logs to find the error message.
 
 For example, in https://ews-build.webkit.org/#/builders/16/builds/6054, in 
 step #7 WebKit failed to compile. Complete logs (stdio) are 38,000+ lines, 
 and the error is not at the end of the logs. Normally, it requires some 
 searching through the logs to find the relevant errors. But now, there is 
 another 'errors' log, which contains just the relevant 11 lines 
 (containing error and few related lines to provide additional context).
 
 Hopefully this would save some time and efforts previously spent on 
 searching through the large logs.
 
 Note that this information is not displayed in status-bubble tool-tip, 
 since this might be lot of text to display in the tooltip. My further plan 
 is to make this information more readily available, by adding it to a 
 custom designed page which will open on clicking the status bubble 
 https://webkit.org/b/197522
 
 Please let me know if you notice any issues or have any feedback.
 
 Thanks
 Aakash
 
 Reference: https://webkit.org/b/203418
 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 https://lists.webkit.org/mailman/listinfo/webkit-dev
 -- 
 - R. Niwa
>>> 
>>> ___
>>> webkit-dev mailing list
>>> webkit-dev@lists.webkit.org
>>> https://lists.webkit.org/mailman/listinfo/webkit-dev
>> 
> 
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] EWS Comments on Bugzilla (Was: EWS now parses error logs in case of build failure)

[Aakash Jain]

Sounds good. I prefer the single comment when the first failure occur. That way 
notification would be sent as soon as the first failure happens.

I'll implement that (assuming it's acceptable to everyone).

Thanks
Aakash

> On Nov 1, 2019, at 8:35 PM, Maciej Stachowiak  wrote:
> 
> 
> How about only a single comment when the first failure occurs? (Or else when 
> all bots pass, if there is never a failure.)
> 
> This should help the author, the reviewer, and anyone else cc’d, without 
> being too spammy.
> 
>> On Nov 1, 2019, at 5:20 PM, Aakash Jain  wrote:
>> 
>> Hi Ryosuke,
>> 
>> Many people didn't like the noise by the EWS comments, and we removed the 
>> comments as per previous discussion in: 
>> https://lists.webkit.org/pipermail/webkit-dev/2019-June/030683.html.
>> 
>> I agree with your point that having some kind of notification might be 
>> useful.
>> 
>> I proposed some ideas in 
>> https://lists.webkit.org/pipermail/webkit-dev/2019-September/030798.html, 
>> but didn't get much feedback. If we can all agree on a solution, I can look 
>> into implementing it.
>> 
>> Thanks
>> Aakash
>> 
>>> On Oct 30, 2019, at 1:03 AM, Ryosuke Niwa  wrote:
>>> 
>>> These enhancements are great. There is one feature of the old EWS that I 
>>> really miss, which is that I used to get emails when some EWS failed. With 
>>> new EWS, I have to keep checking back the bugzilla to see if any of them 
>>> have failed periodically.
>>> 
>>> Can we add a feature to opt into such an email notification? Maybe a flag 
>>> on a patch or JSON configuration file somewhere.
>>> 
>>> - R. Niwa
>>> 
>>> On Tue, Oct 29, 2019 at 4:05 PM Aakash Jain  wrote:
>>> Hi Everyone,
>>> 
>>> I am happy to announce another EWS feature.
>>> 
>>> From now on, in case of build failure, EWS will parse the errors and 
>>> display them in a separate 'errors' log. You wouldn't have to search 
>>> through thousands of lines of logs to find the error message.
>>> 
>>> For example, in https://ews-build.webkit.org/#/builders/16/builds/6054, in 
>>> step #7 WebKit failed to compile. Complete logs (stdio) are 38,000+ lines, 
>>> and the error is not at the end of the logs. Normally, it requires some 
>>> searching through the logs to find the relevant errors. But now, there is 
>>> another 'errors' log, which contains just the relevant 11 lines (containing 
>>> error and few related lines to provide additional context).
>>> 
>>> Hopefully this would save some time and efforts previously spent on 
>>> searching through the large logs.
>>> 
>>> Note that this information is not displayed in status-bubble tool-tip, 
>>> since this might be lot of text to display in the tooltip. My further plan 
>>> is to make this information more readily available, by adding it to a 
>>> custom designed page which will open on clicking the status bubble 
>>> https://webkit.org/b/197522
>>> 
>>> Please let me know if you notice any issues or have any feedback.
>>> 
>>> Thanks
>>> Aakash
>>> 
>>> Reference: https://webkit.org/b/203418
>>> ___
>>> webkit-dev mailing list
>>> webkit-dev@lists.webkit.org
>>> https://lists.webkit.org/mailman/listinfo/webkit-dev
>>> -- 
>>> - R. Niwa
>> 
>> ___
>> webkit-dev mailing list
>> webkit-dev@lists.webkit.org
>> https://lists.webkit.org/mailman/listinfo/webkit-dev
> 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] EWS Comments on Bugzilla (Was: EWS now parses error logs in case of build failure)

[Maciej Stachowiak]

How about only a single comment when the first failure occurs? (Or else when 
all bots pass, if there is never a failure.)

This should help the author, the reviewer, and anyone else cc’d, without being 
too spammy.

> On Nov 1, 2019, at 5:20 PM, Aakash Jain  wrote:
> 
> Hi Ryosuke,
> 
> Many people didn't like the noise by the EWS comments, and we removed the 
> comments as per previous discussion in: 
> https://lists.webkit.org/pipermail/webkit-dev/2019-June/030683.html.
> 
> I agree with your point that having some kind of notification might be useful.
> 
> I proposed some ideas in 
> https://lists.webkit.org/pipermail/webkit-dev/2019-September/030798.html, but 
> didn't get much feedback. If we can all agree on a solution, I can look into 
> implementing it.
> 
> Thanks
> Aakash
> 
>> On Oct 30, 2019, at 1:03 AM, Ryosuke Niwa  wrote:
>> 
>> These enhancements are great. There is one feature of the old EWS that I 
>> really miss, which is that I used to get emails when some EWS failed. With 
>> new EWS, I have to keep checking back the bugzilla to see if any of them 
>> have failed periodically.
>> 
>> Can we add a feature to opt into such an email notification? Maybe a flag on 
>> a patch or JSON configuration file somewhere.
>> 
>> - R. Niwa
>> 
>> On Tue, Oct 29, 2019 at 4:05 PM Aakash Jain  wrote:
>> Hi Everyone,
>> 
>> I am happy to announce another EWS feature.
>> 
>> From now on, in case of build failure, EWS will parse the errors and display 
>> them in a separate 'errors' log. You wouldn't have to search through 
>> thousands of lines of logs to find the error message.
>> 
>> For example, in https://ews-build.webkit.org/#/builders/16/builds/6054, in 
>> step #7 WebKit failed to compile. Complete logs (stdio) are 38,000+ lines, 
>> and the error is not at the end of the logs. Normally, it requires some 
>> searching through the logs to find the relevant errors. But now, there is 
>> another 'errors' log, which contains just the relevant 11 lines (containing 
>> error and few related lines to provide additional context).
>> 
>> Hopefully this would save some time and efforts previously spent on 
>> searching through the large logs.
>> 
>> Note that this information is not displayed in status-bubble tool-tip, since 
>> this might be lot of text to display in the tooltip. My further plan is to 
>> make this information more readily available, by adding it to a custom 
>> designed page which will open on clicking the status bubble 
>> https://webkit.org/b/197522
>> 
>> Please let me know if you notice any issues or have any feedback.
>> 
>> Thanks
>> Aakash
>> 
>> Reference: https://webkit.org/b/203418
>> ___
>> webkit-dev mailing list
>> webkit-dev@lists.webkit.org
>> https://lists.webkit.org/mailman/listinfo/webkit-dev
>> -- 
>> - R. Niwa
> 
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] EWS Comments on Bugzilla (Was: EWS now parses error logs in case of build failure)

[Aakash Jain]

Hi Ryosuke,

Many people didn't like the noise by the EWS comments, and we removed the 
comments as per previous discussion in: 
https://lists.webkit.org/pipermail/webkit-dev/2019-June/030683.html.

I agree with your point that having some kind of notification might be useful.

I proposed some ideas in 
https://lists.webkit.org/pipermail/webkit-dev/2019-September/030798.html, but 
didn't get much feedback. If we can all agree on a solution, I can look into 
implementing it.

Thanks
Aakash

> On Oct 30, 2019, at 1:03 AM, Ryosuke Niwa  wrote:
> 
> These enhancements are great. There is one feature of the old EWS that I 
> really miss, which is that I used to get emails when some EWS failed. With 
> new EWS, I have to keep checking back the bugzilla to see if any of them have 
> failed periodically.
> 
> Can we add a feature to opt into such an email notification? Maybe a flag on 
> a patch or JSON configuration file somewhere.
> 
> - R. Niwa
> 
> On Tue, Oct 29, 2019 at 4:05 PM Aakash Jain  wrote:
> Hi Everyone,
> 
> I am happy to announce another EWS feature.
> 
> From now on, in case of build failure, EWS will parse the errors and display 
> them in a separate 'errors' log. You wouldn't have to search through 
> thousands of lines of logs to find the error message.
> 
> For example, in https://ews-build.webkit.org/#/builders/16/builds/6054, in 
> step #7 WebKit failed to compile. Complete logs (stdio) are 38,000+ lines, 
> and the error is not at the end of the logs. Normally, it requires some 
> searching through the logs to find the relevant errors. But now, there is 
> another 'errors' log, which contains just the relevant 11 lines (containing 
> error and few related lines to provide additional context).
> 
> Hopefully this would save some time and efforts previously spent on searching 
> through the large logs.
> 
> Note that this information is not displayed in status-bubble tool-tip, since 
> this might be lot of text to display in the tooltip. My further plan is to 
> make this information more readily available, by adding it to a custom 
> designed page which will open on clicking the status bubble 
> https://webkit.org/b/197522
> 
> Please let me know if you notice any issues or have any feedback.
> 
> Thanks
> Aakash
> 
> Reference: https://webkit.org/b/203418
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
> -- 
> - R. Niwa

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev