Hi,
There has been some complaints / discussions about how syntax in
test_expectations.txt is confusing (and I agree with you) on webkit-dev and
at contributors' meeting.
So I have a patch on https://bugs.webkit.org/show_bug.cgi?id=86691 to
simplify syntax in test_expectation.txt as follows:
On Wed, May 16, 2012 at 9:08 PM, Ryosuke Niwa wrote:
> e.g. 12345 WIN MAC TEXT IMAGE test.html instead of BUGWK12345 WIN MAC :
> test.html = TEXT IMAGE
I wonder how this is gonna help making test_expectation.txt less
obscure. If anything, I would make the file more verbose.
I also think it could
Any concrete proposals?
On May 16, 2012 9:23 PM, "Benjamin Poulain" wrote:
> On Wed, May 16, 2012 at 9:08 PM, Ryosuke Niwa wrote:
> > e.g. 12345 WIN MAC TEXT IMAGE test.html instead of BUGWK12345 WIN MAC :
> > test.html = TEXT IMAGE
>
> I wonder how this is gonna help making test_expectation.txt
On Wed, May 16, 2012 at 9:08 PM, Ryosuke Niwa wrote:
> Hi,
>
> There has been some complaints / discussions about how syntax in
> test_expectations.txt is confusing (and I agree with you) on webkit-dev and
> at contributors' meeting.
>
> So I have a patch on https://bugs.webkit.org/show_bug.cgi?id
On May 16, 2012, at 9:08 PM, Ryosuke Niwa wrote:
> Hi,
>
> There has been some complaints / discussions about how syntax in
> test_expectations.txt is confusing (and I agree with you) on webkit-dev and
> at contributors' meeting.
>
> So I have a patch on https://bugs.webkit.org/show_bug.cgi?
On Wed, May 16, 2012 at 9:59 PM, Dirk Pranke wrote:
> On Wed, May 16, 2012 at 9:08 PM, Ryosuke Niwa wrote:
> > For those who are interested, we're also going to rename
> > test_expectations.txt to something more WebKitty
> > on https://bugs.webkit.org/show_bug.cgi?id=86690.
>
> I would have pref
On Wed, May 16, 2012 at 9:34 PM, Ryosuke Niwa wrote:
> Any concrete proposals?
I would go for something verbose in a easy to understand format. E.g.
{ test: 'fast/html/keygen.html',
platform: 'mac, linux',
bug: 'webkit.org/b/6',
expectedResults: ['crash', 'text', 'image'],
action: 'sk
On Wed, May 16, 2012 at 10:30 PM, Maciej Stachowiak wrote:
>
> On May 16, 2012, at 9:08 PM, Ryosuke Niwa wrote:
>
> Hi,
>
> There has been some complaints / discussions about how syntax in
> test_expectations.txt is confusing (and I agree with you) on webkit-dev and
> at contributors' meeting.
>
On Wed, May 16, 2012 at 10:39 PM, Dirk Pranke wrote:
> On Wed, May 16, 2012 at 10:30 PM, Maciej Stachowiak wrote:
> > On May 16, 2012, at 9:08 PM, Ryosuke Niwa wrote:
> > There has been some complaints / discussions about how syntax in
> > test_expectations.txt is confusing (and I agree with yo
On Wed, May 16, 2012 at 10:37 PM, Benjamin Poulain wrote:
> On Wed, May 16, 2012 at 9:34 PM, Ryosuke Niwa wrote:
> I would go for something verbose in a easy to understand format. E.g.
{ test: 'fast/html/keygen.html',
> platform: 'mac, linux',
> bug: 'webkit.org/b/6',
> expectedResults:
On Wed, May 16, 2012 at 11:04 PM, Ryosuke Niwa wrote:
>> { test: 'fast/html/keygen.html',
>> platform: 'mac, linux',
>> bug: 'webkit.org/b/6',
>> expectedResults: ['crash', 'text', 'image'],
>> action: 'skip' }
>
> As far as I know, a lot of people edit this file on a text editor so the
>
On Wed, May 16, 2012 at 11:33 PM, Benjamin Poulain wrote:
> On Wed, May 16, 2012 at 11:04 PM, Ryosuke Niwa wrote:
> > Also, complex syntax tends to be error-prone. One of the pain points of
> the
> > current syntax is that many people forget to add : and =, or put
> modifiers
> > in where expecta
On May 16, 2012, at 10:39 PM, Dirk Pranke wrote:
>
> There was a semi-logical basis, in that the stuff on the right of the
> test clearly specified the outcome of the test (PASS, IMAGE, TEXT,
> etc.). The stuff on the left was less well defined: there's the bug
> numbers, the platform/configura
On Thu, May 17, 2012 at 1:06 AM, Maciej Stachowiak wrote:
>
> On May 16, 2012, at 10:39 PM, Dirk Pranke wrote:
> > There was a semi-logical basis, in that the stuff on the right of the
> > test clearly specified the outcome of the test (PASS, IMAGE, TEXT,
> > etc.). The stuff on the left was les
I have a proposal that hopefully addresses everyone's concerns, is
minimally different from the current format *and* unifies the format with
Skipped lists (i.e. Skipped lists as they exist today are valid
test_expectations.txt format). The changes from the current format are as
follows:
-Leaving ou
On May 17, 2012, at 4:30 AM, Ojan Vafai wrote:
> I have a proposal that hopefully addresses everyone's concerns, is minimally
> different from the current format *and* unifies the format with Skipped lists
> (i.e. Skipped lists as they exist today are valid test_expectations.txt
> format). The
On Thu, May 17, 2012 at 9:11 AM, Darin Adler wrote:
> On May 17, 2012, at 4:30 AM, Ojan Vafai wrote:
>
> I have a proposal that hopefully addresses everyone's concerns, is minimally
> different from the current format *and* unifies the format with Skipped
> lists (i.e. Skipped lists as they exist
Seriously, syntax is a significant barrier. Having to know which special
characters to use. I don’t see this “clear delineation” you speak of. Just
special punctuation I have to use to satisfy the computer.
-- Darin
___
webkit-dev mailing list
webkit-d
Could that be because you haven't actually tried using it? :)
On Thu, May 17, 2012 at 4:40 PM, Darin Adler wrote:
> Seriously, syntax is a significant barrier. Having to know which special
> characters to use. I don’t see this “clear delineation” you speak of. Just
> special punctuation I have
Sorry, didn't mean to come across as flip. To illustrate, here's an example:
WIN SNOWLEOPARD : foo.html = TEXT IMAGE # on all windows platforms and
SL, test foo.html is flaky -- fails either with text or pixel results.
Easy.
TEXT WIN SNOWLEOPARD IMAGE foo.html # fails with text on all windows
pl
On Thu, May 17, 2012 at 9:11 AM, Darin Adler wrote:
>
> I don’t understand why we need the ":" or "+" separators.
>
Unfortunately, "image+text" is different is "image text". The former
indicates that the test will have mismatches in both .txt and .png files
while the latter indicates that either
On May 17, 2012, at 9:26 AM, Dimitri Glazkov wrote:
>
> Maybe it's the Stockholm syndrome talking, but I actually quite like
> the clear delineation between test modifiers and test expectations. My
> intuitive reaction to the attempts to bunch them all together is that
> it will make both writi
On Thu, May 17, 2012 at 9:26 AM, Dimitri Glazkov wrote:
> I actually quite like
> the clear delineation between test modifiers and test expectations.
Me too. Please please please leave them on opposite sides. All these
proposals that try to put them both before the test name are much harder
fo
On Thu, May 17, 2012 at 11:08 AM, Peter Kasting wrote:
> On Thu, May 17, 2012 at 9:26 AM, Dimitri Glazkov wrote:
>
>> I actually quite like
>> the clear delineation between test modifiers and test expectations.
>
>
> Me too. Please please please leave them on opposite sides. All these
> proposa
On May 17, 2012, at 11:28 AM, Ryosuke Niwa wrote:
> On Thu, May 17, 2012 at 11:08 AM, Peter Kasting wrote:
> On Thu, May 17, 2012 at 9:26 AM, Dimitri Glazkov
> wrote:
> I actually quite like
> the clear delineation between test modifiers and test expectations.
>
> Me too. Please please plea
On Thu, May 17, 2012 at 11:54 AM, Maciej Stachowiak wrote:
>
> On May 17, 2012, at 11:28 AM, Ryosuke Niwa wrote:
>
> On Thu, May 17, 2012 at 11:08 AM, Peter Kasting wrote:
>
>> On Thu, May 17, 2012 at 9:26 AM, Dimitri Glazkov
>> wrote:
>>
>>> I actually quite like
>>> the clear delineation betwe
On Thu, May 17, 2012 at 12:19 PM, Ryosuke Niwa wrote:
> On Thu, May 17, 2012 at 11:54 AM, Maciej Stachowiak wrote:
>>
>> Perhaps aligning the fields column after the bug URL would improve
>> readability (though it makes things a little harder to edit):
>>
>> We certainly could. We treat \s+ as \
There's lot of good discussion going on in this thread ... I'm going
to attempt to reply to various threads in one message, hixie-style :)
On Thu, May 17, 2012 at 1:06 AM, Maciej Stachowiak wrote:
> SKIP and WONTFIX seem parallel to PASS to me.
[and]
On Thu, May 17, 2012 at 11:07 AM, Maciej Sta
On Thu, May 17, 2012 at 12:36 PM, Dirk Pranke wrote:
> On Thu, May 17, 2012 at 4:30 AM, Ojan Vafai wrote:
> > -Make everything but the test name case-insensitive.
>
> I don't think I like this; it could lead to a lot of arbitrarily
> different formatting in the file, making things harder to rea
On Thu, May 17, 2012 at 1:06 AM, Maciej Stachowiak wrote:
> What does the build configuration info do? Does it apply the line to only
> those configurations? If that is the case, it does seem potentially different
> in kind, though maybe also better expressed by being able to combine multiple
>
On Thu, May 17, 2012 at 12:48 PM, Dirk Pranke wrote:
> On Thu, May 17, 2012 at 1:06 AM, Maciej Stachowiak wrote:
>> What does the build configuration info do? Does it apply the line to only
>> those configurations? If that is the case, it does seem potentially
>> different in kind, though maybe
On Thu, May 17, 2012 at 12:47 PM, Ryosuke Niwa wrote:
> On Thu, May 17, 2012 at 12:36 PM, Dirk Pranke wrote:
>>
>> On Thu, May 17, 2012 at 4:30 AM, Ojan Vafai wrote:
>> > -Make everything but the test name case-insensitive.
>>
>> I don't think I like this; it could lead to a lot of arbitrarily
>
We are arguing about too many orthogonal things at once. It seems like
there are a few things we can agree on. Lets make incremental progress on
those and after have a more targeted discussion on the controversial issues
that are left. We don't need to make this change in one big commit (and
should
On Thu, May 17, 2012 at 1:34 PM, Ojan Vafai wrote:
> 2. Make outcomes optional. If they are left out, then the test is skipped
> (unless the test is marked SLOW, in which case it's expected to pass).
> There is no SKIP modifier.
>
I don't think we should do this. It seems very subtle. I'd rath
On Thu, May 17, 2012 at 1:37 PM, Peter Kasting wrote:
> On Thu, May 17, 2012 at 1:34 PM, Ojan Vafai wrote:
>
>> 2. Make outcomes optional. If they are left out, then the test is skipped
>> (unless the test is marked SLOW, in which case it's expected to pass).
>> There is no SKIP modifier.
>>
>
>
Seems reasonable as the first step except:
On Thu, May 17, 2012 at 1:34 PM, Ojan Vafai wrote:
>
> 6. Use the same character as a separator before and after the test (":").
>
I counter-propose to drop ":" or any other delimiter altogether. Delimiters
are the biggest pain points in the current syn
On Thu, May 17, 2012 at 2:02 PM, Ryosuke Niwa wrote:
> Seems reasonable as the first step except:
>
> On Thu, May 17, 2012 at 1:34 PM, Ojan Vafai wrote:
>
>> 6. Use the same character as a separator before and after the test (":").
>>
>
> I counter-propose to drop ":" or any other delimiter alto
On Thu, May 17, 2012 at 2:05 PM, Ojan Vafai wrote:
> On Thu, May 17, 2012 at 2:02 PM, Ryosuke Niwa wrote:
>
>> Seems reasonable as the first step except:
>>
>> On Thu, May 17, 2012 at 1:34 PM, Ojan Vafai wrote:
>>
>>> 6. Use the same character as a separator before and after the test (":").
>>>
On Thu, May 17, 2012 at 2:06 PM, Ryosuke Niwa wrote:
> On Thu, May 17, 2012 at 2:05 PM, Ojan Vafai wrote:
>
>> On Thu, May 17, 2012 at 2:02 PM, Ryosuke Niwa wrote:
>>
>>> Seems reasonable as the first step except:
>>>
>>> On Thu, May 17, 2012 at 1:34 PM, Ojan Vafai wrote:
>>>
6. Use the s
On Thu, May 17, 2012 at 2:11 PM, Ojan Vafai wrote:
>
> Oh, I supposed I misread Peter's earlier email as being opposed to this.
> But, I for one am opposed to it. I find the jumble of modifiers in the
> second set of lines here much harder to quickly parse than the groupings in
> the first set of
On Thu, May 17, 2012 at 2:16 PM, Ryosuke Niwa wrote:
> On Thu, May 17, 2012 at 2:11 PM, Ojan Vafai wrote:
>>
>> Oh, I supposed I misread Peter's earlier email as being opposed to this.
>> But, I for one am opposed to it. I find the jumble of modifiers in the
>> second set of lines here much hard
On Thu, May 17, 2012 at 2:11 PM, Ojan Vafai wrote:
> Oh, I supposed I misread Peter's earlier email as being opposed to this.
>
You didn't misread me. I have the same opinion as you: this would be a
change for the worse.
PK
___
webkit-dev mailing lis
On Thu, May 17, 2012 at 2:19 PM, Ojan Vafai wrote:
> On Thu, May 17, 2012 at 2:16 PM, Ryosuke Niwa wrote:
>
>> On Thu, May 17, 2012 at 2:11 PM, Ojan Vafai wrote:
>>>
>>> Oh, I supposed I misread Peter's earlier email as being opposed to this.
>>> But, I for one am opposed to it. I find the jumb
On Thu, May 17, 2012 at 2:29 PM, Ryosuke Niwa wrote:
> On Thu, May 17, 2012 at 2:19 PM, Ojan Vafai wrote:
>
>> On Thu, May 17, 2012 at 2:16 PM, Ryosuke Niwa wrote:
>>
>>> On Thu, May 17, 2012 at 2:11 PM, Ojan Vafai wrote:
Oh, I supposed I misread Peter's earlier email as being oppose
On Thu, May 17, 2012 at 2:28 PM, Peter Kasting wrote:
> On Thu, May 17, 2012 at 2:11 PM, Ojan Vafai wrote:
>>
>> Oh, I supposed I misread Peter's earlier email as being opposed to this.
>
>
> You didn't misread me. I have the same opinion as you: this would be a
> change for the worse.
>
I thou
On Thu, May 17, 2012 at 2:36 PM, Dirk Pranke wrote:
> On Thu, May 17, 2012 at 2:28 PM, Peter Kasting
> wrote:
> > On Thu, May 17, 2012 at 2:11 PM, Ojan Vafai wrote:
> >>
> >> Oh, I supposed I misread Peter's earlier email as being opposed to this.
> >
> >
> > You didn't misread me. I have the
On Thu, May 17, 2012 at 2:34 PM, Ojan Vafai wrote:
>
> I'm just trying to make forward progress on the things we agree on.
>
Anyway, the said changes (replace bogus BUGCR/BUGWK with URLs) and moving
SKIP/WONTFIX to after test names sound like a good change anyways. I have
no intention in opposing
On Thu, May 17, 2012 at 2:39 PM, Ryosuke Niwa wrote:
> On Thu, May 17, 2012 at 2:36 PM, Dirk Pranke wrote:
>>
>> On Thu, May 17, 2012 at 2:28 PM, Peter Kasting
>> wrote:
>> > On Thu, May 17, 2012 at 2:11 PM, Ojan Vafai wrote:
>> >>
>> >> Oh, I supposed I misread Peter's earlier email as being o
On Thu, May 17, 2012 at 3:01 PM, Dirk Pranke wrote:
>
> As I said before, I believe they increase the readability of the file.
>
I see them as pure noise.
I believe the cost of learning to put delimiters in is near zero,
That clearly isn't near zero. Or else people wouldn't be complaining abou
On Thu, May 17, 2012 at 3:21 PM, Ryosuke Niwa wrote:
> On Thu, May 17, 2012 at 3:01 PM, Dirk Pranke wrote:
>>
>> As I said before, I believe they increase the readability of the file.
>
>
> I see them as pure noise.
>
Clearly, different people can have different syntactic preferences :).
>> I b
On Thu, May 17, 2012 at 3:37 PM, Dirk Pranke wrote:
> On Thu, May 17, 2012 at 3:21 PM, Ryosuke Niwa wrote:
>
> Perhaps we can address these two problems using some tools. e.g. I don't
> > care about the format of test_expectations.txt at all if there was a GUI
> > tool that lets me add/edit entr
On Thu, May 17, 2012 at 3:37 PM, Dirk Pranke wrote:
> On Thu, May 17, 2012 at 3:21 PM, Ryosuke Niwa wrote:
> > On Thu, May 17, 2012 at 3:01 PM, Dirk Pranke
> wrote:
> >> As I said before, I believe they increase the readability of the file.
> > I see them as pure noise.
>
> Clearly, different p
On May 17, 2012, at 12:53 PM, Dirk Pranke wrote:
> On Thu, May 17, 2012 at 12:47 PM, Ryosuke Niwa wrote:
>>
>> I find either all-lowercase or all-caps to be much harder to read than
>> capitalized words. They look like a blob of letters to me.
>
> We might have to agree to disagree here, then
On May 17, 2012, at 3:37 PM, Dirk Pranke wrote:
> On Thu, May 17, 2012 at 3:21 PM, Ryosuke Niwa wrote:
>> On Thu, May 17, 2012 at 3:01 PM, Dirk Pranke wrote:
>>>
>>> As I said before, I believe they increase the readability of the file.
>>
>>
>> I see them as pure noise.
>>
>
> Clearly, d
On Thu, May 17, 2012 at 4:00 PM, Maciej Stachowiak wrote:
>
> On May 17, 2012, at 12:53 PM, Dirk Pranke wrote:
>
>> On Thu, May 17, 2012 at 12:47 PM, Ryosuke Niwa wrote:
>>>
>>> I find either all-lowercase or all-caps to be much harder to read than
>>> capitalized words. They look like a blob of
On May 17, 2012, at 4:17 PM, Dirk Pranke wrote:
> On Thu, May 17, 2012 at 4:00 PM, Maciej Stachowiak wrote:
>>
>> On May 17, 2012, at 12:53 PM, Dirk Pranke wrote:
>>
>>> On Thu, May 17, 2012 at 12:47 PM, Ryosuke Niwa wrote:
I find either all-lowercase or all-caps to be much harde
On May 17, 2012, at 1:42 PM, Ojan Vafai wrote:
> On Thu, May 17, 2012 at 1:37 PM, Peter Kasting wrote:
> On Thu, May 17, 2012 at 1:34 PM, Ojan Vafai wrote:
> 2. Make outcomes optional. If they are left out, then the test is skipped
> (unless the test is marked SLOW, in which case it's expecte
On Thu, May 17, 2012 at 4:16 PM, Maciej Stachowiak wrote:
>
> On May 17, 2012, at 3:37 PM, Dirk Pranke wrote:
>
>> On Thu, May 17, 2012 at 3:21 PM, Ryosuke Niwa wrote:
>>> On Thu, May 17, 2012 at 3:01 PM, Dirk Pranke wrote:
As I said before, I believe they increase the readability of
On May 17, 2012, at 4:40 PM, Dirk Pranke wrote:
> On Thu, May 17, 2012 at 4:16 PM, Maciej Stachowiak wrote:
>>
>> Let's take an example. "TEXT" next to a test name apparently means that the
>> text fails. There is no way in the world I would guess that just from
>> reading an expectations fi
On Thu, May 17, 2012 at 4:50 PM, Maciej Stachowiak wrote:
>
> On May 17, 2012, at 4:40 PM, Dirk Pranke wrote:
>
> > On Thu, May 17, 2012 at 4:16 PM, Maciej Stachowiak
> wrote:
> >>
> >> Let's take an example. "TEXT" next to a test name apparently means that
> the text fails. There is no way in
On Thu, May 17, 2012 at 4:50 PM, Maciej Stachowiak wrote:
>
> On May 17, 2012, at 4:40 PM, Dirk Pranke wrote:
>
>> On Thu, May 17, 2012 at 4:16 PM, Maciej Stachowiak wrote:
>>>
>>> Let's take an example. "TEXT" next to a test name apparently means that the
>>> text fails. There is no way in the
Closing the loop...bug filed
https://bugs.webkit.org/show_bug.cgi?id=86796 minus
item 2 since that turned out to be controversial.
On Thu, May 17, 2012 at 1:34 PM, Ojan Vafai wrote:
> We are arguing about too many orthogonal things at once. It seems like
> there are a few things we can agree on.
On May 17, 2012, at 5:39 PM, Dirk Pranke wrote:
> I probably polarized things by saying that your input was less
> valuable than those of people who were long-time users. I did not mean
> to offend, and I'm sorry. I certainly didn't mean to imply that I was
> not listening or not open to feedbac
63 matches
Mail list logo
