On 20 Mar 2010, at 10:26, Noah Slater wrote:
> Jan, should this block the release? From what I can tell, it should.
I don't think a faulty test case should block the release.
Cheers
Jan
--
>
>
>
> On 20 Mar 2010, at 11:32, Robert Dionne <dio...@dionne-associates.com> wrote:
>
>>
>>
>>
>> On Mar 19, 2010, at 7:25 PM, Jan Lehnardt wrote:
>>
>>>
>>> On 19 Mar 2010, at 18:07, J Chris Anderson wrote:
>>>
>>>>
>>>> On Mar 19, 2010, at 11:43 AM, Paul Davis wrote:
>>>>
>>>>> On Fri, Mar 19, 2010 at 2:02 PM, Jan Lehnardt <j...@apache.org> wrote:
>>>>>>
>>>>>> On 19 Mar 2010, at 12:50, Noah Slater wrote:
>>>>>>
>>>>>>>
>>>>>>> On 19 Mar 2010, at 17:11, Jan Lehnardt wrote:
>>>>>>>
>>>>>>>> We want to test the CouchDB code, not the browser's HTTP handling.
>>>>>>>
>>>>>>> Sure, but as one of CouchDB's primary interfaces is the browser, it
>>>>>>> seems to makes sense that we would want to test how this works. Testing
>>>>>>> from the browser allows us to test for and catch problems introduced by
>>>>>>> caching, etc - which is what our real world users would be running into.
>>>>>>>
>>>>>>> Unless I'm missing something?
>>>>>>
>>>>>> I fully agree, but we should have a separate browser interaction
>>>>>> suite for that. The test suite is a very untypical browser client and
>>>>>> doesn't really test real-world browser use-cases.
>>>>>>
>>>>>> Cheers
>>>>>> Jan
>>>>>> --
>>>>>
>>>>> +a bajillion.
>>>>>
>>>>
>>>> I prefer the browser tests because I'm much happier with JavaScript.
>>>
>>> I'm not saying we should get rid of the browser tests. But intermittent
>>> errors
>>> in the current test suite are not to be worried about to block a release.
>>
>> I agree with all the comments about separation of tests and so forth. This
>> particular changes test is not intermittent, it consistently fails (on my
>> machine), enough that it's a pleasant surprise when it succeeds. When
>> running from the CLI I get the following:
>>
>> not ok 10 changes expected '3', got '1'
>>
>> When running in FF I also get the message above and occasionally:
>>
>> • Exception raised:
>> {"message":"JSON.parse","fileName":"http://127.0.0.1:5984/_utils/script/couch_test_runner.js?0.11.0","lineNumber":154,"stack":";(false)@http://127.0.0.1:5984/_utils/script/couch_test_runner.js?0.11.0:154\u000arun(-2)@http://127.0.0.1:5984/_utils/script/couch_test_runner.js?0.11.0:83\u000a"}
>>
>> I haven't looked into it closely to find the root cause, it might just be
>> the test, but it's definitely not intermittent. From the CLI it happens
>> almost always
>>
>>
>>
>>>
>>> If we want proper browser client testing, we'd need an additional test suite
>>> that covers common and uncommon use-cases. I believe the current test
>>> suite is as untypical as a browser client can be.
>>>
>>> Cheers
>>> Jan
>>> --
>>>
>>>
>>>>
>>>> But maybe I'm crazy
>>>>
>>>>
>>>>> I think its important to maintain *some* tests in the browser to test
>>>>> its ability to use CouchDB as a client, but we should put more work
>>>>> into separating API tests and core tests.
>>>>>
>>>>> Also, Zed Shaw has a very informative (and colorful) description of
>>>>> confounding factors [1]. Its about two thirds of the way down under a
>>>>> heading of "Confounding, Confounding, Confounding."
>>>>>
>>>>> http://www.zedshaw.com/essays/programmer_stats.html
>>>>
>>>
>>
