On Wed, Oct 19, 2011 at 5:00 PM, Sam Bisbee <s...@sbisbee.com> wrote: > On Wed, Oct 19, 2011 at 3:52 PM, Jan Lehnardt <j...@apache.org> wrote: >> >> On Oct 19, 2011, at 21:25 , Sam Bisbee wrote: >> >>> Yeah, it looks like it's just an unstable test. Which makes me wonder >>> why we even have it or include it in the release process. >> >> It is only unstable in certain environments and we are constantly improving >> all tests to be more stable in all environments, but the browser world is >> harsh :) > > I'm all for improvement. :) But is this improvement being tracked > anywhere, JIRA or otherwise? And if not, then could it be? I'm sure > that folks (self included) would be willing to pitch in if we knew > what had to be done. >
There was a thread about splitting Futon tests out and making them proper CLI tests that are run as part of `make check` and become actual release blockers. I have often and loudly voiced my support of this approach. I refer you to the number of varying reports for 1.1.1 with various browsers as Exhibit A in favor of this switch. >> I'd still consider having the test better than not having it at all :) > > I semi remember this being discussed before, but are the Futon tests > meant to test CouchDB, Futon, or the integration of the two? > The test suite is there to test CouchDB. There's some merit to testing Futon/Browser access as well. Jan's recent approach to the "verify installation" seems to me like a good way forward. We could use a small number of tests that check various browser capabilities and what not. But I think a large amount of the Futon test code should be permanently removed from the browser based test runner. > Cheers, > > -- > Sam Bisbee >
