You're right. We should never have "expected
failure" type tests in a test suite so that we can get back to things we
know we want to fix. That is so crazy; what are we thinking here…ha ha ha :) Of course you should test non-happy path / expected failure / exception condition. But c
reating
failing test to be patched who-knows-when... Continuous
integration server fails every time but it's fine (are you using CI?). All the time you have to check
if tests that fail are the one that suppose to fail or not (steals time / error prone)... How can developer know if the codebase in svn is not broken? - only by comparing list of failures with list of expected failures. And you guys have to make this comparison every time you evaluate someone's patch...
And as for a projects choice of how to define
tests impacting that projects credibility in *your
projects* mind… Well, lets just say I now have a severe
impacting regarding your project's credibility ;)
ha ha :) let's defend my credibility ;p - Years ago I tried approach of committing into source control deliberately failing test cases corresponding to particular log in issue management tool... It just doesn't work in CI based environment. I see no reason of creating only testcase (w/o fix) since you have the information about the bug in jira. You defer the bugfix to vague future... when something changes regarding the bug on jira you have to update testcase... Bug should be covered with test, then fixed, then checked into svn... Does having failing testcases of known bugs is a reason to be proud?
Perhaps you are encouraging contributors to fix bugs by creating failing testcase's? BTW is it working?You may have process of estimation/analysis of a jira log with the output of failing testcase. If it's working for you - that's great. But in my opinion developer should have a clear understanding of stable code base which is green color on junit progress bar. And the development should be red - green - refactor not red - red - refactor.
And do you rather want us to remove tests for known issues ?I'd prefer refactor to separate source folder, perhaps not taking part in main build and in future not checking into svn without an actual bugfix :)
Thanks,Szczepan FaberOn 6/9/06, Max Rydahl Andersen [EMAIL PROTECTED] wrote:
The day you write a (needed and usefull!) unittest that is not possible in our current setup then lets talk ;)
I've already created patch with couple testcases using same package layout on purpose.ok. No reason to change what just works. reasons: every time the developer cannot unit test non-public method /
class w/o public constructor. (every day :) ?)well, it has never been an issue since we have more than enough tests thatdoes this, so again it just works. Anyway I will just contribute a patch and let's see what you say...
ok. PS Whatever you say, the failing tests / unreasonable test packaging just impact the project credibility. But it's just my opinion and my collegues.unreasonable test packaging ? Nothing *prevents* you from using another
layout - andsince our testsuite contains considerable more test than I've seencompared to otherapplications/frameworks it doesn't seem to be an issue in real life vs.theoretical rants.And do you rather want us to remove tests for known issues ? That sounds
like you wantus to hide the fact we know some part has a bug/issue ? how is that forcredibility ?/max Thanks, Szczepan On 6/9/06, Max Rydahl Andersen
[EMAIL PROTECTED] wrote: b) But what's the reason of making surprising test subpackage (I've never seen something like that)? You can still have integration/acceptance
test cases in 'normal' package or even in different source folder. Unreasonable subpackage makes it hard to write real unit test, you cannot test non public methods, you cannot instantiate some classes etc. Don't you
have a refactoring plan to remove test subpackage? No reason to change what just works. The day you write a (needed and usefull!) unittest that is not possible
in our current setup then lets talk ;) /max Thanks, SzczepanOn 6/8/06, Max Rydahl Andersen
[EMAIL PROTECTED] wrote:1. Why there are about 10 failing test after getting project from svn?
a) if the method ends in FailureExpected, then it is an expected failure which represents a known bug/issue. To make the test pass, fix the bug ;)
b) others depend on your db, but for the moment I only have failureExpected methods.2. Why do you keep test files in strange
org.hibernate.test package? Shouldn't it be same package as sources (e.g. org.hibernate...) Not strange at all and there is no need to have them in the same
package. Alot of our tests is usecase based tests which does not fit 100% into the implmentation layout.
-- -- Max Rydahl Andersen callto://max.rydahl.andersen Hibernate
[EMAIL PROTECTED] http://hibernate.org JBoss Inc [EMAIL PROTECTED]
-- -- Max Rydahl Andersen callto://max.rydahl.andersen Hibernate
[EMAIL
