Peter,
I see you clearly understand what I wanted to get into and don't give up
because of my aggressive tone in the first place.
Am 26.10.15 um 23:53 schrieb Peter Uhnák:
On Mon, Oct 26, 2015 at 8:27 AM, jtuc...@objektfabrik.de
<mailto:jtuc...@objektfabrik.de> <jtuc...@objektfabrik.de
<mailto:jtuc...@objektfabrik.de>> wrote:
Am 25.10.15 um 16:33 schrieb Peter Uhnák:
> assert:equals: it's just more typing than #= with no
additional outcome
I also disagree, but that may be also because maybe we write
tests for different purpse.
If you write tests just to test your code, then whatever... I
don't do that so I can't comment on that.
However if you do TDD, then tests are to provide feedback, and
from that perspective you want to see immediately what is the
difference. If I see that the assertion failed I have to start
digging to find out what is the difference, which is extra work
and bad from feedback perspective. If I instead immediately see
what I need to see it allows me to faster deduce the source of
the problem and resolve it.
I'm sorry, but what you are saying doesn't make any sense. Even if
I "only" want to test code (which is exactly what you do in TDD,
btw.), I need good feedback.
Well, that was a bit of generalization on my part. The point was, that
many people write tests as an afterthought... which means they have
different means of getting feedback from the system (e.g. logging,
running manually the code, etc.). In such situation they often end up
testing already mostly working code, and thus the feedback from the
tests is not as important, because you end up less digging up problems
(since large portion were found and resolved during manual testing).
I don't think there really is a difference between test first and tests
as an afterthought. You always need good and precise feedback. What you
say about having learned much more during development of a body of code
is not necessarily true in teams or when you maintain software that has
grown for years or decades, Remember, there are Smalltalk systems out
there whose development started in the late 80s or early 90s. In fact,
most commercial Smalltalk projects out in the wild are rather old.
If you have to make sure you don't break anything by putting a system
under test, you need precise feeddback, because errors can happen in
areas you haven't touched in the last 5 or 10 years or even never before.
There is, btw, another area in which SUnit can be extremely helpful:
Understanding an existing body of code and proving your theories about
code to yourself while you dive deeper. Here, feedback is also very
important.
So let's just assume that feedback of SUnit can never be too specific
and is one of the very most important aspects of the framework.
We could think about subclassing TestFailure and a way to hand
information to the TestFailure so that a nice String can be
produced. Like a method like cull: that adds arguments'
printString representation into the failure description.
You mean #assert:description: ? Because we already have that.
Well, I actually mean a new generation of #assert:description: that
concentrates more on the #description: part. Remember: some assertion
methods have been added as a way to provide "more suitable" feedback,
like handing in an object or two to be mentioned in the description
string. So the problem at hand wasn't that the standard assert: method
was insufficient, just its output.
I clearly understand how this can happen. You want to provide a better
log output and since you are testing something that even others might
need, you just add that damn method that writes a nice "expecte $A but
got $B".
In fact, this solves a problem in the wrong way, because it will result
in many more or less well-named assertion methods that basically are
there to provide a better failure String.
So what we have is a way to display a description without parameters.
But we sometimes want more than that. In case of #assert:equals: we
wanted a String that says "Expected $A and got $B". Let's not speculate
whether this String really saves us much time if we are in a TDD
scenario or on a build server. Let's just assume it is never bad to know
as much as possible about a problem to do something about it.
Let's also not forget that Kent Beck had this clear idea from the very
beginning of SUnit that it must be lightweight and easy to understand to
not shy people away from trying. You must be able to use SUnit within an
hour and it must be easy and provide instant feedback. If I have to type
too much, or keep too many things in mind, I won't use it.
Please step back for a second and think again: these are two very
different things. The job of an Assertion is to make a problem
visible. The representation of the problem is something else, even
if these are closely related. This is object thinking lesson #2 or so.
Object thinking lesson #3 tells me that I should not care about what
is going on behind the curtains.
Well, that is probably not really true for unit tests, because you may
want to test something in a very specific way to prove your thesis about
how to solve a problem is true under several circumstances.
And while I could explicitly separate the two, if I do it all the time
I don't see what's bad about having a convenience method. (And by
looking at Object or String protocol, Pharo is a lot about having
convenience over engineering rigidness).
Convenience methods are not bad per se, as long as they add value and
clearly communicate what they do. Pharo and other Smalltalks are stuffed
with lots of bad examples here. Sometimes Convenience methods are a
symptom of design problems or the lack of multiple inheritance.
Sometimes they are just a good idea.
But: adding more and more misnamed and misleading assertion
methods makes the use of SUnit frustrating and will make it
obsolete over time. If I have to hunt for design problems in SUnit
because it assumes something to be wrong even though my
understanding of waht I tested is different, I lose way more time
than I am ready to accept. This doesn't happen to me often. If
finding that I misunderstood an assertion method means I lost a
few hours, the best thing that may happen is that I never use that
method again. In the worst case, I decide I think SUnit is useless
for me. That would be really bad, don't you think?
I am not sure if we are talking about the same SUnit. Sure, there are
32 methods in the "asserting" protocol, however most of them are
either opposites of one another "#assert: vs #deny:", or they provide
some customization such has "#assert:description:", "#should:raise:"
... so if I count only meaningfully different methods the number 7
(not to mention that some of the methods are not even used). But if
you have trouble understanding the purpose of seven methods, then the
problem is on your end, and don't blame SUnit for it.
I think should:/shouldnt are great examples of convenience methods that
are not named good enough. I know nobody who can use them without taking
a second to think about them.
So what you are saying is that SUnit already has roughly 25 convenience
methods that aren't used or hard to understand. And the ones we
understand are used because they are a good shortcut to a better
description string (assert:equals:). So we are not far away from each
other... ;-)
> assertCollection:hasSameElements:
> So, let's start by asking what the question really means.
Does it mean that one collection is a subset of the other?
What about ordering then? Does it mean both contain the same
elements at the same position or just the fact that if they
both were Sets contained the exact same elements. The
question itself is not exact, so how could an answer possibly be?
There is no question about this method, since this is implemented
in code there is nothing ambiguous. This method effectively
converts both arguments to sets and compares them like that.
Sorry to say that, but this is ambiguity by design: you define
hasSameElements: as "both result in the same Set". So the name of
this assertion method is a great example of bad naming, IMO.
Yes, the naming is confusing. My point was, that instead of
philosophizing about the meaning you can look at the code. Of course
if you use the method for the first time (like I did), you will get
burned by it (as I did).
Hmm - just by reading these 3 last sentences, you see that naming is
important. But we don't disagree here. Let's forget about the concrete
naming of the method from now on. It's an example.
My point was that if the naming of a method opens room for speculation
and philosophy, it is most likely either named badly or it is a bad idea
in itself. Maybe it is not a candidate for addition as a core method in
SUnit. SUnit should make you think about your code, not about itself.
self assert: result asOrderedCollection asSortedCollection
equals: (1 to: 10) asOrderedCollection
This is what I usually do now (although I convert to Array, not
OrderedCollection, because the expected one is usually created by
hand with #() or {} ).
I don't really care. If what you try t say is that the testing
code can be ugly and long, then I agree. If you need tests like
this very often and want something to make this easier, I
understand and agree that some additions to SUnit can be helpful.
But the way this has been tried so far seems completely wrong to me.
How would you test it then? Some problem domains deal with certain
kind of problems more than others and thus benefit more from
appropriate assertions.
This is not about how to test for something. Our discussion is about the
question whether a certain wayto test something should be added to the
framework or not. Is the test generic and important enough so that it
should make its way into SUnit. And if it should be added, what is a
good name for it, so that others find the method as the right thing to
use for their purpose or will it disturb them in their proces more than
it helps.
You can always use helper objects and non-SUnit-classes in your test
packages for generalizations of your project's needs. If you need to
walk trees a lot for your purposes, you can add Visitors and stuff, to
your test package, put them under test to make sure they work and use
them in your assertions. This is not necessarily a candidate for
inclusion in SUnit, just because chances are others need to test Trees
as well.
> Just a few weeks ago, we discussed something similar about
the equality of two Collections. Endless discussions where
people try convince others that their definition of equality
and/or sameness is correct with no outcome.
I don't see a problem with that because collections truly can
have different equalities based on the context and their purpose.
And while you can call this rat poison, it effectively tells what
kind of behavior you expect from the collections, which seems ok.
So what, again, was the point of naming a method after a general
Collection class and use a question that is very unspecific? A
Collection has the same elements as another does not necessarily
mean they both result in the same set. Can we agree on that? All
the question asks if all Elements in Collection A can also be
found in Collection B. The method name states nothing more than that.
We agree that the method is badly named. (However what equality of two
Collections means is context-dependent.)
Phew. Thank you. I am glad we agree on this. So let's carry on with the
SUnit related stuff.
My point here is that a general purpose framework like SUnit
should be free of such debatable things. SUnit has to be reliable
and understandable.
There is nothing wrong with providing some "plugins" for problems
like Collections that make life easier.
It would be desirable to have more control over SUnit's feedback
with little typing.
Well but we need a way to provide context for the assertion. If I
could type less then I would be happier user, however currently I
don't see a way how to make it more general (so I don't need to type)
and more precise (it still understands context) at the same time.
Now I have you in the corner I want you in ;-)
We should talk about ways to make it easier to provide clearer feedback
Strings
So going back to your earlier suggestion with the class... you imagine
something like this?
self assert: a equals: b strategy:
CollectionHasExactlyTheSameElementsAsAnotherAtTheSamePosition
No. For several reasons (based on how I interpret the names you provide
as an example):
* You are back to the idea that you need a way to specify a specialized
testing strategy. That is exactly the wrong thought here. What and how
to test is highly dependent on your code base and needs. A simple
#asserts: and your very special way of testing formulated either by
writing complex code or using your own specialized helpers is the only
way to keep SUnit lean and mean (BTW, there is nobody keeping us from
providing SUnit helper packages that make life easier, like the Tree
example - but not as part of core SUnit)
* what if I need a test to prove/check that two collections are NOT the
same in a very special case? would you add a #deny:equals:strategy:
also? It will blow the testing protocol up from 32 to 100 very soon.
I would like to start a discussion on ways to separate the assertion
stuff from the feedback generation. The first steps I can think of are
not sufficient in the "little typing" arena, and I must admit I have not
yet found a good idea.
But to start with something, I'd like to start with the assumption that
we only have #assert: and all we want is a second parameter, like in
#assert:description:
So I am now just brainstorming, not saying this is a good solution already.
Let's assume we have some way to add parameters into Strings, like
#bindWith: and friends (or Mustache, or whatever). Let's also assume
there is something like Block>>#cull: that makes it easy to fill such a
String with any number of arguments (e.g. 'I accept up to %1 arguments'
cull: {arg1, arg2}).
Couldn't we just put some machinery into the test runner that uses this
facility to provide better feedback? like in: #assert:failureText:
'Expected %1 but got %2'.
At first sight, this is not so easy, because we need a place to store
the arguments into, and since assert: just accepts a Block or simply the
result of executing an expression, there is no such thing at hand (if it
was easy, SUnit would have had such a feature from the beginning).
So the assertion needs to be at least a Block or an object which offers
its arguments to the "feedback machine" as well.
The api might change from assert: to something like #assertExecuting:
aBlockOrObject with: listOfArguments resultsIn: expectedResult
Typing this, I see that this is not nice, because you cannot just write
some expression like self assert: myObject *3 = 15. You'd have to write
assertExecuting: [:myObject| myObject * 3] with: 5 resultsIn: 15
Adding the feedback generation into makes the API even more complicated,
because you'd have to ar least pass a String as a fourth parameter.
So we'd end up with:
assertExecuting: aBlock with: listOfArguments resultsIn: expectedResult
otherwise: failureString.
In this form, I am sure nobody would use it. Or just for very special
and important tests. So all that I can come up with for now is
insufficient, unfortunately. Sure, there can be variants that provide
some standard parameters for most of the arguments, but the base problem
remains that the API is clumsy.
So what I am just showing is that what has happened with the assert:
variants was the best thing one could possibly do without making SUnit
heavy-weight and unpleasant to use.
Does it mean I am wrong? Does it mean it would be a good idea to put
some more thought into this? Does it mean we should just go on and add
more convenience methods? Does it mean we shouldn't change SUnit and
keep it as it is?
I think it means we shouldn't just go on adding new convenience methods,
but think about ways to improve its internals and make it more flexible
in its feedback generation facilities.
Maybe we'll end up in something that you can use as a base
implementation for today's #assert: method...
1. #assertCollection:hasSameElements: is a bad name, because it treats
arguments as sets, not generic collections
it could be renamed or removed (no code in Pharo itself uses it (only
Spec, but the use there is wrong))
let's put a check mark on this one. It was just an example
2. Providing selectors such as #assertCollection:equals: and similar
just to get different description may not be the best idea, which
leads to point 3
I think I made it clear that this will be a less than perfect solution
that doesn't jump far enough
3. An easy way to extend printing/assertions/plugins; but what would
that look like?
I am convinced this is the way to go. But I have no good answer to it.
Just a spark for now.
Maybe
#assert: a = b description: [ Diff collectionDiffBetween: a and: b ]
Possibly the best thing we can come up with for now. By repeating the
arguments in the feedback generation part, we keep the complexity out of
the test runner's internals
or
#assert: a equals: b strategy:
CollectionHasExactlyTheSameElementsAsAnotherAtTheSamePosition
See above. Better than another assert:-variation, but still too heavy
or
something else...
Peter
Joachim
--
-----------------------------------------------------------------------
Objektfabrik Joachim Tuchel mailto:jtuc...@objektfabrik.de
Fliederweg 1 http://www.objektfabrik.de
D-71640 Ludwigsburg http://joachimtuchel.wordpress.com
Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1
