Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Richard Sargent]

Ben, I think you understand the dilemma.

My philosophy is to avoid claims that are known to be not provably true.
i.e. don't colour it green, because the test runner cannot claim that
knowledge.

I am agnostic to whether it should be left uncoloured or coloured red to
document the fact that it failed (as in was unable to run to completion
without a problem). I truly don't care.

Our profession has a tendency to make claims which aren't provably true. It
is one of the many reasons we have poor software. So I am adamant about
avoiding that particular mistake. And, I am adamant about dissuading others
from the same kinds of mistakes.

Write software which conveys exactly what it can claim without Deus ex
machina intervention. The test failed. It was unable to run to completion
without an error. It doesn't matter whether that particular error was
corrected or not. The software running the test is incapable of determining
whether the change that allowed the test to run to completion was code,
data editing, or something else and it is incapable of knowing whether
whatever allowed the test to finish is a correct fix for the error.

Be adamant about what your software can know to be true. Equally important,
be adamant about your own approach to problems and solutions. Godel had
some good advice.

On Nov 23, 2017 19:31, "Ben Coman"  wrote:

>
>
> On 16 November 2017 at 00:20, Richard Sargent  gemtalksystems.com> wrote:
>
>> I think that setting the button back to gray is a good behaviour.
>>>  - it is the same thing that happens once you modify a method (which is
>>> what is happening during debugging)
>>>  - it explicitly says "please rerun the test because you may have
>>> introduced side effects"
>>>
>>
>> I would go a little further. Any method modified by the developer during
>> the course of running a test voids the ability to claim the test succeeded.
>> Likewise, for any object editted in an inspector.ere
>>
>
> So to summarise the viewpoints as I understand them, consider an
> interrupted test that later runs to completion
> and is then...
>
> A. incorrectly marked red (or grey) [false negative]
> ==> frequent during TDD
> ==> need to manually rerun test *every* time
> ==> large extra effort for developer
>
> B. incorrectly marked green [false positive]
> ==> infrequent  (presumed)
> ==> error picked up anytime test is run again, or when test group is run
> ==> small extra effort for developer
> ==> developer may be aware when they make suspect changes which undermine
> test result, to judge to run it again.
>
> C. test is automatically rerun a second time
> ==> infrequently some tests run too long for this to be practical
>
> A and B are like the philosophical difference between engineers and
> scientists.  i.e. engineers deal with approximations** that make the design
> process more efficient.
> For C, I still don't fully understand the concrete problem.  How much time
> do such tests take?
>
> Anyway, I prefer early efficiency with late bound correctness, so my vote
> would be to avoid A.
>
>
> In other words, if the test run was interrupted, it cannot be considered
>> successful. Keep things simple. (as simple as possible)
>>
>
> There are costs associated with perfect correctness.  i.e. costs
> associated with both false-negatives and false-positives.
> The alternate approach is to consider the consequence of temporary
> false-positive.
> I'd vote for making things as simple and *efficient* as possible.
>
> cheers -ben
>
>
> **A mathematician, a scientist, and an engineer are given the task of
> finding how high a particular red rubber ball will bounce when dropped from
> a given height onto a given surface.
> The mathematician derives the elasticity of the ball from its chemical
> makeup, derives the equations to determine how high it will bounce and
> calculates it.
> The physicist takes the ball into the lab, measures its elasticity, and
> plugs the variables into a formula.
> The engineer looks it up in his red rubber ball book.
>
>
>
>
>>
>>
>> On Wed, Nov 15, 2017 at 2:49 AM, Guillermo Polito <
>> guillermopol...@gmail.com> wrote:
>>
>>>
>>>
>>> On Wed, Nov 15, 2017 at 11:41 AM, Denis Kudriashov >> > wrote:
>>>

 2017-11-15 11:08 GMT+01:00 Guillermo Polito 
 :
>
> On Wed, Nov 15, 2017 at 11:06 AM, Denis Kudriashov <
> dionisi...@gmail.com> wrote:
>
>>
>>
>> 2017-11-15 11:00 GMT+01:00 Guillermo Polito <
>> guillermopol...@gmail.com>:
>>
>>> And just putting it back to gray? As "not run"?
>>>
>>
>> We can implement any logic.
>> Personally I need current behaviour.
>>
>
> But it is not about you personally. It is about implementing the most
> common and the less strange for newcomers.
>

 To know what is the most common case people should tell personal
 opinion.
 And in this thread only Richard was 

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Ben Coman]

On 16 November 2017 at 00:20, Richard Sargent <
richard.sarg...@gemtalksystems.com> wrote:

> I think that setting the button back to gray is a good behaviour.
>>  - it is the same thing that happens once you modify a method (which is
>> what is happening during debugging)
>>  - it explicitly says "please rerun the test because you may have
>> introduced side effects"
>>
>
> I would go a little further. Any method modified by the developer during
> the course of running a test voids the ability to claim the test succeeded.
> Likewise, for any object editted in an inspector.ere
>

So to summarise the viewpoints as I understand them, consider an
interrupted test that later runs to completion
and is then...

A. incorrectly marked red (or grey) [false negative]
==> frequent during TDD
==> need to manually rerun test *every* time
==> large extra effort for developer

B. incorrectly marked green [false positive]
==> infrequent  (presumed)
==> error picked up anytime test is run again, or when test group is run
==> small extra effort for developer
==> developer may be aware when they make suspect changes which undermine
test result, to judge to run it again.

C. test is automatically rerun a second time
==> infrequently some tests run too long for this to be practical

A and B are like the philosophical difference between engineers and
scientists.  i.e. engineers deal with approximations** that make the design
process more efficient.
For C, I still don't fully understand the concrete problem.  How much time
do such tests take?

Anyway, I prefer early efficiency with late bound correctness, so my vote
would be to avoid A.


In other words, if the test run was interrupted, it cannot be considered
> successful. Keep things simple. (as simple as possible)
>

There are costs associated with perfect correctness.  i.e. costs associated
with both false-negatives and false-positives.
The alternate approach is to consider the consequence of temporary
false-positive.
I'd vote for making things as simple and *efficient* as possible.

cheers -ben


**A mathematician, a scientist, and an engineer are given the task of
finding how high a particular red rubber ball will bounce when dropped from
a given height onto a given surface.
The mathematician derives the elasticity of the ball from its chemical
makeup, derives the equations to determine how high it will bounce and
calculates it.
The physicist takes the ball into the lab, measures its elasticity, and
plugs the variables into a formula.
The engineer looks it up in his red rubber ball book.




>
>
> On Wed, Nov 15, 2017 at 2:49 AM, Guillermo Polito <
> guillermopol...@gmail.com> wrote:
>
>>
>>
>> On Wed, Nov 15, 2017 at 11:41 AM, Denis Kudriashov 
>> wrote:
>>
>>>
>>> 2017-11-15 11:08 GMT+01:00 Guillermo Polito :

 On Wed, Nov 15, 2017 at 11:06 AM, Denis Kudriashov <
 dionisi...@gmail.com> wrote:

>
>
> 2017-11-15 11:00 GMT+01:00 Guillermo Polito  >:
>
>> And just putting it back to gray? As "not run"?
>>
>
> We can implement any logic.
> Personally I need current behaviour.
>

 But it is not about you personally. It is about implementing the most
 common and the less strange for newcomers.

>>>
>>> To know what is the most common case people should tell personal opinion.
>>> And in this thread only Richard was against current logic.
>>>
>>
>> But you're assuming here that:
>>  - people that is not reading this email do not care and don't have a say
>>  - so pleople that is not subscribed to the mailing list don't care
>>  - and that includes newbies
>>
>> Our role of experienced guys it not only look after "our" best defaults.
>> But also after the defaults of people without experience.
>>
>> I think that setting the button back to gray is a good behaviour.
>>  - it is the same thing that happens once you modify a method (which is
>> what is happening during debugging)
>>  - it explicitly says "please rerun the test because you may have
>> introduced side effects"
>>
>> Unless you make the debugger more intelligent, you cannot be sure that
>> the result you obtained at the end of the test is really reproducible. And
>> moreover, to be able to make such assumption you should be an expert that
>> understands how the underlying framework behaves.
>>
>>
>>>
>>>


>
>
>>
>> On Wed, Nov 15, 2017 at 10:44 AM, Denis Kudriashov <
>> dionisi...@gmail.com> wrote:
>>
>>> 2017-11-15 1:49 GMT+01:00 Sean P. DeNigris :
>>>
 Ben Coman wrote
 > Or it could go to Amber, half-way between green & red to mean
 probably
 > correct.

 Ha ha.

 Again, it seems that just automatically rerunning the test
 immediately after
 a human-manipulated run and setting the color based on that second
 run

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Stephane Ducasse]

BTW I like grey for a method that was edited in the debugger.
Because red is not good. The proof is that when I rerun my test is green.
While saying I do not know you have to rerun is a good solution.
I like it.

I got always frustrated with the red.

On Fri, Nov 24, 2017 at 12:05 AM, Stephane Ducasse
 wrote:
> Hi andrew
>
> I like your idea.
> It is fun and at least we can spot test.
> Now sometimes we will have to have self assert: true because running a
> code can be also considered as a test.
> Now what I do not like is empty test method because there are green.
>
> stef
>
> On Tue, Nov 21, 2017 at 10:07 PM, Prof. Andrew P. Black
>  wrote:
>> While we are discussing colors, what should we do about a test that does not 
>> make any assertions at all?
>>
>> A couple of years ago, a smart student who was working on a testing dialect 
>> for me decided that such tests should be in a new category all of their own. 
>>  Later I simplified things and just made these tests fail.  I am pretty 
>> convinced now that this is the right behaviour.  It certainly suits my 
>> purposes (teaching TDD); if the student omits to make any assertions, they 
>> have a failing test.  The message is: "Failure: test made no assertions".
>>
>> Currently, such a test is green in Pharo.  In 2013, I filed a bug report on 
>> a bunch of tests of #printOn: on collections that made no assertions.  (The 
>> test writer didn’t understand how streams worked, and made assertions for 
>> each element of the empty string.)  I was reminded of this just this week, 
>> because those tests have yet to be fixed.  I suspect that if they had been 
>> yellow, rather than green, then they would have been fixed before now.
>>
>> I plan to fix those tests on Friday, but I also wonder about changing the 
>> behaviour of the testing framework.  What do you think?
>>
>> Andrew
>>
>>

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Stephane Ducasse]

Hi andrew

I like your idea.
It is fun and at least we can spot test.
Now sometimes we will have to have self assert: true because running a
code can be also considered as a test.
Now what I do not like is empty test method because there are green.

stef

On Tue, Nov 21, 2017 at 10:07 PM, Prof. Andrew P. Black
 wrote:
> While we are discussing colors, what should we do about a test that does not 
> make any assertions at all?
>
> A couple of years ago, a smart student who was working on a testing dialect 
> for me decided that such tests should be in a new category all of their own.  
> Later I simplified things and just made these tests fail.  I am pretty 
> convinced now that this is the right behaviour.  It certainly suits my 
> purposes (teaching TDD); if the student omits to make any assertions, they 
> have a failing test.  The message is: "Failure: test made no assertions".
>
> Currently, such a test is green in Pharo.  In 2013, I filed a bug report on a 
> bunch of tests of #printOn: on collections that made no assertions.  (The 
> test writer didn’t understand how streams worked, and made assertions for 
> each element of the empty string.)  I was reminded of this just this week, 
> because those tests have yet to be fixed.  I suspect that if they had been 
> yellow, rather than green, then they would have been fixed before now.
>
> I plan to fix those tests on Friday, but I also wonder about changing the 
> behaviour of the testing framework.  What do you think?
>
> Andrew
>
>

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Richard Sargent]

A test that asserts nothing has only asserted that the code ran through
without throwing an error. :-)

I like your proposal that such a test is an inherently failing test.


(Of course, that will result in the student adding a single assertion at
the end, of the form "self assert: true"!)

On Tue, Nov 21, 2017 at 4:07 PM, Prof. Andrew P. Black 
wrote:

> While we are discussing colors, what should we do about a test that does
> not make any assertions at all?
>
> A couple of years ago, a smart student who was working on a testing
> dialect for me decided that such tests should be in a new category all of
> their own.  Later I simplified things and just made these tests fail.  I am
> pretty convinced now that this is the right behaviour.  It certainly suits
> my purposes (teaching TDD); if the student omits to make any assertions,
> they have a failing test.  The message is: "Failure: test made no
> assertions".
>
> Currently, such a test is green in Pharo.  In 2013, I filed a bug report
> on a bunch of tests of #printOn: on collections that made no assertions.
> (The test writer didn’t understand how streams worked, and made assertions
> for each element of the empty string.)  I was reminded of this just this
> week, because those tests have yet to be fixed.  I suspect that if they had
> been yellow, rather than green, then they would have been fixed before now.
>
> I plan to fix those tests on Friday, but I also wonder about changing the
> behaviour of the testing framework.  What do you think?
>
> Andrew
>
>
>

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Prof. Andrew P. Black]

While we are discussing colors, what should we do about a test that does not 
make any assertions at all?

A couple of years ago, a smart student who was working on a testing dialect for 
me decided that such tests should be in a new category all of their own.  Later 
I simplified things and just made these tests fail.  I am pretty convinced now 
that this is the right behaviour.  It certainly suits my purposes (teaching 
TDD); if the student omits to make any assertions, they have a failing test.  
The message is: "Failure: test made no assertions".

Currently, such a test is green in Pharo.  In 2013, I filed a bug report on a 
bunch of tests of #printOn: on collections that made no assertions.  (The test 
writer didn’t understand how streams worked, and made assertions for each 
element of the empty string.)  I was reminded of this just this week, because 
those tests have yet to be fixed.  I suspect that if they had been yellow, 
rather than green, then they would have been fixed before now.

I plan to fix those tests on Friday, but I also wonder about changing the 
behaviour of the testing framework.  What do you think?

Andrew

 

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Tim Mackinnon]

I would agree that grey is better than red - but I personally think we’re being 
too pedantic on this - particularly when doing TDD and coding in the debugger. 
If I’m writing straight forward tests and like to see a red failure (either by 
deliberately returning false, or a subclass responsibility, or -1) and then 
correct that failure in the debugger to return the correct result - then its 
tedious to have to run the test again (in fact it feels odd). For the rare time 
that I got it wrong - it will show up when I run all tests on the next phase.

I accept that others may see this the other way around - but I’m a more 
optimistic guy. This said - maybe we make it an option (or an easy code switch) 
- I’d default it to the optimistic TDD mode personally. My CI server will give 
me the full lowdown.

Tim

> On 15 Nov 2017, at 21:50, Sean P. DeNigris  wrote:
> 
> Richard Sargent wrote
>> I would go a little further. Any method modified by the developer during
>> the course of running a test voids the ability to claim the test
>> succeeded.
>> Likewise, for any object editted in an inspector.
> 
> That makes sense to me.
> 
> 
> 
> -
> Cheers,
> Sean
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
> 

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Sean P. DeNigris]

Richard Sargent wrote
> I would go a little further. Any method modified by the developer during
> the course of running a test voids the ability to claim the test
> succeeded.
> Likewise, for any object editted in an inspector.

That makes sense to me.



-
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Denis Kudriashov]

2017-11-15 17:16 GMT+01:00 Richard Sargent <
richard.sarg...@gemtalksystems.com>:

> And in this thread only Richard was against current logic.
>>
>
> I am against colouring it green without having run the test without
> interruption or error. I don't think that is the current logic.
>

Fix is already in Pharo 7. So it is current logic now.
And it was same for manual halt (instead of failure).


>
> On Wed, Nov 15, 2017 at 2:41 AM, Denis Kudriashov 
> wrote:
>
>>
>> 2017-11-15 11:08 GMT+01:00 Guillermo Polito :
>>>
>>> On Wed, Nov 15, 2017 at 11:06 AM, Denis Kudriashov >> > wrote:
>>>


 2017-11-15 11:00 GMT+01:00 Guillermo Polito 
 :

> And just putting it back to gray? As "not run"?
>

 We can implement any logic.
 Personally I need current behaviour.

>>>
>>> But it is not about you personally. It is about implementing the most
>>> common and the less strange for newcomers.
>>>
>>
>> To know what is the most common case people should tell personal opinion.
>> And in this thread only Richard was against current logic.
>>
>>
>>>
>>>


>
> On Wed, Nov 15, 2017 at 10:44 AM, Denis Kudriashov <
> dionisi...@gmail.com> wrote:
>
>> 2017-11-15 1:49 GMT+01:00 Sean P. DeNigris :
>>
>>> Ben Coman wrote
>>> > Or it could go to Amber, half-way between green & red to mean
>>> probably
>>> > correct.
>>>
>>> Ha ha.
>>>
>>> Again, it seems that just automatically rerunning the test
>>> immediately after
>>> a human-manipulated run and setting the color based on that second
>>> run
>>> addresses all points on both sides, no?
>>>
>>
>> Except that sometimes we are debugging slow test and running it
>> second time automatically after "proceed" can be not appropriate.
>> We are talking about single test run. If user have any doubts about
>> result It is his responsibility to rerun the test. User knows what he is
>> doing when he debug and fix the test. No intelligence is required here.
>>
>> And anyway current fix just provides consistent behaviour to
>> debugging from explicit breakpoint/halt. In that case the result was 
>> always
>> in sync with debug session.
>>
>>
>>>
>>>
>>> -
>>> Cheers,
>>> Sean
>>> --
>>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>>>
>>>
>>
>
>
> --
>
>
>
> Guille Polito
>
> Research Engineer
>
> Centre de Recherche en Informatique, Signal et Automatique de Lille
>
> CRIStAL - UMR 9189
>
> French National Center for Scientific Research - *http://www.cnrs.fr
> *
>
>
> *Web:* *http://guillep.github.io* 
>
> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013>
>


>>>
>>>
>>> --
>>>
>>>
>>>
>>> Guille Polito
>>>
>>> Research Engineer
>>>
>>> Centre de Recherche en Informatique, Signal et Automatique de Lille
>>>
>>> CRIStAL - UMR 9189
>>>
>>> French National Center for Scientific Research - *http://www.cnrs.fr
>>> *
>>>
>>>
>>> *Web:* *http://guillep.github.io* 
>>>
>>> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013>
>>>
>>
>>
>

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Richard Sargent]

>
> I think that setting the button back to gray is a good behaviour.
>  - it is the same thing that happens once you modify a method (which is
> what is happening during debugging)
>  - it explicitly says "please rerun the test because you may have
> introduced side effects"
>

I would go a little further. Any method modified by the developer during
the course of running a test voids the ability to claim the test succeeded.
Likewise, for any object editted in an inspector.

In other words, if the test run was interrupted, it cannot be considered
successful. Keep things simple. (as simple as possible)


On Wed, Nov 15, 2017 at 2:49 AM, Guillermo Polito  wrote:

>
>
> On Wed, Nov 15, 2017 at 11:41 AM, Denis Kudriashov 
> wrote:
>
>>
>> 2017-11-15 11:08 GMT+01:00 Guillermo Polito :
>>>
>>> On Wed, Nov 15, 2017 at 11:06 AM, Denis Kudriashov >> > wrote:
>>>


 2017-11-15 11:00 GMT+01:00 Guillermo Polito 
 :

> And just putting it back to gray? As "not run"?
>

 We can implement any logic.
 Personally I need current behaviour.

>>>
>>> But it is not about you personally. It is about implementing the most
>>> common and the less strange for newcomers.
>>>
>>
>> To know what is the most common case people should tell personal opinion.
>> And in this thread only Richard was against current logic.
>>
>
> But you're assuming here that:
>  - people that is not reading this email do not care and don't have a say
>  - so pleople that is not subscribed to the mailing list don't care
>  - and that includes newbies
>
> Our role of experienced guys it not only look after "our" best defaults.
> But also after the defaults of people without experience.
>
> I think that setting the button back to gray is a good behaviour.
>  - it is the same thing that happens once you modify a method (which is
> what is happening during debugging)
>  - it explicitly says "please rerun the test because you may have
> introduced side effects"
>
> Unless you make the debugger more intelligent, you cannot be sure that the
> result you obtained at the end of the test is really reproducible. And
> moreover, to be able to make such assumption you should be an expert that
> understands how the underlying framework behaves.
>
>
>>
>>
>>>
>>>


>
> On Wed, Nov 15, 2017 at 10:44 AM, Denis Kudriashov <
> dionisi...@gmail.com> wrote:
>
>> 2017-11-15 1:49 GMT+01:00 Sean P. DeNigris :
>>
>>> Ben Coman wrote
>>> > Or it could go to Amber, half-way between green & red to mean
>>> probably
>>> > correct.
>>>
>>> Ha ha.
>>>
>>> Again, it seems that just automatically rerunning the test
>>> immediately after
>>> a human-manipulated run and setting the color based on that second
>>> run
>>> addresses all points on both sides, no?
>>>
>>
>> Except that sometimes we are debugging slow test and running it
>> second time automatically after "proceed" can be not appropriate.
>> We are talking about single test run. If user have any doubts about
>> result It is his responsibility to rerun the test. User knows what he is
>> doing when he debug and fix the test. No intelligence is required here.
>>
>> And anyway current fix just provides consistent behaviour to
>> debugging from explicit breakpoint/halt. In that case the result was 
>> always
>> in sync with debug session.
>>
>>
>>>
>>>
>>> -
>>> Cheers,
>>> Sean
>>> --
>>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>>>
>>>
>>
>
>
> --
>
>
>
> Guille Polito
>
> Research Engineer
>
> Centre de Recherche en Informatique, Signal et Automatique de Lille
>
> CRIStAL - UMR 9189
>
> French National Center for Scientific Research - *http://www.cnrs.fr
> *
>
>
> *Web:* *http://guillep.github.io* 
>
> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013>
>


>>>
>>>
>>> --
>>>
>>>
>>>
>>> Guille Polito
>>>
>>> Research Engineer
>>>
>>> Centre de Recherche en Informatique, Signal et Automatique de Lille
>>>
>>> CRIStAL - UMR 9189
>>>
>>> French National Center for Scientific Research - *http://www.cnrs.fr
>>> *
>>>
>>>
>>> *Web:* *http://guillep.github.io* 
>>>
>>> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013>
>>>
>>
>>
>
>
> --
>
>
>
> Guille Polito
>
> Research Engineer
>
> Centre de Recherche en Informatique, Signal et Automatique de Lille
>
> CRIStAL - UMR 9189
>
> French National Center for Scientific Research - *http://www.cnrs.fr
> *
>
>
> *Web:* *http://guillep.github.io* 
>
> *Phone: 

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Richard Sargent]

>
> And in this thread only Richard was against current logic.
>

I am against colouring it green without having run the test without
interruption or error. I don't think that is the current logic.

On Wed, Nov 15, 2017 at 2:41 AM, Denis Kudriashov 
wrote:

>
> 2017-11-15 11:08 GMT+01:00 Guillermo Polito :
>>
>> On Wed, Nov 15, 2017 at 11:06 AM, Denis Kudriashov 
>> wrote:
>>
>>>
>>>
>>> 2017-11-15 11:00 GMT+01:00 Guillermo Polito :
>>>
 And just putting it back to gray? As "not run"?

>>>
>>> We can implement any logic.
>>> Personally I need current behaviour.
>>>
>>
>> But it is not about you personally. It is about implementing the most
>> common and the less strange for newcomers.
>>
>
> To know what is the most common case people should tell personal opinion.
> And in this thread only Richard was against current logic.
>
>
>>
>>
>>>
>>>

 On Wed, Nov 15, 2017 at 10:44 AM, Denis Kudriashov <
 dionisi...@gmail.com> wrote:

> 2017-11-15 1:49 GMT+01:00 Sean P. DeNigris :
>
>> Ben Coman wrote
>> > Or it could go to Amber, half-way between green & red to mean
>> probably
>> > correct.
>>
>> Ha ha.
>>
>> Again, it seems that just automatically rerunning the test
>> immediately after
>> a human-manipulated run and setting the color based on that second run
>> addresses all points on both sides, no?
>>
>
> Except that sometimes we are debugging slow test and running it second
> time automatically after "proceed" can be not appropriate.
> We are talking about single test run. If user have any doubts about
> result It is his responsibility to rerun the test. User knows what he is
> doing when he debug and fix the test. No intelligence is required here.
>
> And anyway current fix just provides consistent behaviour to debugging
> from explicit breakpoint/halt. In that case the result was always in sync
> with debug session.
>
>
>>
>>
>> -
>> Cheers,
>> Sean
>> --
>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>>
>>
>


 --



 Guille Polito

 Research Engineer

 Centre de Recherche en Informatique, Signal et Automatique de Lille

 CRIStAL - UMR 9189

 French National Center for Scientific Research - *http://www.cnrs.fr
 *


 *Web:* *http://guillep.github.io* 

 *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013>

>>>
>>>
>>
>>
>> --
>>
>>
>>
>> Guille Polito
>>
>> Research Engineer
>>
>> Centre de Recherche en Informatique, Signal et Automatique de Lille
>>
>> CRIStAL - UMR 9189
>>
>> French National Center for Scientific Research - *http://www.cnrs.fr
>> *
>>
>>
>> *Web:* *http://guillep.github.io* 
>>
>> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013>
>>
>
>

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Guillermo Polito]

On Wed, Nov 15, 2017 at 11:41 AM, Denis Kudriashov 
wrote:

>
> 2017-11-15 11:08 GMT+01:00 Guillermo Polito :
>>
>> On Wed, Nov 15, 2017 at 11:06 AM, Denis Kudriashov 
>> wrote:
>>
>>>
>>>
>>> 2017-11-15 11:00 GMT+01:00 Guillermo Polito :
>>>
 And just putting it back to gray? As "not run"?

>>>
>>> We can implement any logic.
>>> Personally I need current behaviour.
>>>
>>
>> But it is not about you personally. It is about implementing the most
>> common and the less strange for newcomers.
>>
>
> To know what is the most common case people should tell personal opinion.
> And in this thread only Richard was against current logic.
>

But you're assuming here that:
 - people that is not reading this email do not care and don't have a say
 - so pleople that is not subscribed to the mailing list don't care
 - and that includes newbies

Our role of experienced guys it not only look after "our" best defaults.
But also after the defaults of people without experience.

I think that setting the button back to gray is a good behaviour.
 - it is the same thing that happens once you modify a method (which is
what is happening during debugging)
 - it explicitly says "please rerun the test because you may have
introduced side effects"

Unless you make the debugger more intelligent, you cannot be sure that the
result you obtained at the end of the test is really reproducible. And
moreover, to be able to make such assumption you should be an expert that
understands how the underlying framework behaves.


>
>
>>
>>
>>>
>>>

 On Wed, Nov 15, 2017 at 10:44 AM, Denis Kudriashov <
 dionisi...@gmail.com> wrote:

> 2017-11-15 1:49 GMT+01:00 Sean P. DeNigris :
>
>> Ben Coman wrote
>> > Or it could go to Amber, half-way between green & red to mean
>> probably
>> > correct.
>>
>> Ha ha.
>>
>> Again, it seems that just automatically rerunning the test
>> immediately after
>> a human-manipulated run and setting the color based on that second run
>> addresses all points on both sides, no?
>>
>
> Except that sometimes we are debugging slow test and running it second
> time automatically after "proceed" can be not appropriate.
> We are talking about single test run. If user have any doubts about
> result It is his responsibility to rerun the test. User knows what he is
> doing when he debug and fix the test. No intelligence is required here.
>
> And anyway current fix just provides consistent behaviour to debugging
> from explicit breakpoint/halt. In that case the result was always in sync
> with debug session.
>
>
>>
>>
>> -
>> Cheers,
>> Sean
>> --
>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>>
>>
>


 --



 Guille Polito

 Research Engineer

 Centre de Recherche en Informatique, Signal et Automatique de Lille

 CRIStAL - UMR 9189

 French National Center for Scientific Research - *http://www.cnrs.fr
 *


 *Web:* *http://guillep.github.io* 

 *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013>

>>>
>>>
>>
>>
>> --
>>
>>
>>
>> Guille Polito
>>
>> Research Engineer
>>
>> Centre de Recherche en Informatique, Signal et Automatique de Lille
>>
>> CRIStAL - UMR 9189
>>
>> French National Center for Scientific Research - *http://www.cnrs.fr
>> *
>>
>>
>> *Web:* *http://guillep.github.io* 
>>
>> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013>
>>
>
>


-- 



Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - *http://www.cnrs.fr
*


*Web:* *http://guillep.github.io* 

*Phone: *+33 06 52 70 66 13

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Denis Kudriashov]

2017-11-15 11:08 GMT+01:00 Guillermo Polito :
>
> On Wed, Nov 15, 2017 at 11:06 AM, Denis Kudriashov 
> wrote:
>
>>
>>
>> 2017-11-15 11:00 GMT+01:00 Guillermo Polito :
>>
>>> And just putting it back to gray? As "not run"?
>>>
>>
>> We can implement any logic.
>> Personally I need current behaviour.
>>
>
> But it is not about you personally. It is about implementing the most
> common and the less strange for newcomers.
>

To know what is the most common case people should tell personal opinion.
And in this thread only Richard was against current logic.


>
>
>>
>>
>>>
>>> On Wed, Nov 15, 2017 at 10:44 AM, Denis Kudriashov >> > wrote:
>>>
 2017-11-15 1:49 GMT+01:00 Sean P. DeNigris :

> Ben Coman wrote
> > Or it could go to Amber, half-way between green & red to mean
> probably
> > correct.
>
> Ha ha.
>
> Again, it seems that just automatically rerunning the test immediately
> after
> a human-manipulated run and setting the color based on that second run
> addresses all points on both sides, no?
>

 Except that sometimes we are debugging slow test and running it second
 time automatically after "proceed" can be not appropriate.
 We are talking about single test run. If user have any doubts about
 result It is his responsibility to rerun the test. User knows what he is
 doing when he debug and fix the test. No intelligence is required here.

 And anyway current fix just provides consistent behaviour to debugging
 from explicit breakpoint/halt. In that case the result was always in sync
 with debug session.


>
>
> -
> Cheers,
> Sean
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>
>

>>>
>>>
>>> --
>>>
>>>
>>>
>>> Guille Polito
>>>
>>> Research Engineer
>>>
>>> Centre de Recherche en Informatique, Signal et Automatique de Lille
>>>
>>> CRIStAL - UMR 9189
>>>
>>> French National Center for Scientific Research - *http://www.cnrs.fr
>>> *
>>>
>>>
>>> *Web:* *http://guillep.github.io* 
>>>
>>> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013>
>>>
>>
>>
>
>
> --
>
>
>
> Guille Polito
>
> Research Engineer
>
> Centre de Recherche en Informatique, Signal et Automatique de Lille
>
> CRIStAL - UMR 9189
>
> French National Center for Scientific Research - *http://www.cnrs.fr
> *
>
>
> *Web:* *http://guillep.github.io* 
>
> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013>
>

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Guillermo Polito]

On Wed, Nov 15, 2017 at 11:06 AM, Denis Kudriashov 
wrote:

>
>
> 2017-11-15 11:00 GMT+01:00 Guillermo Polito :
>
>> And just putting it back to gray? As "not run"?
>>
>
> We can implement any logic.
> Personally I need current behaviour.
>

But it is not about you personally. It is about implementing the most
common and the less strange for newcomers.


>
>
>>
>> On Wed, Nov 15, 2017 at 10:44 AM, Denis Kudriashov 
>> wrote:
>>
>>> 2017-11-15 1:49 GMT+01:00 Sean P. DeNigris :
>>>
 Ben Coman wrote
 > Or it could go to Amber, half-way between green & red to mean probably
 > correct.

 Ha ha.

 Again, it seems that just automatically rerunning the test immediately
 after
 a human-manipulated run and setting the color based on that second run
 addresses all points on both sides, no?

>>>
>>> Except that sometimes we are debugging slow test and running it second
>>> time automatically after "proceed" can be not appropriate.
>>> We are talking about single test run. If user have any doubts about
>>> result It is his responsibility to rerun the test. User knows what he is
>>> doing when he debug and fix the test. No intelligence is required here.
>>>
>>> And anyway current fix just provides consistent behaviour to debugging
>>> from explicit breakpoint/halt. In that case the result was always in sync
>>> with debug session.
>>>
>>>


 -
 Cheers,
 Sean
 --
 Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html


>>>
>>
>>
>> --
>>
>>
>>
>> Guille Polito
>>
>> Research Engineer
>>
>> Centre de Recherche en Informatique, Signal et Automatique de Lille
>>
>> CRIStAL - UMR 9189
>>
>> French National Center for Scientific Research - *http://www.cnrs.fr
>> *
>>
>>
>> *Web:* *http://guillep.github.io* 
>>
>> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013>
>>
>
>


-- 



Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - *http://www.cnrs.fr
*


*Web:* *http://guillep.github.io* 

*Phone: *+33 06 52 70 66 13

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Denis Kudriashov]

2017-11-15 11:00 GMT+01:00 Guillermo Polito :

> And just putting it back to gray? As "not run"?
>

We can implement any logic.
Personally I need current behaviour.


>
> On Wed, Nov 15, 2017 at 10:44 AM, Denis Kudriashov 
> wrote:
>
>> 2017-11-15 1:49 GMT+01:00 Sean P. DeNigris :
>>
>>> Ben Coman wrote
>>> > Or it could go to Amber, half-way between green & red to mean probably
>>> > correct.
>>>
>>> Ha ha.
>>>
>>> Again, it seems that just automatically rerunning the test immediately
>>> after
>>> a human-manipulated run and setting the color based on that second run
>>> addresses all points on both sides, no?
>>>
>>
>> Except that sometimes we are debugging slow test and running it second
>> time automatically after "proceed" can be not appropriate.
>> We are talking about single test run. If user have any doubts about
>> result It is his responsibility to rerun the test. User knows what he is
>> doing when he debug and fix the test. No intelligence is required here.
>>
>> And anyway current fix just provides consistent behaviour to debugging
>> from explicit breakpoint/halt. In that case the result was always in sync
>> with debug session.
>>
>>
>>>
>>>
>>> -
>>> Cheers,
>>> Sean
>>> --
>>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>>>
>>>
>>
>
>
> --
>
>
>
> Guille Polito
>
> Research Engineer
>
> Centre de Recherche en Informatique, Signal et Automatique de Lille
>
> CRIStAL - UMR 9189
>
> French National Center for Scientific Research - *http://www.cnrs.fr
> *
>
>
> *Web:* *http://guillep.github.io* 
>
> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013>
>

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Guillermo Polito]

And just putting it back to gray? As "not run"?

On Wed, Nov 15, 2017 at 10:44 AM, Denis Kudriashov 
wrote:

> 2017-11-15 1:49 GMT+01:00 Sean P. DeNigris :
>
>> Ben Coman wrote
>> > Or it could go to Amber, half-way between green & red to mean probably
>> > correct.
>>
>> Ha ha.
>>
>> Again, it seems that just automatically rerunning the test immediately
>> after
>> a human-manipulated run and setting the color based on that second run
>> addresses all points on both sides, no?
>>
>
> Except that sometimes we are debugging slow test and running it second
> time automatically after "proceed" can be not appropriate.
> We are talking about single test run. If user have any doubts about result
> It is his responsibility to rerun the test. User knows what he is doing
> when he debug and fix the test. No intelligence is required here.
>
> And anyway current fix just provides consistent behaviour to debugging
> from explicit breakpoint/halt. In that case the result was always in sync
> with debug session.
>
>
>>
>>
>> -
>> Cheers,
>> Sean
>> --
>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>>
>>
>


-- 



Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - *http://www.cnrs.fr
*


*Web:* *http://guillep.github.io* 

*Phone: *+33 06 52 70 66 13

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Denis Kudriashov]

2017-11-15 1:49 GMT+01:00 Sean P. DeNigris :

> Ben Coman wrote
> > Or it could go to Amber, half-way between green & red to mean probably
> > correct.
>
> Ha ha.
>
> Again, it seems that just automatically rerunning the test immediately
> after
> a human-manipulated run and setting the color based on that second run
> addresses all points on both sides, no?
>

Except that sometimes we are debugging slow test and running it second time
automatically after "proceed" can be not appropriate.
We are talking about single test run. If user have any doubts about result
It is his responsibility to rerun the test. User knows what he is doing
when he debug and fix the test. No intelligence is required here.

And anyway current fix just provides consistent behaviour to debugging from
explicit breakpoint/halt. In that case the result was always in sync with
debug session.


>
>
> -
> Cheers,
> Sean
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>
>

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Prof. Andrew P. Black]

> On Nov 15, 2017, at 08:30, Richard Sargent 
>  wrote:
> 
> I would say that is an entirely different question

You are right: this is a different scenario.  But the suggested fix sounds like 
a good one to me.  After editing a test, even in the debugger, it makes sense 
to run the test (again?), and display the appropriate colored balloon. 

   Andrew

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Richard Sargent]

I would say that is an entirely different question. If a test isn't stable,
you have a completely different scenario from a test that interrupts
execution and is resumed.

On Nov 14, 2017 20:45, "Stephan Eggermont"  wrote:

Op 15-11-2017 om 01:49 schreef Sean P. DeNigris:

Again, it seems that just automatically rerunning the test immediately after
> a human-manipulated run and setting the color based on that second run
> addresses all points on both sides, no?
>

How many times do you want to restart execution? I have writen enough code
where running it twice was not enough to get to green :)

Stephan

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Stephan Eggermont]

Op 15-11-2017 om 01:49 schreef Sean P. DeNigris:

Again, it seems that just automatically rerunning the test immediately after
a human-manipulated run and setting the color based on that second run
addresses all points on both sides, no?


How many times do you want to restart execution? I have writen enough 
code where running it twice was not enough to get to green :)


Stephan

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Richard Sargent]

>
> Again, it seems that just automatically rerunning the test immediately
> after a human-manipulated run and setting the color based on that second run
>

Absolutely!

(Although, I was kind of looking forward to the third colour discussions.
My vote would have been for paisley!)



On Tue, Nov 14, 2017 at 4:49 PM, Sean P. DeNigris 
wrote:

> Ben Coman wrote
> > Or it could go to Amber, half-way between green & red to mean probably
> > correct.
>
> Ha ha.
>
> Again, it seems that just automatically rerunning the test immediately
> after
> a human-manipulated run and setting the color based on that second run
> addresses all points on both sides, no?
>
>
>
> -
> Cheers,
> Sean
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>
>

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Sean P. DeNigris]

Ben Coman wrote
> Or it could go to Amber, half-way between green & red to mean probably
> correct.

Ha ha.

Again, it seems that just automatically rerunning the test immediately after
a human-manipulated run and setting the color based on that second run
addresses all points on both sides, no?



-
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Ben Coman]

On 15 November 2017 at 00:14, Richard Sargent <rsarg...@5x5.on.ca> wrote:

> >> I would expect it to turn green if I press resume.
>
> I disagree with your expectations. You are arguing that we should operate
> is if "probably correct" is the same as "correct". That's why we have
> ty software.
>
> I have no objection to leaving the method uncoloured when you resume after
> correcting the error.
> I have no objection to eliminating the red colouring in these scenarios.
> I strongly object to pretending that it is known to be correct.
>


Or it could go to Amber, half-way between green & red to mean probably
correct.

cheers -ben



>
>
> In my experience, we end up with better results when we act on what we
> absolutely know to be factual and stop relying on guessing.
>
>
> --
> *From:* Pharo-users [mailto:pharo-users-boun...@lists.pharo.org] *On
> Behalf Of *Tim Mackinnon
> *Sent:* November 14, 2017 06:19
> *To:* Any question about pharo is welcome
> *Subject:* Re: [Pharo-users] Why does the test runner show red when I
> correct a test?
>
> Richard - I better understand what you are saying now. If you change the
> method and save it (hence restarting on the stack) I would expect it to
> turn green if I press resume. That is the TDD flow I expect, and one which
> sunit and coding in the debugger was predicated on.
>
> However, there is the potential that some memory object that cached a
> result and is now running a second time could be the cause of a pass and so
> it is remotely possible that this is a false pass…. (And this I think is
> the crux of your argument - if any memory object is affected, theoretically
> you should rerun the whole transaction from a virgin state - which is
> effectively rerun the test from the beginning). So I guess we are
> discussing that we don’t have fully transactional memory executions?
>
> However I would argue that its way more common that you edit a method and
> fix a text and have 0 side effects than mucking around in memory or having
> something that rerunning locally messes up memory as well. So its much more
> useful to get the confirmation of green and move on. YES you could have a
> subtle error, and when you re-run it may later go red - but I would favour
> the 99% case as its a annoying if you are doing TDD.
>
> Tim
>
> On 10 Nov 2017, at 19:45, Richard Sargent <richard.sargent@
> gemtalksystems.com> wrote:
>
> I hear you and I understand your pain.
>
> However, if you corrected the problem, not by a code change, but by
> playing in the object's inspector or otherwise causing its internal state
> to change, and then resumed from the debugger, would you still claim the
> method was successful and should be coloured green?
>
> The only thing we can claim is that there were or were not further errors
> in the test. Colour it red if there were, but you cannot honestly colour it
> green. The code doesn't actually work.
>
>
> On Fri, Nov 10, 2017 at 11:29 AM, Tim Mackinnon <tim@testit.works> wrote:
>
>> My specific usecase is from a pragmatic TDD perspective - failing test,
>> in the debugger you fix the test and press proceed - expecting green.
>> Getting red, and then immediately running again to get red takes away from
>> our story of love coding and loving your debugger - and even Cassie me to
>> mistrust the tools.
>>
>> I get the idea that you can jiffy in the debugger and cause a false pass
>> - but I feel you are penalised for doing it right at the moment.
>>
>> Of course these tests will get run again, but I like the idea that if I
>> did it right, it should recognise this, not incur an extra click and moment
>> of doubt.
>>
>> A button to rerun the whole lot, or automatically rerun, or just run it
>> would work for me.
>>
>> Tim
>>
>> Sent from my iPhone
>>
>> On 10 Nov 2017, at 17:56, Richard Sargent <richard.sargent@gemtalksystem
>> s.com> wrote:
>>
>> That would be fine.
>> The point is that, without running the test in its entirety, from start
>> to finish, without interruption, error, or failure, one cannot claim
>> success.
>>
>> On Fri, Nov 10, 2017 at 9:34 AM, Sean P. DeNigris <s...@clipperadams.com>
>> wrote:
>>
>>> Richard Sargent wrote
>>> > The only reliable conclusion one can make from such an interrupted run
>>> is
>>> > whether it failed again. So, it would be possible to determine that the
>>> > test should be coloured red, but it is impossible to *reliably* claim
>>> that
>>> > the test should be coloured green.
>>>
>>> What if we ran the test again as if from the browser/runner before
>>> setting
>>> the icon?
>>>
>>>
>>>
>>> -
>>> Cheers,
>>> Sean
>>> --
>>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>>>
>>>
>>
>
>

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Tim Mackinnon]

Richard - I better understand what you are saying now. If you change the method 
and save it (hence restarting on the stack) I would expect it to turn green if 
I press resume. That is the TDD flow I expect, and one which sunit and coding 
in the debugger was predicated on.

However, there is the potential that some memory object that cached a result 
and is now running a second time could be the cause of a pass and so it is 
remotely possible that this is a false pass…. (And this I think is the crux of 
your argument - if any memory object is affected, theoretically you should 
rerun the whole transaction from a virgin state - which is effectively rerun 
the test from the beginning). So I guess we are discussing that we don’t have 
fully transactional memory executions?

However I would argue that its way more common that you edit a method and fix a 
text and have 0 side effects than mucking around in memory or having something 
that rerunning locally messes up memory as well. So its much more useful to get 
the confirmation of green and move on. YES you could have a subtle error, and 
when you re-run it may later go red - but I would favour the 99% case as its a 
annoying if you are doing TDD.

Tim

> On 10 Nov 2017, at 19:45, Richard Sargent 
>  wrote:
> 
> I hear you and I understand your pain.
> 
> However, if you corrected the problem, not by a code change, but by playing 
> in the object's inspector or otherwise causing its internal state to change, 
> and then resumed from the debugger, would you still claim the method was 
> successful and should be coloured green?
> 
> The only thing we can claim is that there were or were not further errors in 
> the test. Colour it red if there were, but you cannot honestly colour it 
> green. The code doesn't actually work.
> 
> 
> On Fri, Nov 10, 2017 at 11:29 AM, Tim Mackinnon  > wrote:
> My specific usecase is from a pragmatic TDD perspective - failing test, in 
> the debugger you fix the test and press proceed - expecting green. Getting 
> red, and then immediately running again to get red takes away from our story 
> of love coding and loving your debugger - and even Cassie me to mistrust the 
> tools.
> 
> I get the idea that you can jiffy in the debugger and cause a false pass - 
> but I feel you are penalised for doing it right at the moment.
> 
> Of course these tests will get run again, but I like the idea that if I did 
> it right, it should recognise this, not incur an extra click and moment of 
> doubt.
> 
> A button to rerun the whole lot, or automatically rerun, or just run it would 
> work for me.
> 
> Tim
> 
> Sent from my iPhone
> 
> On 10 Nov 2017, at 17:56, Richard Sargent  > wrote:
> 
>> That would be fine. 
>> The point is that, without running the test in its entirety, from start to 
>> finish, without interruption, error, or failure, one cannot claim success.
>> 
>> On Fri, Nov 10, 2017 at 9:34 AM, Sean P. DeNigris > > wrote:
>> Richard Sargent wrote
>> > The only reliable conclusion one can make from such an interrupted run is
>> > whether it failed again. So, it would be possible to determine that the
>> > test should be coloured red, but it is impossible to *reliably* claim that
>> > the test should be coloured green.
>> 
>> What if we ran the test again as if from the browser/runner before setting
>> the icon?
>> 
>> 
>> 
>> -
>> Cheers,
>> Sean
>> --
>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html 
>> 
>> 
>> 
> 

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Ben Coman]

The CI sometimes fails due to heisenbugs, which of course we should fix but
mostly someone just kicks the CI off again and the test passes and the code
is integrated.  So these heisenbugs slow down development when the CI
fails, and also we are not gaining any systematic statistics on where the
heisenbugs come from.

So for a while I've thought it would be useful for the CI to automatically
rerun a failed test to bypass the heisenbugs and at the same time log them
to a central location for anaylsis.  So I think such a feature could also
suit this situation.  After you correct the code and resume execution to
the end, the test is automatically rerun and turns green, or fails again
bringing you straight back to the debugger.  Actually that last itself
would be a useful feature for a tight cycle dev/testing a Test.

cheers -ben

On Sat, Nov 11, 2017 at 3:45 AM, Richard Sargent <
richard.sarg...@gemtalksystems.com> wrote:

> I hear you and I understand your pain.
>
> However, if you corrected the problem, not by a code change, but by
> playing in the object's inspector or otherwise causing its internal state
> to change, and then resumed from the debugger, would you still claim the
> method was successful and should be coloured green?
>
> The only thing we can claim is that there were or were not further errors
> in the test. Colour it red if there were, but you cannot honestly colour it
> green. The code doesn't actually work.
>
>
> On Fri, Nov 10, 2017 at 11:29 AM, Tim Mackinnon  wrote:
>
>> My specific usecase is from a pragmatic TDD perspective - failing test,
>> in the debugger you fix the test and press proceed - expecting green.
>> Getting red, and then immediately running again to get red takes away from
>> our story of love coding and loving your debugger - and even Cassie me to
>> mistrust the tools.
>>
>> I get the idea that you can jiffy in the debugger and cause a false pass
>> - but I feel you are penalised for doing it right at the moment.
>>
>> Of course these tests will get run again, but I like the idea that if I
>> did it right, it should recognise this, not incur an extra click and moment
>> of doubt.
>>
>> A button to rerun the whole lot, or automatically rerun, or just run it
>> would work for me.
>>
>> Tim
>>
>> Sent from my iPhone
>>
>> On 10 Nov 2017, at 17:56, Richard Sargent > s.com> wrote:
>>
>> That would be fine.
>> The point is that, without running the test in its entirety, from start
>> to finish, without interruption, error, or failure, one cannot claim
>> success.
>>
>> On Fri, Nov 10, 2017 at 9:34 AM, Sean P. DeNigris 
>> wrote:
>>
>>> Richard Sargent wrote
>>> > The only reliable conclusion one can make from such an interrupted run
>>> is
>>> > whether it failed again. So, it would be possible to determine that the
>>> > test should be coloured red, but it is impossible to *reliably* claim
>>> that
>>> > the test should be coloured green.
>>>
>>> What if we ran the test again as if from the browser/runner before
>>> setting
>>> the icon?
>>>
>>>
>>>
>>> -
>>> Cheers,
>>> Sean
>>> --
>>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>>>
>>>
>>
>

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Richard Sargent]

>
> So after a restart we would expect green
>
Yes.

Add having the test tool able to automate the tear-down and restart of the
test would be a good automation.


On Fri, Nov 10, 2017 at 11:41 AM, stephan  wrote:

> On 10-11-17 18:56, Richard Sargent wrote:
>
>> That would be fine.
>> The point is that, without running the test in its entirety, from start
>> to finish, without interruption, error, or failure, one cannot claim
>> success.
>>
>
> So after a restart we whould expect green
>
> Stephan
>
>
>

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Richard Sargent]

I hear you and I understand your pain.

However, if you corrected the problem, not by a code change, but by playing
in the object's inspector or otherwise causing its internal state to
change, and then resumed from the debugger, would you still claim the
method was successful and should be coloured green?

The only thing we can claim is that there were or were not further errors
in the test. Colour it red if there were, but you cannot honestly colour it
green. The code doesn't actually work.


On Fri, Nov 10, 2017 at 11:29 AM, Tim Mackinnon  wrote:

> My specific usecase is from a pragmatic TDD perspective - failing test, in
> the debugger you fix the test and press proceed - expecting green. Getting
> red, and then immediately running again to get red takes away from our
> story of love coding and loving your debugger - and even Cassie me to
> mistrust the tools.
>
> I get the idea that you can jiffy in the debugger and cause a false pass -
> but I feel you are penalised for doing it right at the moment.
>
> Of course these tests will get run again, but I like the idea that if I
> did it right, it should recognise this, not incur an extra click and moment
> of doubt.
>
> A button to rerun the whole lot, or automatically rerun, or just run it
> would work for me.
>
> Tim
>
> Sent from my iPhone
>
> On 10 Nov 2017, at 17:56, Richard Sargent  gemtalksystems.com> wrote:
>
> That would be fine.
> The point is that, without running the test in its entirety, from start to
> finish, without interruption, error, or failure, one cannot claim success.
>
> On Fri, Nov 10, 2017 at 9:34 AM, Sean P. DeNigris 
> wrote:
>
>> Richard Sargent wrote
>> > The only reliable conclusion one can make from such an interrupted run
>> is
>> > whether it failed again. So, it would be possible to determine that the
>> > test should be coloured red, but it is impossible to *reliably* claim
>> that
>> > the test should be coloured green.
>>
>> What if we ran the test again as if from the browser/runner before setting
>> the icon?
>>
>>
>>
>> -
>> Cheers,
>> Sean
>> --
>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>>
>>
>

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[stephan]

On 10-11-17 18:56, Richard Sargent wrote:

That would be fine.
The point is that, without running the test in its entirety, from start 
to finish, without interruption, error, or failure, one cannot claim 
success.


So after a restart we whould expect green

Stephan

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Tim Mackinnon]

Sorry "immediately running again to get red " should have been "to get green"

Tim

Sent from my iPhone

> On 10 Nov 2017, at 19:29, Tim Mackinnon  wrote:
> 
> immediately running again to get red

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Tim Mackinnon]

My specific usecase is from a pragmatic TDD perspective - failing test, in the 
debugger you fix the test and press proceed - expecting green. Getting red, and 
then immediately running again to get red takes away from our story of love 
coding and loving your debugger - and even Cassie me to mistrust the tools.

I get the idea that you can jiffy in the debugger and cause a false pass - but 
I feel you are penalised for doing it right at the moment.

Of course these tests will get run again, but I like the idea that if I did it 
right, it should recognise this, not incur an extra click and moment of doubt.

A button to rerun the whole lot, or automatically rerun, or just run it would 
work for me.

Tim

Sent from my iPhone

> On 10 Nov 2017, at 17:56, Richard Sargent 
>  wrote:
> 
> That would be fine. 
> The point is that, without running the test in its entirety, from start to 
> finish, without interruption, error, or failure, one cannot claim success.
> 
>> On Fri, Nov 10, 2017 at 9:34 AM, Sean P. DeNigris  
>> wrote:
>> Richard Sargent wrote
>> > The only reliable conclusion one can make from such an interrupted run is
>> > whether it failed again. So, it would be possible to determine that the
>> > test should be coloured red, but it is impossible to *reliably* claim that
>> > the test should be coloured green.
>> 
>> What if we ran the test again as if from the browser/runner before setting
>> the icon?
>> 
>> 
>> 
>> -
>> Cheers,
>> Sean
>> --
>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>> 
> 

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Kjell Godo]

On Fri, Nov 10, 2017 at 08:12 Richard Sargent <
richard.sarg...@gemtalksystems.com> wrote:

> I think it is correct because icon shows the last run state.
>>
> My claim is that is not a true statement.
>
>
> Then it is up to user to check how test is working in normal way.
>>
> This is my point. You must do this or you cannot know.
>
> The first statement should say "the last run state from the point of
> interruption". Any changes made in the code after the interruption
> invalidate your knowledge of the correctness of the code leading up to that
> point.
>
> The only reliable conclusion one can make from such an interrupted run is
> whether it failed again. So, it would be possible to determine that the
> test should be coloured red, but it is impossible to *reliably* claim that
> the test should be coloured green.
>

+1


>
> Many people in our industry, our profession have a disturbing tendency to
> be vague and practice self-deception. Take the concept of a "bug"as an
> example. They don't "fly into" our code; we *put* them in it. *We* put them
> in. That's why they should be called errors or defects.
>
> Likewise, claiming that a test that failed run successfully and
> uninterruptedly to completion "succeeded" is a fallacy. In general, the
> test runner cannot make that claim. It lacks the knowledge to do so.
>
> On Fri, Nov 10, 2017 at 1:21 AM, Denis Kudriashov 
> wrote:
>
>> Hi Richard
>>
>> 2017-11-09 23:51 GMT+01:00 Richard Sargent <
>> richard.sarg...@gemtalksystems.com>:
>>
>>> I think it is correct to show green if you fix your problem in the
 debugger and then proceed and  it passes all assertions. But it should
 equally fail if you proceed and it asserts false or craps out.

>>>
>>> It would be a lie. The truth is only that from that point on, it is
>>> correct. You have no guarantee that your correction hasn't broken something
>>> that would occur in the next run prior to the point at which you applied
>>> the correction.
>>>
>>
>> I think it is correct because icon shows the last run state. It is not
>> depends on the way how user manages to pass or fail the test.
>> So user can hack the test from the debugger and make it pass or fail
>> using inspector, scripting or whatever interaction with available objects.
>> When he closes the debugger (by proceed or window close) it is expected to
>> see the result of this hacking in the UI. So test icon should be updated
>> accordingly.
>> Then it is up to user to check how test is working in normal way.
>>
>>
>>>
>>> Hence my assertion that you cannot determine the colour until the next
>>> run.
>>>
>>
>>
>>
>>>
>>>
>>> On Thu, Nov 9, 2017 at 2:47 PM, Tim Mackinnon  wrote:
>>>
 Thanks Denis - I'll try my simple TDD test case in Pharo 7 (I haven't
 tried it yet - probably time to).

 I think it is correct to show green if you fix your problem in the
 debugger and then proceed and  it passes all assertions. But it should
 equally fail if you proceed and it asserts false or craps out.

 Strange it would vary in 6.1 - but I'll check.

 Tim

 Sent from my iPhone



 Sent from my iPhone
 On 9 Nov 2017, at 19:54, Denis Kudriashov  wrote:

 In Pharo 7 it works.

 2017-11-09 18:29 GMT+01:00 Tim Mackinnon :

> Thanks for looking at this - there is an issue however - when you
> apply that change (at least in a Pharo 6.1 image) - it shows green even
> when a test fails? So I think its turned one problem into the opposite 
> one.
>
> Unfortunately I haven’t got a chance to look a bit deeper to help -
> but it might be worth rolling back this change for now. We should fix it
> though - and the answer must be in the area you have identified.
>
> tim
>
> On 9 Nov 2017, at 12:43, Denis Kudriashov 
> wrote:
>
> And now it is in latest Pharo
>
> 2017-11-09 12:16 GMT+01:00 Denis Kudriashov :
>
>> Hi Tim.
>>
>> Fix is here
>> 20661-Fixing-test-from-debugger-should-mark-test-as-green-when-proceed
>>  .
>> Thank's for reporting. It forces me to fix. I always noticed it but
>> never take it seriously :)
>>
>> 2017-11-09 11:32 GMT+01:00 Tim Mackinnon :
>>
>>> Hi - I really like the build in test runner in the Pharo browsers,
>>> and I was preparing a talk to show how great TDD is in Pharo and how we
>>> aren’t ashamed of our debugger when testing (it augments the experience 
>>> in
>>> fact - letting you poke around and get your thoughts straight).
>>>
>>> However - if I pick rerun in the test runner debugger - and step
>>> through a test and then correct the failing code, and hit resume - the
>>> browser always shows a red failure, even 

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Richard Sargent]

That would be fine.
The point is that, without running the test in its entirety, from start to
finish, without interruption, error, or failure, one cannot claim success.

On Fri, Nov 10, 2017 at 9:34 AM, Sean P. DeNigris 
wrote:

> Richard Sargent wrote
> > The only reliable conclusion one can make from such an interrupted run is
> > whether it failed again. So, it would be possible to determine that the
> > test should be coloured red, but it is impossible to *reliably* claim
> that
> > the test should be coloured green.
>
> What if we ran the test again as if from the browser/runner before setting
> the icon?
>
>
>
> -
> Cheers,
> Sean
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>
>

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Sean P. DeNigris]

Richard Sargent wrote
> The only reliable conclusion one can make from such an interrupted run is
> whether it failed again. So, it would be possible to determine that the
> test should be coloured red, but it is impossible to *reliably* claim that
> the test should be coloured green.

What if we ran the test again as if from the browser/runner before setting
the icon?



-
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Richard Sargent]

>
> I think it is correct because icon shows the last run state.
>
My claim is that is not a true statement.

Then it is up to user to check how test is working in normal way.
>
This is my point. You must do this or you cannot know.

The first statement should say "the last run state from the point of
interruption". Any changes made in the code after the interruption
invalidate your knowledge of the correctness of the code leading up to that
point.

The only reliable conclusion one can make from such an interrupted run is
whether it failed again. So, it would be possible to determine that the
test should be coloured red, but it is impossible to *reliably* claim that
the test should be coloured green.


Many people in our industry, our profession have a disturbing tendency to
be vague and practice self-deception. Take the concept of a "bug"as an
example. They don't "fly into" our code; we *put* them in it. *We* put them
in. That's why they should be called errors or defects.

Likewise, claiming that a test that failed run successfully and
uninterruptedly to completion "succeeded" is a fallacy. In general, the
test runner cannot make that claim. It lacks the knowledge to do so.

On Fri, Nov 10, 2017 at 1:21 AM, Denis Kudriashov 
wrote:

> Hi Richard
>
> 2017-11-09 23:51 GMT+01:00 Richard Sargent  gemtalksystems.com>:
>
>> I think it is correct to show green if you fix your problem in the
>>> debugger and then proceed and  it passes all assertions. But it should
>>> equally fail if you proceed and it asserts false or craps out.
>>>
>>
>> It would be a lie. The truth is only that from that point on, it is
>> correct. You have no guarantee that your correction hasn't broken something
>> that would occur in the next run prior to the point at which you applied
>> the correction.
>>
>
> I think it is correct because icon shows the last run state. It is not
> depends on the way how user manages to pass or fail the test.
> So user can hack the test from the debugger and make it pass or fail using
> inspector, scripting or whatever interaction with available objects. When
> he closes the debugger (by proceed or window close) it is expected to see
> the result of this hacking in the UI. So test icon should be updated
> accordingly.
> Then it is up to user to check how test is working in normal way.
>
>
>>
>> Hence my assertion that you cannot determine the colour until the next
>> run.
>>
>
>
>
>>
>>
>> On Thu, Nov 9, 2017 at 2:47 PM, Tim Mackinnon  wrote:
>>
>>> Thanks Denis - I'll try my simple TDD test case in Pharo 7 (I haven't
>>> tried it yet - probably time to).
>>>
>>> I think it is correct to show green if you fix your problem in the
>>> debugger and then proceed and  it passes all assertions. But it should
>>> equally fail if you proceed and it asserts false or craps out.
>>>
>>> Strange it would vary in 6.1 - but I'll check.
>>>
>>> Tim
>>>
>>> Sent from my iPhone
>>>
>>>
>>>
>>> Sent from my iPhone
>>> On 9 Nov 2017, at 19:54, Denis Kudriashov  wrote:
>>>
>>> In Pharo 7 it works.
>>>
>>> 2017-11-09 18:29 GMT+01:00 Tim Mackinnon :
>>>
 Thanks for looking at this - there is an issue however - when you apply
 that change (at least in a Pharo 6.1 image) - it shows green even when a
 test fails? So I think its turned one problem into the opposite one.

 Unfortunately I haven’t got a chance to look a bit deeper to help - but
 it might be worth rolling back this change for now. We should fix it though
 - and the answer must be in the area you have identified.

 tim

 On 9 Nov 2017, at 12:43, Denis Kudriashov  wrote:

 And now it is in latest Pharo

 2017-11-09 12:16 GMT+01:00 Denis Kudriashov :

> Hi Tim.
>
> Fix is here 20661-Fixing-test-from-debugger-should-mark-test-as-gre
> en-when-proceed  .
> Thank's for reporting. It forces me to fix. I always noticed it but
> never take it seriously :)
>
> 2017-11-09 11:32 GMT+01:00 Tim Mackinnon :
>
>> Hi - I really like the build in test runner in the Pharo browsers,
>> and I was preparing a talk to show how great TDD is in Pharo and how we
>> aren’t ashamed of our debugger when testing (it augments the experience 
>> in
>> fact - letting you poke around and get your thoughts straight).
>>
>> However - if I pick rerun in the test runner debugger - and step
>> through a test and then correct the failing code, and hit resume - the
>> browser always shows a red failure, even though the execution is now
>> correct. I have to run the test again.
>>
>> This doesn’t seem right to me - are we missing a success event or
>> something?
>>
>>
>> Tim
>>
>
>


>>>
>>
>

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Denis Kudriashov]

Hi Richard

2017-11-09 23:51 GMT+01:00 Richard Sargent <
richard.sarg...@gemtalksystems.com>:

> I think it is correct to show green if you fix your problem in the
>> debugger and then proceed and  it passes all assertions. But it should
>> equally fail if you proceed and it asserts false or craps out.
>>
>
> It would be a lie. The truth is only that from that point on, it is
> correct. You have no guarantee that your correction hasn't broken something
> that would occur in the next run prior to the point at which you applied
> the correction.
>

I think it is correct because icon shows the last run state. It is not
depends on the way how user manages to pass or fail the test.
So user can hack the test from the debugger and make it pass or fail using
inspector, scripting or whatever interaction with available objects. When
he closes the debugger (by proceed or window close) it is expected to see
the result of this hacking in the UI. So test icon should be updated
accordingly.
Then it is up to user to check how test is working in normal way.


>
> Hence my assertion that you cannot determine the colour until the next run.
>



>
>
> On Thu, Nov 9, 2017 at 2:47 PM, Tim Mackinnon  wrote:
>
>> Thanks Denis - I'll try my simple TDD test case in Pharo 7 (I haven't
>> tried it yet - probably time to).
>>
>> I think it is correct to show green if you fix your problem in the
>> debugger and then proceed and  it passes all assertions. But it should
>> equally fail if you proceed and it asserts false or craps out.
>>
>> Strange it would vary in 6.1 - but I'll check.
>>
>> Tim
>>
>> Sent from my iPhone
>>
>>
>>
>> Sent from my iPhone
>> On 9 Nov 2017, at 19:54, Denis Kudriashov  wrote:
>>
>> In Pharo 7 it works.
>>
>> 2017-11-09 18:29 GMT+01:00 Tim Mackinnon :
>>
>>> Thanks for looking at this - there is an issue however - when you apply
>>> that change (at least in a Pharo 6.1 image) - it shows green even when a
>>> test fails? So I think its turned one problem into the opposite one.
>>>
>>> Unfortunately I haven’t got a chance to look a bit deeper to help - but
>>> it might be worth rolling back this change for now. We should fix it though
>>> - and the answer must be in the area you have identified.
>>>
>>> tim
>>>
>>> On 9 Nov 2017, at 12:43, Denis Kudriashov  wrote:
>>>
>>> And now it is in latest Pharo
>>>
>>> 2017-11-09 12:16 GMT+01:00 Denis Kudriashov :
>>>
 Hi Tim.

 Fix is here 20661-Fixing-test-from-debugger-should-mark-test-as-gre
 en-when-proceed  .
 Thank's for reporting. It forces me to fix. I always noticed it but
 never take it seriously :)

 2017-11-09 11:32 GMT+01:00 Tim Mackinnon :

> Hi - I really like the build in test runner in the Pharo browsers, and
> I was preparing a talk to show how great TDD is in Pharo and how we aren’t
> ashamed of our debugger when testing (it augments the experience in fact -
> letting you poke around and get your thoughts straight).
>
> However - if I pick rerun in the test runner debugger - and step
> through a test and then correct the failing code, and hit resume - the
> browser always shows a red failure, even though the execution is now
> correct. I have to run the test again.
>
> This doesn’t seem right to me - are we missing a success event or
> something?
>
>
> Tim
>


>>>
>>>
>>
>

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Denis Kudriashov]

And give me concrete steps to see that it is not working (if you will find
it)

2017-11-09 23:47 GMT+01:00 Tim Mackinnon :

> Thanks Denis - I'll try my simple TDD test case in Pharo 7 (I haven't
> tried it yet - probably time to).
>
> I think it is correct to show green if you fix your problem in the
> debugger and then proceed and  it passes all assertions. But it should
> equally fail if you proceed and it asserts false or craps out.
>
> Strange it would vary in 6.1 - but I'll check.
>
> Tim
>
> Sent from my iPhone
>
>
>
> Sent from my iPhone
> On 9 Nov 2017, at 19:54, Denis Kudriashov  wrote:
>
> In Pharo 7 it works.
>
> 2017-11-09 18:29 GMT+01:00 Tim Mackinnon :
>
>> Thanks for looking at this - there is an issue however - when you apply
>> that change (at least in a Pharo 6.1 image) - it shows green even when a
>> test fails? So I think its turned one problem into the opposite one.
>>
>> Unfortunately I haven’t got a chance to look a bit deeper to help - but
>> it might be worth rolling back this change for now. We should fix it though
>> - and the answer must be in the area you have identified.
>>
>> tim
>>
>> On 9 Nov 2017, at 12:43, Denis Kudriashov  wrote:
>>
>> And now it is in latest Pharo
>>
>> 2017-11-09 12:16 GMT+01:00 Denis Kudriashov :
>>
>>> Hi Tim.
>>>
>>> Fix is here 20661-Fixing-test-from-debugger-should-mark-test-as-gre
>>> en-when-proceed  .
>>> Thank's for reporting. It forces me to fix. I always noticed it but
>>> never take it seriously :)
>>>
>>> 2017-11-09 11:32 GMT+01:00 Tim Mackinnon :
>>>
 Hi - I really like the build in test runner in the Pharo browsers, and
 I was preparing a talk to show how great TDD is in Pharo and how we aren’t
 ashamed of our debugger when testing (it augments the experience in fact -
 letting you poke around and get your thoughts straight).

 However - if I pick rerun in the test runner debugger - and step
 through a test and then correct the failing code, and hit resume - the
 browser always shows a red failure, even though the execution is now
 correct. I have to run the test again.

 This doesn’t seem right to me - are we missing a success event or
 something?


 Tim

>>>
>>>
>>
>>
>

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Richard Sargent]

>
> I think it is correct to show green if you fix your problem in the
> debugger and then proceed and  it passes all assertions. But it should
> equally fail if you proceed and it asserts false or craps out.
>

It would be a lie. The truth is only that from that point on, it is
correct. You have no guarantee that your correction hasn't broken something
that would occur in the next run prior to the point at which you applied
the correction.

Hence my assertion that you cannot determine the colour until the next run.


On Thu, Nov 9, 2017 at 2:47 PM, Tim Mackinnon  wrote:

> Thanks Denis - I'll try my simple TDD test case in Pharo 7 (I haven't
> tried it yet - probably time to).
>
> I think it is correct to show green if you fix your problem in the
> debugger and then proceed and  it passes all assertions. But it should
> equally fail if you proceed and it asserts false or craps out.
>
> Strange it would vary in 6.1 - but I'll check.
>
> Tim
>
> Sent from my iPhone
>
>
>
> Sent from my iPhone
> On 9 Nov 2017, at 19:54, Denis Kudriashov  wrote:
>
> In Pharo 7 it works.
>
> 2017-11-09 18:29 GMT+01:00 Tim Mackinnon :
>
>> Thanks for looking at this - there is an issue however - when you apply
>> that change (at least in a Pharo 6.1 image) - it shows green even when a
>> test fails? So I think its turned one problem into the opposite one.
>>
>> Unfortunately I haven’t got a chance to look a bit deeper to help - but
>> it might be worth rolling back this change for now. We should fix it though
>> - and the answer must be in the area you have identified.
>>
>> tim
>>
>> On 9 Nov 2017, at 12:43, Denis Kudriashov  wrote:
>>
>> And now it is in latest Pharo
>>
>> 2017-11-09 12:16 GMT+01:00 Denis Kudriashov :
>>
>>> Hi Tim.
>>>
>>> Fix is here 20661-Fixing-test-from-debugger-should-mark-test-as-gre
>>> en-when-proceed  .
>>> Thank's for reporting. It forces me to fix. I always noticed it but
>>> never take it seriously :)
>>>
>>> 2017-11-09 11:32 GMT+01:00 Tim Mackinnon :
>>>
 Hi - I really like the build in test runner in the Pharo browsers, and
 I was preparing a talk to show how great TDD is in Pharo and how we aren’t
 ashamed of our debugger when testing (it augments the experience in fact -
 letting you poke around and get your thoughts straight).

 However - if I pick rerun in the test runner debugger - and step
 through a test and then correct the failing code, and hit resume - the
 browser always shows a red failure, even though the execution is now
 correct. I have to run the test again.

 This doesn’t seem right to me - are we missing a success event or
 something?


 Tim

>>>
>>>
>>
>>
>

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Tim Mackinnon]

Thanks Denis - I'll try my simple TDD test case in Pharo 7 (I haven't tried it 
yet - probably time to).

I think it is correct to show green if you fix your problem in the debugger and 
then proceed and  it passes all assertions. But it should equally fail if you 
proceed and it asserts false or craps out.

Strange it would vary in 6.1 - but I'll check.

Tim

Sent from my iPhone



Sent from my iPhone
> On 9 Nov 2017, at 19:54, Denis Kudriashov  wrote:
> 
> In Pharo 7 it works.
> 
> 2017-11-09 18:29 GMT+01:00 Tim Mackinnon :
>> Thanks for looking at this - there is an issue however - when you apply that 
>> change (at least in a Pharo 6.1 image) - it shows green even when a test 
>> fails? So I think its turned one problem into the opposite one.
>> 
>> Unfortunately I haven’t got a chance to look a bit deeper to help - but it 
>> might be worth rolling back this change for now. We should fix it though - 
>> and the answer must be in the area you have identified.
>> 
>> tim
>> 
>>> On 9 Nov 2017, at 12:43, Denis Kudriashov  wrote:
>>> 
>>> And now it is in latest Pharo
>>> 
>>> 2017-11-09 12:16 GMT+01:00 Denis Kudriashov :
 Hi Tim.
 
 Fix is here 
 20661-Fixing-test-from-debugger-should-mark-test-as-green-when-proceed . 
 Thank's for reporting. It forces me to fix. I always noticed it but never 
 take it seriously :)
 
 2017-11-09 11:32 GMT+01:00 Tim Mackinnon :
> Hi - I really like the build in test runner in the Pharo browsers, and I 
> was preparing a talk to show how great TDD is in Pharo and how we aren’t 
> ashamed of our debugger when testing (it augments the experience in fact 
> - letting you poke around and get your thoughts straight).
> 
> However - if I pick rerun in the test runner debugger - and step through 
> a test and then correct the failing code, and hit resume - the browser 
> always shows a red failure, even though the execution is now correct. I 
> have to run the test again.
> 
> This doesn’t seem right to me - are we missing a success event or 
> something?
> 
> 
> Tim
> 

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Denis Kudriashov]

In Pharo 7 it works.

2017-11-09 18:29 GMT+01:00 Tim Mackinnon :

> Thanks for looking at this - there is an issue however - when you apply
> that change (at least in a Pharo 6.1 image) - it shows green even when a
> test fails? So I think its turned one problem into the opposite one.
>
> Unfortunately I haven’t got a chance to look a bit deeper to help - but it
> might be worth rolling back this change for now. We should fix it though -
> and the answer must be in the area you have identified.
>
> tim
>
> On 9 Nov 2017, at 12:43, Denis Kudriashov  wrote:
>
> And now it is in latest Pharo
>
> 2017-11-09 12:16 GMT+01:00 Denis Kudriashov :
>
>> Hi Tim.
>>
>> Fix is here 20661-Fixing-test-from-debugger-should-mark-test-as-gre
>> en-when-proceed  .
>> Thank's for reporting. It forces me to fix. I always noticed it but never
>> take it seriously :)
>>
>> 2017-11-09 11:32 GMT+01:00 Tim Mackinnon :
>>
>>> Hi - I really like the build in test runner in the Pharo browsers, and I
>>> was preparing a talk to show how great TDD is in Pharo and how we aren’t
>>> ashamed of our debugger when testing (it augments the experience in fact -
>>> letting you poke around and get your thoughts straight).
>>>
>>> However - if I pick rerun in the test runner debugger - and step through
>>> a test and then correct the failing code, and hit resume - the browser
>>> always shows a red failure, even though the execution is now correct. I
>>> have to run the test again.
>>>
>>> This doesn’t seem right to me - are we missing a success event or
>>> something?
>>>
>>>
>>> Tim
>>>
>>
>>
>
>

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Richard Sargent]

I saw this earlier and thought it was wrong to make that change. You cannot
assign a colour to the test if it does not run all the way through without
interruption. You cannot know whether it truly succeeded or not until you
finally run it all the way through without error or interruption.

Leave it uncoloured until you (can) know for certain.


On Thu, Nov 9, 2017 at 9:29 AM, Tim Mackinnon  wrote:

> Thanks for looking at this - there is an issue however - when you apply
> that change (at least in a Pharo 6.1 image) - it shows green even when a
> test fails? So I think its turned one problem into the opposite one.
>
> Unfortunately I haven’t got a chance to look a bit deeper to help - but it
> might be worth rolling back this change for now. We should fix it though -
> and the answer must be in the area you have identified.
>
> tim
>
> On 9 Nov 2017, at 12:43, Denis Kudriashov  wrote:
>
> And now it is in latest Pharo
>
> 2017-11-09 12:16 GMT+01:00 Denis Kudriashov :
>
>> Hi Tim.
>>
>> Fix is here 20661-Fixing-test-from-debugger-should-mark-test-as-gre
>> en-when-proceed  .
>> Thank's for reporting. It forces me to fix. I always noticed it but never
>> take it seriously :)
>>
>> 2017-11-09 11:32 GMT+01:00 Tim Mackinnon :
>>
>>> Hi - I really like the build in test runner in the Pharo browsers, and I
>>> was preparing a talk to show how great TDD is in Pharo and how we aren’t
>>> ashamed of our debugger when testing (it augments the experience in fact -
>>> letting you poke around and get your thoughts straight).
>>>
>>> However - if I pick rerun in the test runner debugger - and step through
>>> a test and then correct the failing code, and hit resume - the browser
>>> always shows a red failure, even though the execution is now correct. I
>>> have to run the test again.
>>>
>>> This doesn’t seem right to me - are we missing a success event or
>>> something?
>>>
>>>
>>> Tim
>>>
>>
>>
>
>

Re: [Pharo-users] Why does the test runner show red when I correct a test?

[Tim Mackinnon]

Thanks for looking at this - there is an issue however - when you apply that 
change (at least in a Pharo 6.1 image) - it shows green even when a test fails? 
So I think its turned one problem into the opposite one.

Unfortunately I haven’t got a chance to look a bit deeper to help - but it 
might be worth rolling back this change for now. We should fix it though - and 
the answer must be in the area you have identified.

tim

> On 9 Nov 2017, at 12:43, Denis Kudriashov  wrote:
> 
> And now it is in latest Pharo
> 
> 2017-11-09 12:16 GMT+01:00 Denis Kudriashov  >:
> Hi Tim.
> 
> Fix is here 
> 20661-Fixing-test-from-debugger-should-mark-test-as-green-when-proceed 
>  . 
> Thank's for reporting. It forces me to fix. I always noticed it but never 
> take it seriously :)
> 
> 2017-11-09 11:32 GMT+01:00 Tim Mackinnon  >:
> Hi - I really like the build in test runner in the Pharo browsers, and I was 
> preparing a talk to show how great TDD is in Pharo and how we aren’t ashamed 
> of our debugger when testing (it augments the experience in fact - letting 
> you poke around and get your thoughts straight).
> 
> However - if I pick rerun in the test runner debugger - and step through a 
> test and then correct the failing code, and hit resume - the browser always 
> shows a red failure, even though the execution is now correct. I have to run 
> the test again.
> 
> This doesn’t seem right to me - are we missing a success event or something?
> 
> 
> Tim
> 
>