On Sun, Oct 07, 2012 at 09:09:07PM +0300, Roman Cheplyaka wrote:
> * Simon Hengel <s...@typeful.net> [2012-10-07 15:45:21+0200]
> > On Fri, Oct 05, 2012 at 05:17:18PM +0300, Roman Cheplyaka wrote:
> > > I can do that indeed, and I guess I could reimplement everything I have
> > > at the moment on top of HUnit.
> > >
> > > However, an important part of functionality isn't there at the moment —
> > > golden file management. You should be able to say, "for this test,
> > > take its current output and write it to the corresponding golden file".
> > >
> > > In order to do that, you need to have access to the list of golden tests
> > > in the suite. This is where implementation details of different test
> > > frameworks start to matter. Probably one can make an abstraction over
> > > test frameworks that would give the list of all golden tests.
> > > (Although when you start abstracting over test frameworks, which are
> > > abstractions themselves, it becomes somewhat funny.)
> >
> > Ok, makes sense.
> >
> > I'm looking forward to give it a try, and see how it compares to using
> > operating system primitives (say `cp') for "golden file management".
>
> 1. You often want to update not just one test, but all, or some of the
> tests (when you've made a change and verified that the changes in
> output are expected). Doing it in command line is certainly possible,
> but not trivial nor convenient.
> 2. For some tests (like goldenVsString) the output is not captured in a
> file, so using cp directly is not possible.
Yes and yes. I have no fixed expectations nor any idea how an "ideal"
interface would look like, but I guess there is room for improvement.
So I'm really looking forward to try it ;)
> > > Speaking of such functionality, correct me if I'm wrong, but neither
> > > HUnit nor hspec won't be able to support it anyway, because they
> > > represent tests as opaque IO actions.
> >
> > It would be easy to extend Hspec to support this in the same way you
> > extend test-framework to support this. It requires existentials; the
> > only substantial difference that I can see is that test-framework
> > already uses existentials, while Hspec does not.
>
> Well, if you are willing to make this change, then I'll try to do my
> part of the job and expose a useful abstraction.
If it gives us something that is useful from a users perspective, I'm
happy to make that change. I would hope that something like [1] works,
e.g.:
instance Example GoldenTest where
evaluateExample c = evaluateExample c . goldenTestToHUnitAssetion
exampleMetadata = Just . Metadata
That way the Typeable instance is optional.
Personally, I still think that it may be a good idea to first explore
the design space with test-framework before trying to abstract over it.
Cheers,
Simon
[1] https://github.com/sol/hspec/commit/6927f642aea44803b57c2b77548931f6865b0c38
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
