Re: [Tutor] writing effective unittests
On 3 January 2013 14:46, Luke Thomas Mergner wrote: > * Albert-Jan Roskam wrote: > >> >> >> > Hi, >> >> > >> > I am trying to learn a bit of test-driven programming using unittests and >> > nosetests. I am having trouble finding resources that explain how to write >> > effective tests. I am not a programmer or a student, so I do not have >> > access to > To partially answer my own question, let me tell you what I've learned in the > last 48 hours. It is easy to learn that unittest is the standard testing > module in Python > 2.6, that it is backported, that it is being improved in > Python 3 with new tools. There are a few PyCon videos that discuss this > evolution. It is also easy to learn that unittest is based off of prior > library for Java (JUnit) and SmallTalk. It is fairly easy to find an > introduction to the syntax of writing a test: > And there are a few videos available that walk through how to put this > together into a test suite. (I've included what I found below, so that this > email will have some value to others.) > > What am I missing? The biggest problem is that no one is explaining the > rationale behind testing. The trivial examples compare integers: 2 == 2. At > first glance this seems pointless. I had assumed that tests would attempt to > confuse my functions and teach me how to write more robust code. But I > *think* now that tests are a way to determine if new code has changed old > behaviors. Testing 2 == 2 is trivial, but if the function starts returning 3 > in a few months, it would be helpful to know right away. In general, though, > I'm finding it difficult to conceptualize what my tests should be doing, > testing, and telling me about my code. Hi Luke, First, I should like to commend your post; collecting together the results of your research and posting it is a valuable thing to do. While I agree with Steven that you'd be better off not taking the list as a digest, for me at least, the mild irritation that digest use imposes is small compared to the goodwill the effort shown produces. Second, I don't test nearly as much as I ought. So, read with a grain of salt :-) Now, for the meat. A key idea that was implicit in Steven's first reply might be better made explicit. When testing, you want to look for odd or edge cases where things might break and embed tests that will let you know if they do. What counts as an edge or odd case is obviously context dependent. But, if dealing with an arithmetical function of one input, you'd generally want tests for at least 0, 1, -1, some large positive, some large negative, and small and large positive and negative non-integer rational numbers. If writing against 2.x, you'd also want an integer large enough to be a long rather than an int, etc. If testing a function that takes one string, you'd want tests for at least the empty string, a string consisting solely of whitespace, a single character string, a string with "odd whitespacing that you wouldn ' t expect", strings in mixed case, ones with punctuation, etc. Another principle that I use to generate tests: while writing code, every time I find myself considering some possibility and thinking "But, that's impossible!" I try to put in a test to ensure that the impossible actually is. http://www.youtube.com/watch%3Fv%3DD58LpHBnvsI I didn't try to find the context, but testing 2 == 2 does seem a bit pointless. However, a similar sort of test might be worthwhile. Consider: >>> class A(object): def __eq__(self, other): return False >>> a=A() >>> a==a False >>> Now, one might think: "But that's such a stupid bug. I'd never do that". If so tempted, go watch the link above :-) In seriousness, the way that sort of test will help isn't so much to confirm that the code works as intended when written, but that it continues to work several moths later when you've forgotten most of it and need to tweak just one thing that couldn't possibly affect other parts of the code. HTH, Brian vdB ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] IronPython any tutors with experience out there?
On 03/01/13 18:37, Bjorn Madsen wrote:
However ... to interact with the c# modules which my colleagues wrote, I
need to add "clr" references.
Example:
import clr
clr.AddReferenceToFile("Mapack.dll")
from Mapack import *
dir()
Unfortunately I can't get it to run
So does anything work? Is the import OK?
Are there any error messages?
Can you call any of the other CLR functions?
Can you get a basic 'hello world' to run under .NET?
I've never used IronPython but I have played around
with C# and VB.Net on an earlier .NET framework (V2?)
There should be a bunch of functions you can access
via CLR, check that they work first to prove CLR
integration is OK.
For example can you use AddReference() to access
a standard .NET assembly rather than AddReferenceToFile()
which is I assume for DLLs.
Finally there is an active gmane IronPython news group
which you could try for more focussed help...
gmane.comp.python.ironpython.user
It is probably sourced from the IronPython mailing
list of you prefer that...
http://mail.python.org/mailman/listinfo/ironpython-users
Finally I did find this reference:
http://stackoverflow.com/questions/3692539/clr-addreferencetofile-fails-in-ironpython-2-7
Which notes that you can only import DLLS compiled for .NET v4 in
IronPython 2.7. Might that be the problem?
HTH
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] writing effective unittests
Hi Luke, My responses inline below. On 04/01/13 06:46, Luke Thomas Mergner wrote: I am on the digest version of the list, so I haven't gotten a copy of any replies. All the more reason not to be on the digest version. But you should have received replies *in the digest*. (By the way, thank you for editing the subject line to something meaningful.) What am I missing? The biggest problem is that no one is explaining the rationale behind testing. You test software so that you know it does what you expect. Until you actually try it, how do you know it will work as expected? I'm reminded of a famous quote from Donald Knuth, which if memory serves goes something like this: "Beware of bugs in the above code, I have only proven it is correct, not tested it." All testing is aimed at making sure the software does what you want it to do and does not contain bugs. The trivial examples compare integers: 2 == 2. At first glance this seems pointless. As given, that is pointless, since there is no possible way for 2 to fail to be 2. A less pointless example would be: x = some_calculation(a, b, c) # expected to return 2 assertEqual(x, 2) # x might not be 2, so we test that it is I had assumed that tests would attempt to confuse my functions and teach me how to write more robust code. That is part of it. A solid test suite should cover unexpected input as well as expected input, to ensure that you code does the right thing when faced with bad input. In my earlier post, which I hope you received, I gave an toy example of testing a function that takes at most one integer argument and returns a string. Some of the tests I wrote test that the function *correctly (and safely) fails* with TypeError if you pass more than one argument, or if the argument is not an int. But I *think* now that tests are a way to determine if new code has changed old behaviors. Testing 2 == 2 is trivial, but if the function starts returning 3 in a few months, it would be helpful to know right away. Yes, exactly! We can write different sorts of tests: * Doc tests are (mostly) intended to act as documentation. The problem with documentation is that it can get out of date as the software changes. How do you know when the documentation is out of date? Because the doc tests fail! * Regression tests are intended to warn you if a bug has come back. Every time you fix a reported bug, you should have a regression test for that bug, so that if any future change accidentally *regresses* the code and reintroduces that bug, you will find out immediately. * Unit tests are intended for testing that code behaves as expected, that parts of the public interface that you expect exist actually does exist. There's no point documenting that you have a class Spam if the class doesn't actually exist, so you have a unit test that confirms that Spam exists, and that it behaves as you expect. (Note that the concept of "unit tests" is more general than the things you write with the unittest module. The unittest module is a framework for writing unit tests, but there are other frameworks, like nose.) * Blackbox testing is when you treat functions and methods as black boxes. Since you can't see inside them, you can only test things which are public knowledge. E.g. "if I pass this input to the function, I should get this output". * Whitebox testing is the opposite: you test based on your knowledge of the function's internal details. E.g. "if I pass this input to the function, it will run the code path A->B->C; but if I pass this other input, it will run the code path A->D->E->C. Therefore I need to test both inputs." The problem with whitebox testing is that if you change the function's implementation, your tests may become obsolete. But the advantage is that the tests cover more of the possible things that could happen. * Fuzz testing is when you check how robust your code is by actually making *random changes to the data*, then seeing how your code responds or fails. One early example was "The Monkey" on Apple Macs in 1983, which fed random mouse clicks and key presses to discover bugs in MacPaint. There are many other forms of testing that are possible: https://en.wikipedia.org/wiki/Software_testing But in general, any testing is better than no testing. My further responses to follow later in the day. -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] IronPython any tutors with experience out there?
Bjorn Madsen wrote:
>
> Hello PythonTutor
> - I'm a scientist and very happy with python 2.7. I have been asked to assist
> a development program where
> everything is written in dotNET/ C# (visual studio 2012) and luckily
> Microsoft Visual Studio supports IronPython
> which is a clean Python implementation in C#, so I can use the Python syntax
> I'm so familiar with...
>
> However ... to interact with the c# modules which my colleagues wrote, I need
> to add "clr" references.
>
> Example:
> >>> import clr
> >>> clr.AddReferenceToFile("Mapack.dll")
> >>> from Mapack import *
> >>> dir()
>
> Unfortunately I can't get it to run and the tutorials after 12 hours of
> google-reading are terribly sparse. Is
> there any experience somewhere in the network with IronPython in VSTO-2012 in
> this forum and what could the top
> five clr.addreference bugs be?
If nobody responds satisfactorily on this list, you may have better
luck with the main Python list. Your question is not really in the
realm of "teaching python" which is this list's main focus.
~Ramit
This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] writing effective unittests
On 03/01/13 19:46, Luke Thomas Mergner wrote: What am I missing? The biggest problem is that no one is explaining the rationale behind testing. See my other reply and check out the references to programming by contract, pre/post conditions and invariants. Those are all good things to test. Invariants are probably hardest but also most powerful IMHO. But viesw your code as a contract with the user. What can the user put in, what should they get out? What could break it? I think what I need is a conceptual shift: how do python programmers use unittests? Personally I only use TDD and formal unit tests when I'm writing something a) big and/or complex b) reusable - I'll be coming back to it in the future For small, one-off tools and scripts the overhead is usually not worth while. And they are never 100% reliable, bugs will still sneak through. So for one-offs I just do informal testing. In most cases with TDD I stick to black box testing but if its a reuseable class or a framework then I'll probably drop down to white box and if it's mission or safety critical I'll do glass box testing (ie 100% code coverage) - That's to say in about 3 projects in 30 years... My next step is to look into buying some real books. I'd only go there if you are really comfortable with programming in Python. Learn and use the principles but don't sweat over it until you really need to. It's more important to be comfortable writing good Python code than writing good tests for bad code! HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] python as poetry
I don't know if it's appropriate to post things like this on the list, but I've learned a lot from this group and thought I'd share something I think you all will enjoy: http://www.thehelloworldprogram.com/videos/poetry-corner-red-wheelbarrow/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] writing effective unittests
I think what I need is a conceptual shift: how do python programmers use > unittests? > > Here at the Library, we use unit test to test the cases that have many known inputs. For example, some of the data could come in "foo bar baz", and some others could come in as "foo, bar, baz", and others could come in "foo; bar; baz". As we code for each one of these cases, we write tests to make sure that changes that we make don't break any of our former code. I think that TDD is a good thing to strive for, but in most cases is an ideal that can't be reached in most cases. Hope that helps, Tino ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] writing effective unittests
* Albert-Jan Roskam wrote:
>
>
> > Hi,
>
> >
> > I am trying to learn a bit of test-driven programming using unittests and
> > nosetests. I am having trouble finding resources that explain how to write
> > effective tests. I am not a programmer or a student, so I do not have
> > access to
> > collegues or classes. I'm hoping that someone knows of a good tutorial that
> > I've missed. Based on the O'Reilly 2-Part introduction, I hope that
> > learning to write tests will make my code better.
> >
I am on the digest version of the list, so I haven't gotten a copy of any
replies. Apologies if two copies of my last email hit the list, I'm still
learning to use mutt.
To partially answer my own question, let me tell you what I've learned in the
last 48 hours. It is easy to learn that unittest is the standard testing module
in Python > 2.6, that it is backported, that it is being improved in Python 3
with new tools. There are a few PyCon videos that discuss this evolution. It is
also easy to learn that unittest is based off of prior library for Java (JUnit)
and SmallTalk. It is fairly easy to find an introduction to the syntax of
writing a test:
>class blargTest(unittest.TestCase)
> def testFeature(self):
> ... do some stuff...
> assertTrue('blarg', blarg)
And there are a few videos available that walk through how to put this together
into a test suite. (I've included what I found below, so that this email will
have some value to others.)
What am I missing? The biggest problem is that no one is explaining the
rationale behind testing. The trivial examples compare integers: 2 == 2. At
first glance this seems pointless. I had assumed that tests would attempt to
confuse my functions and teach me how to write more robust code. But I *think*
now that tests are a way to determine if new code has changed old behaviors.
Testing 2 == 2 is trivial, but if the function starts returning 3 in a few
months, it would be helpful to know right away. In general, though, I'm finding
it difficult to conceptualize what my tests should be doing, testing, and
telling me about my code.
The second problem is complexity. I'll paraphrase a well known saying about
regex: "I wrote a unittest for my Python code, now I have two problems." Just
to test my relatively simple code, I'm trying to do things like create
temporary file structures, do some datetime parsing, etc. None of this is easy
code for me to write at this stage. When a test fails, it could just as easily
be a problem with the test code. (I'm unaccountably reminded about infinite
regress here.) I *think* the answer must be that the benefit of writing tests
on large, complex projects far outweighs the cost. But I've seen even small
projects tout their test coverage.
I think what I need is a conceptual shift: how do python programmers use
unittests?
My next step is to look into buying some real books.
Resources
Agile Testing Blog: http://agiletesting.blogspot.com/
Tool Taxonomy: http://wiki.python.org/moin/PythonTestingToolsTaxonomy
Doug Hellman's Blog:
http://www.doughellmann.com/articles/pythonmagazine/completely-different/2008-01-testing-tools/
Unittest Docs: http://docs.python.org/3/library/unittest.html
Nose Docs https://nose.readthedocs.org/en/latest/testing.html
Fixture Docs http://www.farmdev.com/projects/fixture/index.html
PyCon 2010:
http://blip.tv/pycon-us-videos-2009-2010-2011/pycon-2010-new-and-improved-coming-changes-to-unittest-the-standard-library-test-framework-52-3283307
PyCon 2011:
http://blip.tv/pycon-us-videos-2009-2010-2011/pycon-2011-testing-with-mock-4899484
PyOhio July 2012. This looks perfect except it is missing about half the audio.
https://www.youtube.com/watch?v=jTJHQ-zQMk4
Marakana Tech TV: https://www.youtube.com/watch?v=xdY7svOz6n4
At O'Reilly: http://onlamp.com/pub/a/python/2004/12/02/tdd_pyunit.html
- Luke Mergner
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
[Tutor] IronPython any tutors with experience out there?
Hello PythonTutor
- I'm a scientist and very happy with python 2.7. I have been asked to
assist a development program where everything is written in dotNET/ C#
(visual studio 2012) and luckily Microsoft Visual Studio supports
IronPython which is a clean Python implementation in C#, so I can use the
Python syntax I'm so familiar with...
However ... to interact with the c# modules which my colleagues wrote, I
need to add "clr" references.
Example:
>>> import clr
>>> clr.AddReferenceToFile("Mapack.dll")
>>> from Mapack import *
>>> dir()
Unfortunately I can't get it to run and the tutorials after 12 hours of
google-reading are terribly sparse. Is there any experience somewhere in
the network with IronPython in VSTO-2012 in this forum and what could the
top five clr.addreference bugs be?
Kind Regards,
--
Bjorn
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] how to struct.pack a unicode string?
On 03/01/13 23:52, eryksun wrote:
On Tue, Jan 1, 2013 at 1:29 AM, Steven D'Aprano wrote:
2 Since "wide builds" use so much extra memory for the average ASCII
string, hardly anyone uses them.
On Windows (and I think OS X, too) a narrow build has been practical
since the wchar_t type is 16-bit. As to Linux I'm most familiar with
Debian, which uses a wide build. Do you know off-hand which distros
release a narrow build?
Centos, and presumably therefore Red Hat do. Fedora did, and I presume
still do.
I didn't actually realize until now that Debian defaults to a wide
build.
But more important than the memory savings, it means that for the first
time Python's handling of Unicode strings is correct for the entire range
of all one million plus characters, not just the first 65 thousand.
Still, be careful not to split 'characters':
>>> list(normalize('NFC', '\u1ebf'))
['ế']
>>> list(normalize('NFD', '\u1ebf'))
['e', '̂', '́']
Yes, but presumably if you are normalizing to decomposed forms (NFD or NFKD
modes), you're doing it for a reason and are taking care not to let the
accents wander away from their base character, unless you want them to.
By the way, for anyone else trying this, the normalize function above is not
a built-in, it comes from the unicodedata module.
More on normalization:
https://en.wikipedia.org/wiki/Unicode_equivalence
Doing-a-lot-of-presuming-today-ly y'rs,
--
Steven
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Limitation of int() in converting strings
On 2 January 2013 17:59, Alan Gauld wrote: > On 01/02/2013 11:41 AM, Steven D'Aprano wrote: > >[SNIP] >> But __index__ is a special method that converts to int without rounding >> or truncating, intended only for types that emulate ints but not other >> numeric types: > > > And this was the new bit I didn't know about. > > [SNIP] help(int.__index__) > Help on wrapper_descriptor: > > __index__(...) > x[y:z] <==> x[y.__index__():z.__index__()] > > > Bingo! Although still doesn't anything explicitly about the need for an > integer! The operator.index builtin checks that an int/long is returned. The same is true of the underlying C-API that is used internally by indexable sequences (list, tuple, etc.). $ python Python 2.7.1 (r271:86832, Nov 27 2010, 18:30:46) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import operator >>> class A(object): ... def __index__(self): ... return 4.5 ... >>> a = A() >>> a.__index__() 4.5 >>> operator.index(a) Traceback (most recent call last): File "", line 1, in TypeError: __index__ returned non-(int,long) (type float) >>> b = [1,2,3] >>> b[a] Traceback (most recent call last): File "", line 1, in TypeError: __index__ returned non-(int,long) (type float) You only need to know about this feature if you are implementing a custom integer type or a custom sequence type (both of which are things that most Python users will never do). This particular special method is probably only really documented in the PEP: http://www.python.org/dev/peps/pep-0357/ For my purposes, the important thing is that the method is only supposed to be implemented on types that always exactly represent integers, so it is not usable for converting e.g. floats to integers. Oscar ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] how to struct.pack a unicode string?
On Tue, Jan 1, 2013 at 1:29 AM, Steven D'Aprano wrote:
>
> 2 Since "wide builds" use so much extra memory for the average ASCII
> string, hardly anyone uses them.
On Windows (and I think OS X, too) a narrow build has been practical
since the wchar_t type is 16-bit. As to Linux I'm most familiar with
Debian, which uses a wide build. Do you know off-hand which distros
release a narrow build?
> But more important than the memory savings, it means that for the first
> time Python's handling of Unicode strings is correct for the entire range
> of all one million plus characters, not just the first 65 thousand.
Still, be careful not to split 'characters':
>>> list(normalize('NFC', '\u1ebf'))
['ế']
>>> list(normalize('NFD', '\u1ebf'))
['e', '̂', '́']
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] writing effective unittests
> Hi, > > I am trying to learn a bit of test-driven programming using unittests and > nosetests. I am having trouble finding resources that explain how to write > effective tests. I am not a programmer or a student, so I do not have access > to > collegues or classes. I'm hoping that someone knows of a good tutorial that > I've missed. Based on the O'Reilly 2-Part introduction, I hope that > learning to write tests will make my code better. > I am currently readingPython Testing: Beginner's Guideby Daniel Arbuckle http://www.amazon.com/Python-Testing-Beginners-Arbuckle-Daniel/dp/1847198848 It covers doctest, unittest, nose, test-driven programming and more (I am just half way the book). Itś a beginnnerś guide but I like it. Some of the examples are a little too simplistic (the author keeps writing 'fixtures' for datetime.datetime.now(), for example). But again, I am happy that I bought it. The only thing I did wrong: I bought it too late! Apparently hte FIRST thing one has to do is write tests, not the LAST. Cheers, Albert-Jan ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] writing effective unittests
Steven D'Aprano wrote: > Notice that tests are not necessarily definitive. I haven't tested that > spam(n) returns a string for every imaginable integer n, because there > are too many. Instead, I just test a small, representative sample. > > Likewise I haven't tested that spam() might succeed when given four > arguments, or forty-four. I trust that if it fails with two arguments, > and fails with three arguments, it will fail with more than three too. > > Don't make the mistake of thinking that tests need to cover EVERYTHING > or else they are useless. 10% test coverage is better than nothing, > 50% is better still, and 90% is better still. (100% is, of course, > impossible.) There's no way you can test even 10% of the possible inputs for even the simple def mul(a, b): return a * b as there is an infinite number of such inputs. The more common and less ambitious definition of coverage is "What portion of your code is executed by your tests? There's an excellent tool to answer that question: http://nedbatchelder.com/code/coverage/ It reports executed lines, so problematic code like x = 1 if True else undefined will be reported as 100% coverage, but it is still a big help in improving your tests. > Let's add some more tests to SpamTests: > > def testHasDocString(self): > # Test that the spam function has a docstring. > doc = spam.__doc__ > self.assertTrue(doc is not None) You should rather run a tool like pylint to find missing docstrings. > The important thing here is that there process continually goes back and > forth between the tests and the main code. Adding new tests reveals bugs > in the code, or missing functionality, which you then fix, then write > more tests, until you can no longer think of any more tests. If you take this advice seriously you will soon rewrite your application code to make it easier to test it. At least for me the impact of unittests was larger than I had expected. I really should do more of them... ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Limitation of int() in converting strings
On Tue, Jan 1, 2013 at 12:07 AM, Steven D'Aprano wrote: > > Again, I was mistaken. x%1 is not suitable to get the fraction part of a > number in Python: it returns the wrong result for negative values. You need > math.modf: > > py> x = -99.25 > py> x % 1 # want -0.25 > 0.75 > py> math.modf(x) > (-0.25, -99.0) math.modf wraps libm's modf, which takes a double. This isn't suitable for Decimal, Fraction, or a custom number type. What's wrong with using math.trunc for this? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
