Hi Craig,
first of all, I'd like to thank you for bringing up unit tests and TDD
from time to time, because I really think it'll be necessary in the
future, and the sooner we get this done, the more time we'll save in
the long run.
I'd like to offer some help as to how to implement unit tests in a very
simple fashion. There are some complaints about the lack of a good
testing framework, but I think GLib.Test or Gee.TestCase might be
sufficient for implementing good automated tests.
If you want to see how to implement unit tests, you should take a look
at libgee. The tests are really simple and assure that all basic
operations, like adding an element to a list/set/queue work as
intended. If you introduce a bug that breaks one of these fundamental
operations, you'll know immediately what you broke and can fix it even
before any buggy code gets pushed.
One really important prerequisite for unit tests is modular code and
preferably loose coupling, i.e. classes should be independent and
possibly agnostic of each other. This way, you can test classes or
single functions separately which makes tests shorter and simplifies
finding misbehaving code.
Now libgee is very different from GUI applications where you have much
more complex dependencies between different code modules. Here it helps
choosing a good design pattern and separating presentation, logic,
helper classes and so on. I think the Passive View
(http://martinfowler.com/eaaDev/PassiveScreen.html) pattern is great,
because it is rather simple compared to other patterns, but quite
effective in achieving code separation and loose coupling.
A short time ago I restructured and rewrote most of cable's code
(lp:cable), which is luckily still in its infancy, to make it adhere to
the Passive View pattern and to make the code more manageable. When I
was done, I noticed that it made implementing unit test really easy,
because of its highly modular nature and the separation of GUI code and
backend logic and services. I already implemented (very simple) unit
tests in cable, but commented them out for now. The tests would be run
from the command line using "cable --unit-test", but there's no reason
you couldn't implement automated tests using cmake, but I'll let
someone else figure that out. If you're interested, I used GLib.Test,
but I plan to use libgee's TestCase in the future.
If you want to take a look, the tests are in /src/Tests
(http://bazaar.launchpad.net/~cable-developers/cable/trunk/files/head:/src/).
You'll see that there aren't many tests yet, but I do want to implement
tests for every class, and maybe even get real TDD working. I won't
have time to do so in the next 2 or 3 weeks, unfortunately, but I'll
try to get to it as soon as possible.
I hope this helps a little, maybe I'll write a guide if it works out in
cable, but for now it's more of an experiment. (Note that I don't have
any experience with TDD, and this is the first time I actually use unit
tests, so please correct me if I said something stupid)
On Mon, Aug 19, 2013 at 12:58 AM, Daniel Foré
<dan...@elementaryos.org> wrote:
This all sounds great and I think everybody is pro-testing, however
I've yet to see a reproduce-able example or a guide regarding any
kind of tests being implemented (especially by those extremely vocal
about their importance). Not books or articles about why testing is
good, but something that actually shows a person how to write tests
for their apps.
So, as Linus would say, "Talk is cheap. Show me the code."
—
Sent from Mailbox for iPhone
On Sun, Aug 18, 2013 at 3:47 PM, Craig <webe...@gmail.com> wrote:
Hi Alex,
tl;dr: Unit tests are pretty much necessary to have an architecture
on which you can run automatic system-level tests, and if you aren't
automating then testing becomes too impractical.
When you describe "system tests" you are actually describing what
are called "acceptance tests" or "behavioral" tests
(http://www.extremeprogramming.org/rules/functionaltests.html). Unit
tests test small units of code such as classes or functions.
Traditional TDD relies primarily upon unit tests, and those are
primarily what I'm referring to.
One of the primary purposes of unit testing is to ensure good code
architecture. If you don't unit test, you probably won't have good
access points for your acceptance tests (how do you verify that that
Gtk.Label has the correct text when you can only access the top
level window?), so automation will be out of the question. And if
you aren't automating then you can't continuously integrate (running
all tests every time a change is made to the repository in order to
find bugs as soon as they are made). Honestly, if you aren't
automating then testing becomes too impractical.
On Aug 18, 2013 5:10 PM, "Alex Lourie" <djay...@gmail.com> wrote:
Hi Craig
For the clarification purposes, I'd like to separate 'automatic
tests (system testst)' and 'unittests'. I consider them different
things. Unittests are pieces of code that test some other pieces of
the code. System tests are scripts/code/steps that test that your
program (or part of it) works. Unittests are usually run
automatically (by, say, unittesting framework). System tests could
be run automatically or manually. There are, sometimes, frameworks
for that, but in most cases it's either manual or custom developed.
Unittests are (usually) developed by the same developer who
developed the original code, just as in your TDD example. System
tests are best developed by external party (such as users).
From here on, I can agree with you on point 1, and the naming.
Basically, we all agree that having testing is a good practice and
a feasible way to manage the complexity of software. But
unittesting cannot test the logical connections between the blocks
of code in the program. That's the job for system testing.
I don't care how we call it. The more systematic testing we do for
Elementary the better it's going to be, and the more chances we
have to sustain growth.
So I would just like to see testing implemented. Any kind of it.
On Sun, Aug 18, 2013 at 10:56 PM, Craig <webe...@gmail.com> wrote:
Hi Alex,
To correct you on a couple of things:
1. TDD **does not** require you to have all or even several of the
tests written before hand. It simply requires you to have the test
written for the next change you are about to make. The idea is to
write a test, run the test to watch it fail (this helps verify you
wrote your test correctly), add the simplest code to make the test
pass, run the test to watch it pass (and verify your code
additions worked). Then you rinse and repeat.
2. TDD is actually a simplified form of what developers do
already. That is, you usually write some code, run your code, then
visually verify that it worked. TDD just crystalizes this process
in code which can be executed later. TDD isn't hard, so it's well
within the capacity of all of our devs; however, it does taking
some getting used to. TDD is the best and fastest way to develop
quality code and it's the ONLY practical way to raise the ceiling
on the amount of code complexity a team of a given size can handle
(there is a lot of research and professional heuristics about
this). If a developer doesn't have the will to do this, they
aren't taking their discipline seriously and, frankly, are a
danger to any project that values quality. Besides, I've never met
a developer who has hit the complexity ceiling *and* who has
practiced TDD who doesn't advocate this kind of testing.
With those corrections in mind, I can't see any difference between
your first point and "real TDD". I agree that your 2nd point is a
good idea. Automated testing can't capture everything, and it's
definitely important to have some hands-on testing that we could
run through a few times every release. But we should always be
working toward automated testing so developers can, you know,
develop.
On Sun, Aug 18, 2013 at 2:05 PM, Alex Lourie <djay...@gmail.com>
wrote:
Ok, I have not added a single line of code to elementary yet, and
gave some decent amount of headache to real devs around here. I
have though participated in development of many products, small
and large, and so here it goes.
TDD in its pure form requires having tests written and ready
before the code is written. Then failing tests are being
implemented one by one, having the code both tested and working
in the end of this process. I highly doubt that any of Elementary
devs have the capacity or the will to follow this. Especially,
when many of our projects are built using 'try and error'
methodology, that is they follow the 'Hey! Let's try this!' rule.
Which is awesome, as it allows fast development and really quick
release cycles.
Now, what I think would fit Elementary much better is simply
testing (yes, that QA thing everyone loves and adores) - the T
from TDD, which is the most important part. The best way to do
that is, of course, to create a bunch of automatic tests, but
that's not really feasible either. So as I see it, we can do 2
things:
1. Start looking into unittesting as much and as early as
possible, and have devs starting to create tests. This has the
benefit of devs learning to create unittests and having some of
the code tested, and also, potentially, at some point, it could
help moving to real TDD.
2. Start creating manual test procedures for basic staff. Yes, it
requires a lot of human power and time, but also do translations.
This is the most boring part of software engineering, but it is
the one that can bring balance to the force. Also, some of these
can sometimes be automated too, so it's not all that aweful.
I personally think we need both 1 and 2. I am a strong believer
in testing as means to improve the product. But TDD is probably
an overshoot in our case. We need to start with *something*, and
right now we don't really have it.
On Sun, Aug 18, 2013 at 8:41 PM, Craig <webe...@gmail.com> wrote:
Hello,
I posted the following message on Google Plus yesterday, but it
occurred to me that the weekend may not be prime time for
checking that social network. I think this message is pretty
important, so I want to post it again here: (I apologize in
advance for its length)
Congratulations to all the developers who made Luna such a
success! You've done a great job and delivered an incredible
Linux experience!
I know I bring this up periodically, but I'm concerned that
Luna + 1 and future releases will take more and more time to
release, and/or that you will quickly reach a ceiling with
respect to the amount of code we'll be able to maintain before
quality degrades.
The cause for my concern is the nature of complexity: as
software grows (that is, as code is added), bugs grow
exponentially (complexity increases exponentially with logic,
and bugs grow linearly with complexity). If we don't start
working toward solutions that will scale with this problem, we
**will** hit a ceiling with respect to the amount of complexity
we will be able to support (this means fewer features or
less-powerful features). I promise.
I know some in the community are working toward this goal, but
I think it's going to take a concerted effort on the part of
the developers to take this problem seriously. I urge you all
to take this problem as seriously as you take the rest of the
user experience (because bugs are, at the end of the day, a
sharp degradation of the user experience).
In my experience, the silver bullet for combating this problem
is test driven development. If you look around the software
development industry, code is improving, and it's largely
because TDD is catching on. And Google is a good role model in
this regard (not just for us, but for everyone--they are
pioneers of code quality). If you're a developer and you're
unfamiliar with TDD, take some time and research it. It will
pay dividends immediately. If you have any questions about
development, I'm happy to provide my advice as a professional
developer. Also, read up on Google's testing strategies (I
recommend http://www.amazon.com/Google-Tests-Software-James-Whittaker/dp/0321803027
_How Google Tests Software_).
You guys are a _great_ UX shop, now let's become a great code
shop. I hope this analogy doesn't offend anyone who is
passionate about their tech brands, but my advice is this:
Design like Apple, develop like Google.
I really push you developers to continue to strive to hone your
craft the way Daniel and Cassidy (and any other UX designers)
are learning to hone theirs.
P.S., Sorry for the book, and I hope you all take this as
respectful, constructive criticism. _Please_ ask me anything
about development, especially with respect to how we can keep
quality high using processes rather than sheer developer effort
(so as to free you developers to work on interesting problems
rather than bug hunting).
Thanks for reading,
Craig
--
Mailing list: https://launchpad.net/~elementary-dev-community
Post to : elementary-dev-community@lists.launchpad.net
Unsubscribe : https://launchpad.net/~elementary-dev-community
More help : https://help.launchpad.net/ListHelp
--
Alex Lourie
--
Alex Lourie
--
Mailing list: https://launchpad.net/~elementary-dev-community
Post to : elementary-dev-community@lists.launchpad.net
Unsubscribe : https://launchpad.net/~elementary-dev-community
More help : https://help.launchpad.net/ListHelp