You are probably right that integration testing has a lot more value.
However, unit tests are still useful in their own right and may
encourage people to write smaller classes that can be tested easily.
As you point out, integration testing can be done at several levels with
mock implementations used to limit the scope of the tests.
A full set of integration tests would take a long time to run and even
longer to construct.
You may also be right that building a lot of new unit tests to test old
features may not be very valuable but getting developers to move towards
TDD might reduce the number of regressions.
Unit testing does, at least, give positive proof of what has been tested
when new code is presented for review.
It also demonstrates the use of a class which may add to the specifications.
It is usually possible to exclude tests that are redundant or not
relative to the new features being added.
There is not a lot of point in running low-level unit tests repeatedly
in a development cycle if you are not changing the classes being tested.
This is trivial to manage in Maven with jUnit and Surefire.
Not sure about the Gradle build.
Proper integration tests that cover load testing, synchronization,
caching and locking under tests of all UI features would be wonderful to
have but would take a long time to construct given the number of feature
combinations, stack configurations and use cases.
A full test would take a long time to run and would be something that
would be useful only a few times in each release cycle.
Very useful if versions are going to be released with any sort of
implied warranty about quality.
The current activity is a big step forward and Taher's leadership in
modernizing the framework seems to have inspired a lot of new activity
and a more modern way of thinking about software development.
Well done.
Ron
On 18/07/2016 10:19 PM, Scott Gray wrote:
I know I'm late to the party here, but I just want to say that I think
integration tests have far greater value to OFBiz than unit tests. Mostly
because we tend to have quite a low number of tests and integration tests
give us much better coverage per line of test code and the tests are much
closer to the real world scenarios the application might encounter.
I don't really see how unit tests could be applied to non-framework testing
in a useful manner, could you expand on your vision in that regard? I mean
would we be testing something smaller than a service as the 'unit'? What
would we mock? Would transaction management still be active? What happens
when the service calls another service, I guess we mock the response from
that service (how)?
It just seems a very complicated method to achieve a less thorough but
albeit faster (maybe) test result.
A build, data load and full test run takes 4m 9s on my laptop. Excluding
build, data load and framework tests: the application level tests take 35s,
not very expensive IMO. The data load time can be reduced to practically
nothing by copying a clean slate database into runtime for each run.
I'm mostly just suggesting we be wary of adding complicated testing
procedures in the hope of achieving some 'best practice' result which in
reality will provide minimal benefits.
Regards
Scott
On 18 July 2016 at 18:57, Taher Alkhateeb <slidingfilame...@gmail.com>
wrote:
Hello Akash,
Fantastic, I have a few unit tests almost done to be included in the start
component. I will create a new subtask under OFBIZ-1463 to commit the tests
so you can use them as a reference if you like to.
I also recommend that you follow the same directory structure between the
test code and production code. So for example:
Production code: framework/start/src/main/java/org/apache/ofbiz/base/start
Test code: framework/start/src/test/java/org/apache/ofbiz/base/start
The benefit of this hierarchy is that you can access non-public (package
protected) methods for testing. This is in fact exactly what I needed to be
able to apply some of the tests.
Cheers,
Taher Alkhateeb
On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <akash.j...@hotwaxsystems.com>
wrote:
Thanks Taher for nice initiative!
We are planning to written unit tests to all components under OFBIZ-1463
Thanks and Regards
--
Akash Jain
On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
slidingfilame...@gmail.com> wrote:
Hello Everyone,
In reference to this thread and the Jira OFBIZ-7254, I'm very happy to
announce that OFBiz is now ready for applying unit tests with only 8
new
lines of code in the build script (r1753143) :)
I recommend we do the following moving forward:
1- Introduce unit tests as much as we can to all components
2- Migrate most of the integration tests we currently have to unit
tests
(since they are designed to do very little integration).
This is a great chance for us to practice real TDD in OFBiz
Cheers,
Taher Alkhateeb
On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
pranay.pan...@hotwaxsystems.com> wrote:
+1
Best regards,
Pranay Pandey
HotWax Systems
http://www.hotwaxsystems.com/
On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
slidingfilame...@gmail.com
wrote:
Hello Everyone,
I was able to get a few tests running and this is very doable. But
I
faced
a big problem in designing the testing framework because of ANT.
The problem
----------------
The way the build scripts are designed in OFBiz are very complex. A
master
file calls other files which call other files. And in the middle
you
have
external libraries (ant-contrib) and macros, and variables, and
class
path
declarations, and and and ....
I cannot declare the tests programmatically (with JUnit test
suites)
because this means lower level components would depend on higher
level
components. So I have to do it in ANT, by navigating this maze of
build
scripts, and it was a headache for me just to read them, let alone
modify
them to create a testing framework.
Suggested Solution
------------------------
I suggest to implement the testing framework in Gradle, and simply
call
it
from within ant. This is a middle solution that sustains ant for
now,
but
can allow us to switch out later.
This means I will just add one more file called build.gradle in the
top
level directory, and figure out the business logic for calling the
test
suites from that file
I look forward to your feedback.
Regards,
Taher Alkhateeb
On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
slidingfilame...@gmail.com>
wrote:
Hi Everyone,
Thank you all for your support, JIRA created in
https://issues.apache.org/jira/browse/OFBIZ-7254
I will start working on it and try to implement ASAP to get my
focus
back
on refactoring.
Cheers!
Taher Alkhateeb
On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
deepak.di...@hotwaxsystems.com> wrote:
+1
Thanks & Regards
--
Deepak Dixit
www.hotwaxsystems.com
On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
mridul.pat...@hotwaxsystems.com> wrote:
+1
Makes perfect sense.
--
Thanks & Regards,
Mridul Pathak
Senior Manager
HotWax Systems
http://www.hotwaxsystems.com
On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
slidingfilame...@gmail.com>
wrote:
Hello Everyone,
After refactoring the start component and while starting on
the
base
component I realized that the testing framework for OFBiz is
not
good.
You
cannot do real test driven development or red-green-refactor
with
the
current setup, hence my proposal to change it. I explain
below:
Problem with current design
----------------------------------------
- What we have right now is not unit tests, it's really
integration
tests.
You have to start the framework, the database, the service
engine,
the
entity engine and pretty much everything.
- Testing is very slow, because it's an integration test as
I
mentioned
above. 10 minutes on a good computer!
- There is zero mocking! We actually have to --load-data for
things
to
work. Again, these are integration tests.
- Too complex: Integration tests by their nature are
grabbing
too
much.
Mind you, I am not objecting to integration tests (I
actually
like
them)
but I am objecting to not having real unit-tests. Unit tests
should
all
run
in a few seconds.
Proposed solution
--------------------------
- We keep what is considered real integration tests the way
they
are
right
now and keep using them
- We move what should be unit tests into simple JUnit
classes,
and
we
do
not run them using java -jar ofbiz.jar --test, but instead
run
them
directly from the build.xml script, so these files are not
identified
in
any XML document, but are simply called immediately from the
build
scripts.
- We clearly mark the difference between integration tests
and
unit
tests
(inside the source files or in the suite declarations).
- We change the run-tests target in build.xml to run both
unit
tests
and
integration tests.
I intend to heavily refactor the framework and I would feel
better
about
introducing this change while refactoring. What do you guys
think?
Ideas?
Suggestions? Approvals and thumbs up?
Regards,
Taher Alkhateeb
--
Ron Wheeler
President
Artifact Software Inc
email: rwhee...@artifact-software.com
skype: ronaldmwheeler
phone: 866-970-2435, ext 102
