Hi Scott, Well spoken thank you. Okay may I suggest that for any such work we will discuss it here see its Merit and if it makes sense then we take it in. It is a little early to discuss it right now because we did not yet go to the higher-level components. Once we do I'll be sure to have a conversation about this in here and would appreciate your input to it.
Regards, Taher Alkhateeb On Jul 19, 2016 9:00 AM, "Scott Gray" <scott.g...@hotwaxsystems.com> wrote: > Yeah I'm sure unit tests probably worked well for the start component given > it is the lowest level component in the system and closest to a basic java > app. I just think the value proposition might decrease the further up the > stack we move with them. I'm not against unit tests when they prove > useful, but further up the stack I think we should prove the case for them > before doing much work to support the mocking that will be required to keep > them inline with 'best practices'. > > In OFBiz "bad or unwanted" tends to come mostly in the way of increasing > complexity from features that require more effort to maintain than the > value they provide to the community. I think there's a chance attempting > to mock service tests could fall into that bucket. I could be wrong, but I > think it's worth looking into before we declare that unit tests are the > best form of testing for OFBiz. > > Regards > Scott > > On 19 July 2016 at 17:37, Taher Alkhateeb <slidingfilame...@gmail.com> > wrote: > > > Hi Scott, > > > > Thank you for the feedback. To be focused exactly on integration vs > unit, I > > already mentioned above that at least for me the main objective is to > > confidently and quickly run the tests in short bursts of red-green > > refactor. This allows me to refactor code without waiting in front of the > > screen in between test cycles thus giving me immediate feedback on any > > errors I made. Perhaps my intro was too long so this is the squeezed out > > version of it. > > > > I already had one round of testing in the start component which was much > > faster that way and had an immediate impact. Oh and by the way, you > cannot > > test the start component with integration tests for example unless you do > > it from an external component which cannot access package protected > items. > > > > This style of coding is applicable I think to any software project > > inclusive of OFBiz. Maybe in certain components more than others or > certain > > areas more than others. But I can't see how it could be bad or unwanted. > > > > Taher Alkhateeb > > > > On Jul 19, 2016 8:18 AM, "Scott Gray" <scott.g...@hotwaxsystems.com> > > wrote: > > > > > Thanks Ron and Taher for your responses, I appreciate them but I don't > > hear > > > much in the conversation that speaks to the value of unit tests over > > > integration tests. Most of the thoughts shared speaks more to the > value > > of > > > tests rather than differences in the type of tests. > > > > > > Speed: At an application level we have ~685 tests that run in 35 > seconds > > > (excluding build and data load). Another point is that there isn't > much > > > reason why tests can't be run in parallel rather than sequentially. > > > TDD: Integration tests in OFBiz are as simple if not simpler to write > > than > > > unit tests and just as useful for TDD. > > > I'm not sure if I missed any other points raised regarding integration > > vs. > > > unit tests? > > > > > > I'm not looking to start a big long debate on the topic, I just wanted > to > > > speak out that someone out there (me) doesn't think unit tests are the > > best > > > solution for testing OFBiz applications. > > > > > > Regards > > > Scott > > > > > > On 19 July 2016 at 16:52, Taher Alkhateeb <slidingfilame...@gmail.com> > > > wrote: > > > > > > > Hi Scott, > > > > > > > > Thank you for your input. Your ideas are thought provoking and > > enriching > > > to > > > > the conversation. > > > > > > > > The way I look at it, the general rule is usually many unit tests, > less > > > > integration tests, lesser functional tests. So we are not excluding > any > > > > types of test, all of them are important in certain areas for certain > > > > purposes with certain quantity. Usually integration tests are less > > > because > > > > as you said they just grab more. I like the picture below as a > general > > > > guideline > > > > > > > > http://i.stack.imgur.com/fjQvQ.png > > > > > > > > As you already mentioned unit tests are useful for the framework. I > > > > discovered errors in code that I wrote which I was very very careful > > > with. > > > > I immediately learned the lesson that humans are not designed to > code, > > > and > > > > TDD gives you confidence as you build your code with those short > 30-60 > > > > second red-green refactors. I feel much much safer and more confident > > > > writing code that way, the test also documents how to use the api, > > > > refactoring and feature change becomes less terrifying. Also messy > code > > > is > > > > usually not test friendly thus refactoring and unit tests go > > hand-in-hand > > > > for improving the code base. > > > > > > > > Also, I am sorry to say that the framework code is rather messy and > > > brittle > > > > in many places. I think you probably encountered this yourself. The > > same > > > is > > > > said for the applications. Now If we start refactoring without unit > > tests > > > > then we are back to scary business. So much can go wrong so fast and > > > break > > > > things you never expected to break. The framework with all > applications > > > > require heavy refactoring of things like massive ugly methods, > > sandwiched > > > > logic, heavy shared mutable state, hidden dependencies, poor > > > > interfaces, and much more. Every time I touch something I get shocked > > at > > > > how bad it looks, spaghetti logic to no end. If you refactor with > > > > integration tests instead of unit tests then you will come down to a > > > > screeching halt as you wait between these test cycles. My computer > > > actually > > > > takes 10 minutes or more for a full clean load data and testing. > > > > > > > > Now talking about mocking you raise some interesting points, what can > > you > > > > mock vs utilizing integration tests? This is an excellent question > that > > > > does not have a simple answer, and you mostly learn as you code, > > > difficult > > > > to envision without coding. However simple things like Java services > > can > > > be > > > > mocked with a standard mocking class that you use everywhere. > > > Transactions > > > > are difficult to mock and better left to integration tests I think. > > SECAs > > > > might be mocked by simply passing output to input as chaining > methods. > > > Mind > > > > you I am not 100% sure of all of this, coding is the ultimate guide > to > > > what > > > > can or cannot be done. > > > > > > > > So I am not suggesting unit tests as a best-practice (even though it > > is). > > > > Instead I suggest it as something that I and others did and got a > huge > > > > psychological relief and confidence and comfort from. Swimming in > code > > > > without tests is a terrifying business, made more terrifying if the > > code > > > is > > > > bad. Short bursts of red/green/red/green makes you feel good as you > > build > > > > up your logic. And I don't have an exact vision of how to do it, > > because > > > > the details always win. > > > > > > > > I look forward to hearing your thoughts and thank you for enriching > > this > > > > conversation. > > > > > > > > Taher Alkhateeb > > > > > > > > On Tuesday, 19 July 2016, Scott Gray <scott.g...@hotwaxsystems.com> > > > wrote: > > > > > > > > > I know I'm late to the party here, but I just want to say that I > > think > > > > > integration tests have far greater value to OFBiz than unit tests. > > > > Mostly > > > > > because we tend to have quite a low number of tests and integration > > > tests > > > > > give us much better coverage per line of test code and the tests > are > > > much > > > > > closer to the real world scenarios the application might encounter. > > > > > > > > > > I don't really see how unit tests could be applied to non-framework > > > > testing > > > > > in a useful manner, could you expand on your vision in that regard? > > I > > > > mean > > > > > would we be testing something smaller than a service as the 'unit'? > > > What > > > > > would we mock? Would transaction management still be active? What > > > happens > > > > > when the service calls another service, I guess we mock the > response > > > from > > > > > that service (how)? > > > > > > > > > > It just seems a very complicated method to achieve a less thorough > > but > > > > > albeit faster (maybe) test result. > > > > > > > > > > A build, data load and full test run takes 4m 9s on my laptop. > > > Excluding > > > > > build, data load and framework tests: the application level tests > > take > > > > 35s, > > > > > not very expensive IMO. The data load time can be reduced to > > > practically > > > > > nothing by copying a clean slate database into runtime for each > run. > > > > > > > > > > I'm mostly just suggesting we be wary of adding complicated testing > > > > > procedures in the hope of achieving some 'best practice' result > which > > > in > > > > > reality will provide minimal benefits. > > > > > > > > > > Regards > > > > > Scott > > > > > > > > > > On 18 July 2016 at 18:57, Taher Alkhateeb < > > slidingfilame...@gmail.com > > > > > <javascript:;>> > > > > > wrote: > > > > > > > > > > > Hello Akash, > > > > > > > > > > > > Fantastic, I have a few unit tests almost done to be included in > > the > > > > > start > > > > > > component. I will create a new subtask under OFBIZ-1463 to commit > > the > > > > > tests > > > > > > so you can use them as a reference if you like to. > > > > > > > > > > > > I also recommend that you follow the same directory structure > > between > > > > the > > > > > > test code and production code. So for example: > > > > > > > > > > > > Production code: > > > > > framework/start/src/main/java/org/apache/ofbiz/base/start > > > > > > Test code: > > framework/start/src/test/java/org/apache/ofbiz/base/start > > > > > > > > > > > > The benefit of this hierarchy is that you can access non-public > > > > (package > > > > > > protected) methods for testing. This is in fact exactly what I > > needed > > > > to > > > > > be > > > > > > able to apply some of the tests. > > > > > > > > > > > > Cheers, > > > > > > > > > > > > Taher Alkhateeb > > > > > > > > > > > > On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain < > > > > > akash.j...@hotwaxsystems.com <javascript:;>> > > > > > > wrote: > > > > > > > > > > > > > Thanks Taher for nice initiative! > > > > > > > > > > > > > > We are planning to written unit tests to all components under > > > > > OFBIZ-1463 > > > > > > > > > > > > > > Thanks and Regards > > > > > > > -- > > > > > > > Akash Jain > > > > > > > > > > > > > > On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb < > > > > > > > slidingfilame...@gmail.com <javascript:;>> wrote: > > > > > > > > > > > > > > > Hello Everyone, > > > > > > > > > > > > > > > > In reference to this thread and the Jira OFBIZ-7254, I'm very > > > happy > > > > > to > > > > > > > > announce that OFBiz is now ready for applying unit tests with > > > only > > > > 8 > > > > > > new > > > > > > > > lines of code in the build script (r1753143) :) > > > > > > > > > > > > > > > > I recommend we do the following moving forward: > > > > > > > > > > > > > > > > 1- Introduce unit tests as much as we can to all components > > > > > > > > 2- Migrate most of the integration tests we currently have to > > > unit > > > > > > tests > > > > > > > > (since they are designed to do very little integration). > > > > > > > > > > > > > > > > This is a great chance for us to practice real TDD in OFBiz > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > > > > > > > > Taher Alkhateeb > > > > > > > > > > > > > > > > On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey < > > > > > > > > pranay.pan...@hotwaxsystems.com <javascript:;>> wrote: > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > Best regards, > > > > > > > > > > > > > > > > > > Pranay Pandey > > > > > > > > > HotWax Systems > > > > > > > > > http://www.hotwaxsystems.com/ > > > > > > > > > > > > > > > > > > On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb < > > > > > > > > > slidingfilame...@gmail.com <javascript:;> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hello Everyone, > > > > > > > > > > > > > > > > > > > > I was able to get a few tests running and this is very > > > doable. > > > > > But > > > > > > I > > > > > > > > > faced > > > > > > > > > > a big problem in designing the testing framework because > of > > > > ANT. > > > > > > > > > > > > > > > > > > > > The problem > > > > > > > > > > ---------------- > > > > > > > > > > The way the build scripts are designed in OFBiz are very > > > > > complex. A > > > > > > > > > master > > > > > > > > > > file calls other files which call other files. And in the > > > > middle > > > > > > you > > > > > > > > have > > > > > > > > > > external libraries (ant-contrib) and macros, and > variables, > > > and > > > > > > class > > > > > > > > > path > > > > > > > > > > declarations, and and and .... > > > > > > > > > > > > > > > > > > > > I cannot declare the tests programmatically (with JUnit > > test > > > > > > suites) > > > > > > > > > > because this means lower level components would depend on > > > > higher > > > > > > > level > > > > > > > > > > components. So I have to do it in ANT, by navigating this > > > maze > > > > of > > > > > > > build > > > > > > > > > > scripts, and it was a headache for me just to read them, > > let > > > > > alone > > > > > > > > modify > > > > > > > > > > them to create a testing framework. > > > > > > > > > > > > > > > > > > > > Suggested Solution > > > > > > > > > > ------------------------ > > > > > > > > > > I suggest to implement the testing framework in Gradle, > and > > > > > simply > > > > > > > call > > > > > > > > > it > > > > > > > > > > from within ant. This is a middle solution that sustains > > ant > > > > for > > > > > > now, > > > > > > > > but > > > > > > > > > > can allow us to switch out later. > > > > > > > > > > > > > > > > > > > > This means I will just add one more file called > > build.gradle > > > in > > > > > the > > > > > > > top > > > > > > > > > > level directory, and figure out the business logic for > > > calling > > > > > the > > > > > > > test > > > > > > > > > > suites from that file > > > > > > > > > > > > > > > > > > > > I look forward to your feedback. > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > > > > > > > > > > > Taher Alkhateeb > > > > > > > > > > > > > > > > > > > > On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb < > > > > > > > > > > slidingfilame...@gmail.com <javascript:;>> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hi Everyone, > > > > > > > > > > > > > > > > > > > > > > Thank you all for your support, JIRA created in > > > > > > > > > > > https://issues.apache.org/jira/browse/OFBIZ-7254 > > > > > > > > > > > > > > > > > > > > > > I will start working on it and try to implement ASAP to > > get > > > > my > > > > > > > focus > > > > > > > > > back > > > > > > > > > > > on refactoring. > > > > > > > > > > > > > > > > > > > > > > Cheers! > > > > > > > > > > > > > > > > > > > > > > Taher Alkhateeb > > > > > > > > > > > > > > > > > > > > > > On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit < > > > > > > > > > > > deepak.di...@hotwaxsystems.com <javascript:;>> wrote: > > > > > > > > > > > > > > > > > > > > > >> +1 > > > > > > > > > > >> > > > > > > > > > > >> Thanks & Regards > > > > > > > > > > >> -- > > > > > > > > > > >> Deepak Dixit > > > > > > > > > > >> www.hotwaxsystems.com > > > > > > > > > > >> > > > > > > > > > > >> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak < > > > > > > > > > > >> mridul.pat...@hotwaxsystems.com <javascript:;>> > wrote: > > > > > > > > > > >> > > > > > > > > > > >> > +1 > > > > > > > > > > >> > > > > > > > > > > > >> > Makes perfect sense. > > > > > > > > > > >> > > > > > > > > > > > >> > -- > > > > > > > > > > >> > Thanks & Regards, > > > > > > > > > > >> > Mridul Pathak > > > > > > > > > > >> > Senior Manager > > > > > > > > > > >> > HotWax Systems > > > > > > > > > > >> > http://www.hotwaxsystems.com > > > > > > > > > > >> > > > > > > > > > > > >> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb < > > > > > > > > > > >> slidingfilame...@gmail.com <javascript:;>> > > > > > > > > > > >> > wrote: > > > > > > > > > > >> > > > > > > > > > > > > >> > > Hello Everyone, > > > > > > > > > > >> > > > > > > > > > > > > >> > > After refactoring the start component and while > > > starting > > > > > on > > > > > > > the > > > > > > > > > base > > > > > > > > > > >> > > component I realized that the testing framework > for > > > > OFBiz > > > > > is > > > > > > > not > > > > > > > > > > good. > > > > > > > > > > >> > You > > > > > > > > > > >> > > cannot do real test driven development or > > > > > red-green-refactor > > > > > > > > with > > > > > > > > > > the > > > > > > > > > > >> > > current setup, hence my proposal to change it. I > > > explain > > > > > > > below: > > > > > > > > > > >> > > > > > > > > > > > > >> > > Problem with current design > > > > > > > > > > >> > > ---------------------------------------- > > > > > > > > > > >> > > - What we have right now is not unit tests, it's > > > really > > > > > > > > > integration > > > > > > > > > > >> > tests. > > > > > > > > > > >> > > You have to start the framework, the database, the > > > > service > > > > > > > > engine, > > > > > > > > > > the > > > > > > > > > > >> > > entity engine and pretty much everything. > > > > > > > > > > >> > > - Testing is very slow, because it's an > integration > > > test > > > > > as > > > > > > I > > > > > > > > > > >> mentioned > > > > > > > > > > >> > > above. 10 minutes on a good computer! > > > > > > > > > > >> > > - There is zero mocking! We actually have to > > > --load-data > > > > > for > > > > > > > > > things > > > > > > > > > > to > > > > > > > > > > >> > > work. Again, these are integration tests. > > > > > > > > > > >> > > - Too complex: Integration tests by their nature > are > > > > > > grabbing > > > > > > > > too > > > > > > > > > > >> much. > > > > > > > > > > >> > > Mind you, I am not objecting to integration tests > (I > > > > > > actually > > > > > > > > like > > > > > > > > > > >> them) > > > > > > > > > > >> > > but I am objecting to not having real unit-tests. > > Unit > > > > > tests > > > > > > > > > should > > > > > > > > > > >> all > > > > > > > > > > >> > run > > > > > > > > > > >> > > in a few seconds. > > > > > > > > > > >> > > > > > > > > > > > > >> > > Proposed solution > > > > > > > > > > >> > > -------------------------- > > > > > > > > > > >> > > - We keep what is considered real integration > tests > > > the > > > > > way > > > > > > > they > > > > > > > > > are > > > > > > > > > > >> > right > > > > > > > > > > >> > > now and keep using them > > > > > > > > > > >> > > - We move what should be unit tests into simple > > JUnit > > > > > > classes, > > > > > > > > and > > > > > > > > > > we > > > > > > > > > > >> do > > > > > > > > > > >> > > not run them using java -jar ofbiz.jar --test, but > > > > instead > > > > > > run > > > > > > > > > them > > > > > > > > > > >> > > directly from the build.xml script, so these files > > are > > > > not > > > > > > > > > > identified > > > > > > > > > > >> in > > > > > > > > > > >> > > any XML document, but are simply called > immediately > > > from > > > > > the > > > > > > > > build > > > > > > > > > > >> > scripts. > > > > > > > > > > >> > > - We clearly mark the difference between > integration > > > > tests > > > > > > and > > > > > > > > > unit > > > > > > > > > > >> tests > > > > > > > > > > >> > > (inside the source files or in the suite > > > declarations). > > > > > > > > > > >> > > - We change the run-tests target in build.xml to > run > > > > both > > > > > > unit > > > > > > > > > tests > > > > > > > > > > >> and > > > > > > > > > > >> > > integration tests. > > > > > > > > > > >> > > > > > > > > > > > > >> > > I intend to heavily refactor the framework and I > > would > > > > > feel > > > > > > > > better > > > > > > > > > > >> about > > > > > > > > > > >> > > introducing this change while refactoring. What do > > you > > > > > guys > > > > > > > > think? > > > > > > > > > > >> Ideas? > > > > > > > > > > >> > > Suggestions? Approvals and thumbs up? > > > > > > > > > > >> > > > > > > > > > > > > >> > > Regards, > > > > > > > > > > >> > > > > > > > > > > > > >> > > Taher Alkhateeb > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
