Good point. Some additional comments from my side inline
On Wed, Jan 4, 2017 at 6:47 PM Ashish Kumar <[email protected]> wrote: Hello Everyone! I think we have two different approaches here, each approach depends on our definition of unit, what exactly should be our unit to test should it be method-based or functionality based (group of methods defining a functionality). As Serena mentioned, In case of functionality based, when a test would fail... we will easily know which functionality is broken but functionality itself is a pipeline of methods right (correct me if I am wrong)? Are you referring to methods of the same modules or different modules? In this pipeline, we won't know which condition in which method is exactly breaking the test. As per my understanding unit tests should be written at the granularity level and methods are more granular than functionality. Also we have to make a deal with the coverage rate in case of functionality testing. Taking given example in case of method based approach: cli_base.env() and cli_env.prepare(): If we will go method based approach then one unit test for cli_base.env() make sense and depending on the behaviour (considering branches/exception etc.) of cli_env.prepare() multiple tests for the same method could be there ( currently two written: https://gerrit.opnfv.org/gerrit/#/c/26143/13/functest/tests/unit/cli/commands/test_cli_env.py ). I agree we won't have high level overview about the functionality but wondering if that is the motive of writing unit tests but we would have all the possible breaking points covered in each functionality. IMHO, it might be overkilled to cover all possible breaking points. There is a *tradeoff* between comprehensive testing and the workloads required to implement them and keep them up to date. To me, the public interface is the contract given by the module and should be verified. The internal details such as private methods are likely being refactored during development. When a failure is detected, developers may leverage other ways besides unit test to diagnose it. Common ways for this are: 1. log messages in a production system 2. debugging tools in development environment. Benefit of taking method based approach is: 1. we will know the exact method/condition where the issue is 2. There would be separation (non-overlapping) among the tests i.e. ideally unit tests should have 3. We can achieve good code coverage 4. Our test would be much more modular and granular. Also, I have prepared some slides mentioning some points I felt and explored about unit tests. I have make the link editable so that anyone with the link can edit, please correct me if I am wrong somewhere or add more points if needed. Link: https://docs.google.com/presentation/d/1vFVLP1SRPaKL3b90wTlc6NuE6zyf5Q4FhbxcN12W4IA/edit?usp=sharing Thanks On Wed, Jan 4, 2017 at 12:12 PM, Yujun Zhang <[email protected]> wrote: Serena, Maybe the term `interface` fits better here. I agree with you that the target of unit test in Python is *module*. We ensure with unit test the existing interfaces of the module are not broken by new code and new interfaces implemented work as expected. The private methods (which does not expose anything of the module) is not something promised by the module and we don't need to create dedicated unit test for them. They are normally covered by the unit test of related public methods. My another two cents... On Wed, Jan 4, 2017 at 2:04 PM <[email protected]> wrote: Sorry, I didn't express my idea clearly enough during the meeting. When I say function-based, it is not equal to functional test. Unlike functional test, it is still a kind of white-box test. And compared to method-based unit test, which consider the method as a test object, it treats the module as an unit, it guarantees the module works as expected after new requirements come or any refactor requires. Sorry for my poor expression, and let me take 'functest env prepare' of cli module in Functest for example(where the discussion comes from). Instead of testing cli_base.env() and cli_env.prepare() separately, we can test them together by writing the following four test cases(naming is not very good just for description purpose): test_cli_env_first_time_prepare() test_cli_env_re_prepare() test_cli_env_not_prepare_again() test_cli_env_invalid_prepare_answer() In this way, all the code can be covered as method-based way do, and at the same time when something changed in cli_env.prepare() and if it doesn't break the functionality it should provide, we don't need to rewrite the unit test, and on the contrary, if it breaks, the unit test will detect it and notify us with a failure, and of course, if the functionality needs to change, we should rewrite the unit test. If unit test is written in the way of method-based, almost everytime the code change will cause the rewriting of the test code, and further more whether it breaks the original functionality, we cannot tell. BRs Serena -- Ashish Kumar IIIT Hyderabad +91 9581528645 <+91%2095815%2028645>
_______________________________________________ opnfv-tech-discuss mailing list [email protected] https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
