Hi Bob, Thanks for the feedbacks written in your busy time :-). But I have a small comments on Selenium. Please read below.
On Thu, Jun 3, 2010 at 5:17 PM, Bob Jolliffe <bobjolli...@gmail.com> wrote: > Hi > > On 3 June 2010 10:50, Ngoc Thanh Nguyen <thanh.hispviet...@gmail.com> wrote: >> Hi Bob, >> >> Thanks for the comments. I have some replies below. >> >> On Thu, Jun 3, 2010 at 3:40 PM, Bob Jolliffe <bobjolli...@gmail.com> wrote: >>> Hi Thanh >>> >>> We did receive it and i had a quick read through. Its great to see >>> work being done on testing. we need lots of it. >>> >>> Two comments/questions (to prove that I read it!): >>> 1. I am not entirely convinced by your replacement of TSL with Excel. >>> TSL is a formal grammar. It has limitations which have been >>> described elsewhere but I don't think that the typing of '[' has ever >>> really featured highly :-) I think you may be confusing the >>> specification with the authoring tool when you say that your revised >>> TSL is to "use excel". So on the theory side I'd be a bit >>> uncomfortable with this aspect of your paper. >>> (BTW did you look at schematron as an alternative grammar for >>> expressing tests? I know one of the criticisms of TSL is that it does >>> not allow for a very literate human-readable approach. schematron >>> allows for a human readable assertions and - though xpath is usually >>> used for schema validation - tests can be implemented in any language. >>> Perhaps an off-the wall thought ...) >> >> TSL in my paper is about the specification developed by Ostrand 1988. >> They developed this specification within the method they called >> Category Partition - a combination between Equivalence Partition and >> Boundary Analysis. This specification as I feel is a bit complicated >> because it required typing [ or ] character. My revision aims to save >> time for typing this and also make it readable for a computer program. >> I selected Excel because of its popularity and easy-to-use. >> > > Yes I understand all that. So you have selected excel as a tool for > authoring your test cases. Thats fine and I'm sure its a good tool > for the job for the reasons you describe. The point is that TSL is a > language for expressing them. What do you express? An "excel file". > I really don't think thats the the same thing at all. I guess what is > missing is some way to describe those spreadsheets of yours so that > they can be considered as a sort of language like TSL. > >> CPM is the most popular techniques in black box testing. Cause graph >> effect is another method but it requires to draw graphical diagram so >> it seems impossible in practice. >> >> XPath or Schematron seems to be out of the scope of CPM. > > Not at all. See > http://www.computer.org/portal/web/csdl/doi/10.1109/ICSEA.2006.9 for > example. > > I agree XPath is well out of scope, but schematron is not bound to use > XPath. Just so happens that 99% of the time it does because it is > primarily used for testing xml validity. But I guess there is nothing > in principle preventing the actual tests to be implemented as say > javascript, or even selenium scripts. > >> >>> 2. I am not very clear - perhaps didn't read carefully enough - how >>> you generate the actual tests from the source document. It looks to >>> me that it requires quite a significant understanding of the >>> application , the classes etc. I guess this is where understanding >>> the grammar, if any, is important. Could you generate selenium tests >>> from it? This strikes me as the most black-box kind of testing. >> >> No, CPM can not generate actual test cases and test data. It can >> generate test frame (a combination of different input values). Tester >> has to provide the test data, hence, to build test case. I find it >> hard to have any automation tool to generate test cases (with test >> data) automatically. And my tool just does a simple job: 1) read the >> Excel file containing category and partition, 2) combination based on >> their properties and constraints, 3) and build the test frames based >> on the combination. > > OK. Now I understand. >> >>> How easy would it be to use your tool to do wider test coverage of >>> dhis? Would be a good thing. >> >> Here we come with the test coverage. Black box is different from white >> box. It can not produce any code coverage (statement, block, branch, >> predicate etc). Not at all. It can not be used with mutation. It seems >> there is no way to assess whether a test set is adequate - good >> enough. Any one have better idea can correct me on this. >> >> But I am very confident that if we follow the CPM strictly, together >> with the support from my tool, i.e. revise the category and partition >> building back and forth, we can end up with a 100% test coverage. > > Sounds good. > >> >> The challenge is, for web layer of DHIS2, we could not apply white box >> testing. This is partial reason why there is no unit testing for >> action class. It is so difficult to mock all the session and >> interceptor at least in DHIS2. We have only one choice, that is black >> box testing. And so far, we have not done any systematic testing on >> web modules. Our current approach is more ad-hoc. > > This is where I was suggesting something like selenium might be > useful. Does anybody know of any formal testing language for web > applications? I am sure there is bound to be something. I have tried Selenium, indeed, I have built several test cases on Selenium and it run quite well. I also tried sahi, another web automation test tool. These tool help tester to record their tests and save in different format, hence they can be run later without manual re-running. HOWEVER, what to be recoded in Selenium is really a question. And CPM comes to fulfill this space. So I recommend the following process: building test specification (category, partition, properties, constrainst) --> use the tool to generate the test frame (or test case if you want to call) --> record these test cases using Selenium --> enjoy the automation forever. Thanh > Thanh, thanks for sharing this. Now I must get back to work ... bloody > spring. > > Regards > Bob > >> >> More input needed. >> Thanh >> >>> Cheers >>> Bob >>> >>> On 2 June 2010 12:03, Ngoc Thanh Nguyen <thanh.hispviet...@gmail.com> wrote: >>>> Hi all, >>>> >>>> this is a project I did for the testing course this semester. I hope >>>> it could shed a light for testing aspect of dhis2. >>>> >>>> Notice: the case was built based on the assumption that the system >>>> specification requires name of orgunit, group, and group set to be >>>> have between 2-255 characters. Though dhis2 implemented it with text >>>> field max char = 160. >>>> >>>> Thanh >>>> >>>> _______________________________________________ >>>> Mailing list: https://launchpad.net/~dhis2-devs >>>> Post to : dhis2-devs@lists.launchpad.net >>>> Unsubscribe : https://launchpad.net/~dhis2-devs >>>> More help : https://help.launchpad.net/ListHelp >>>> >>>> >>> >> > _______________________________________________ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
