Here is requirement list for the $subject,
*Basic things that will be in $subject ( jaggery test framework)*
- Testing Jaggery (.jag) API with function levels (simple as unit test)
- online
- Jaggery module testing under application (app/{appName}/module)
- Changing to any test case then only the test case file needs to be
updated [Script will remain the same]
- Application-independent. Test resource/spec of test will be inside the
application (app/{appName}/test)
- Easy to expand, maintain, and perpetuate. (Try to follow same JS frame
work structure such as *jasmine[1] *)
- Supportable with automation test framework with some configure
changing/pom/ant
- Should have low cost on maintaining
- Easy to write test (less time)
- REST/JSON report Supporting (Console commanding not implemented, As
above discussion)
- HTML Test coverage reporting
*Scopes for Framework*
Application-oriented Framework, Test resources for application will be
inside application
*Test framework type*
It will be 'Hybrid' as It will be following (Script Modularity Framework,
Library Architecture Framework and Structured Framework will be used in
implementation)
*Responsibilities of $subject*
- Defining the format in which to express expectations
- Running the tests which exercise the public interface (REST interfaces)
- Result with interface of JSON
- Interact with platform testing and build time with simple interface
(interface for REST client)
- Calling test suites and skipping test suit with changing test spec
(Test case js/jag )
- Executing the tests (Without console commanding)
- Reporting results of the test HTML interface also
If there I am missing any requirement for $subject, pls add it here,
Here is some more we will consider before writing $subject[2], Some trying
out existing JS test framework[3] for $subject for matter for comparison
and understating design on existing system.
[1] http://pivotal.github.io/jasmine/
[2]
http://madhukaudantha.blogspot.com/2013/11/what-to-be-know-about-test-framework.html
[3] http://madhukaudantha.blogspot.com/search/label/Test%20Framework
Thanks
On Fri, Nov 15, 2013 at 10:57 AM, Madhuka Udantha <madh...@wso2.com> wrote:
> On Fri, Nov 15, 2013 at 10:30 AM, Krishantha Samaraweera <
> krishan...@wso2.com> wrote:
>
>> Functinalities like Server startup/shutdown, coverage generation and
>> result generation already implemented in WSO2 Test Automation framework. So
>> we need to reuse those functionalities in here.
>>
> Yap, sure we will be reusing those.
>
>
>> Having a Rest API to call tests scripts embedded in each jaggery app
>> would be nice addition. In that way we could easily integrate it with our
>> existing framework and use though out the platform.
>>
>> And this framework should support for continuous integration. So
>> everything have to be executed without human interactions. So avoiding
>> command line test runners etc.. is required.
>>
>
> We will have json/RESTful API for app developer to run the test rather
> command line, (No command line (cmd/ jaggery console)). Therefore there
> will be html (web interface for app developer to run his/her test in
> easily). We will avoid command line testing and will provide web interface
> testing.
>
> Thanks
>
>>
>> Thanks,
>> Krishantha
>>
>>
>> On Thu, Nov 14, 2013 at 11:02 PM, Madhuka Udantha <madh...@wso2.com>wrote:
>>
>>>
>>>
>>>
>>> On Thu, Nov 14, 2013 at 5:14 PM, Udara Rathnayake <uda...@wso2.com>wrote:
>>>
>>>> Hi all,
>>>>
>>>> While designing the test framework, its good if we can keep in mind
>>>> that jaggery2 product also underway.
>>>> Any thoughts?
>>>>
>>> Sure, will think.
>>> Mainly test framework will coming for javascript(jag/js) testing so it
>>> will be written where can be extended.
>>>
>>>>
>>>> Regards,
>>>> UdaraR
>>>>
>>>>
>>>> On Thu, Nov 14, 2013 at 10:15 AM, Madhuka Udantha <madh...@wso2.com>wrote:
>>>>
>>>>> Hi Krishantha and test automation team,
>>>>>
>>>>> I would like to get your opinion on this task regard to
>>>>>
>>>>> is there standard methodology on current testing framework where we
>>>>> can added such testing framework or better to go on fully independent
>>>>> model?
>>>>>
>>>>>
>>>>> My thinking is,
>>>>>
>>>>> - As it is application level testing frame work, better to work
>>>>> with much independent of the current system
>>>>> - Where application writer (developer) can run test for specific
>>>>> application rather running all test for platform or product
>>>>> - It will help to maintain both test framework with low cost
>>>>> - Each test frame have it's own scope (such as
>>>>> platform/product/appilcation). if not, complexity of the code and
>>>>> system
>>>>> will be increase but there will be API interface for interaction
>>>>> among the
>>>>> test framework.
>>>>> - And I thought to have interface (simple REST interface) in
>>>>> jaggery test frame work where current test framework also can interact
>>>>> with
>>>>> it into some extend.
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>>> On Tue, Nov 12, 2013 at 5:57 PM, Madhuka Udantha <madh...@wso2.com>wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>>
>>>>>> Still we do not have jaggery test framework even though there are
>>>>>> considerable jaggery applications were developed. It is not easy maintain
>>>>>> those jaggery applications/APIs steady without testing frame work.
>>>>>>
>>>>>>
>>>>>> Now onward we will starting $subject as it is important for jaggery
>>>>>> as well as there is no JS testing framework under WSO2 Platform. Testing
>>>>>> framework will be under application level such as mocha, jasmine. Test
>>>>>> frame will be apply to (basically)test.
>>>>>>
>>>>>>
>>>>>>
>>>>>> - jaggery API (js functions and arguments )
>>>>>> - REST APIs that are expose under jaggery application
>>>>>> - With test coverage reporting
>>>>>>
>>>>>>
>>>>>> We will be evaluating existing JavaScript test frame works regard
>>>>>> mocha[1], jasmine[2], Qunit[3], Vows[4], expresso[5], nodeunit[6] and
>>>>>> jsunit[7] to $subject.
>>>>>>
>>>>>>
>>>>>>
>>>>>> All suggestions are welcome.
>>>>>>
>>>>>>
>>>>>> [1] http://visionmedia.github.io/mocha/
>>>>>>
>>>>>> [2] http://pivotal.github.io/jasmine/
>>>>>>
>>>>>> [3] http://qunitjs.com/
>>>>>>
>>>>>> [4] http://vowsjs.org/
>>>>>>
>>>>>> [5] http://visionmedia.github.io/expresso/
>>>>>>
>>>>>> [6] https://github.com/caolan/nodeunit
>>>>>>
>>>>>> [7] https://github.com/pivotal/jsunit
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks
>>>>>> --
>>>>>> *Madhuka* Udantha
>>>>>> Senior Software Engineer
>>>>>> Development Technologies
>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>
>>>>>> *Mobile*: +94774066336
>>>>>> *Blog*: http://madhukaudantha.blogspot.com/
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Madhuka* Udantha
>>>>> Senior Software Engineer
>>>>> Development Technologies
>>>>> WSO2 Inc. : http://wso2.com
>>>>>
>>>>> *Mobile*: +94774066336
>>>>> *Blog*: http://madhukaudantha.blogspot.com/
>>>>>
>>>>> _______________________________________________
>>>>> Dev mailing list
>>>>> Dev@wso2.org
>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Udara Rathnayake*
>>>> Software Engineer
>>>> WSO2 Inc. : http://wso2.com
>>>>
>>>> Mobile : +94 772207239
>>>> Twitter : http://twitter.com/udarakr
>>>> Blog : http://udarakr.blogspot.com
>>>>
>>>>
>>>
>>>
>>> --
>>> *Madhuka* Udantha
>>> Senior Software Engineer
>>> Development Technologies
>>> WSO2 Inc. : http://wso2.com
>>>
>>> *Mobile*: +94774066336
>>> *Blog*: http://madhukaudantha.blogspot.com/
>>>
>>
>>
>>
>> --
>> Krishantha Samaraweera
>> Senior Technical Lead - Test Automation
>> Mobile: +94 77 7759918
>> WSO2, Inc.; http://wso2.com/
>> lean . enterprise . middlewear.
>>
>
>
>
> --
> *Madhuka* Udantha
> Senior Software Engineer
> Development Technologies
> WSO2 Inc. : http://wso2.com
>
> *Mobile*: +94774066336
> *Blog*: http://madhukaudantha.blogspot.com/
>
--
*Madhuka* Udantha
Senior Software Engineer
Development Technologies
WSO2 Inc. : http://wso2.com
*Mobile*: +94774066336
*Blog*: http://madhukaudantha.blogspot.com/
_______________________________________________
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
