Re: [action1] Improving our tests -- long [was Re: Standalone Tiles: Status and Call for Help]
If infrastructure would approve my request to run Jive on an ASF server, we wouldn't be having this discussion :) In the meantime, you can watch the WebWork developer forum, which contains the chat transcripts posted each night: http://forums.opensymphony.com/watches!add.jspa?forumID=32 Don Martin Cooper wrote: On 4/28/06, James Mitchell [EMAIL PROTECTED] wrote: So, after a long winded discussion on the WW Chat line(no need to rehash, it's in your inbox -- or will be)... I've seen people make this type of comment before, but I have yet to see any of these chat transcripts show up in my mailbox. I'd appreciate it if someone could fix that, because right now, there are development discussions going on that are not visible to everyone on this list, which is not good. -- Martin Cooper several ideas were mentioned, and so I thought I'd throw it out there for comment. Patrick has graciously offered to provide the Struts community with free beer and back rubs woo hooo!!! - ok, just kidding about the beer. Patrick has offered us free resources for automated testing. I'll let Patrick explain in greater detail as I'm not that familiar with what he has to offer and I don't want to misquote. The discussion (as you've read from the chat) originally started just as chatter about the below plugin (TestGen4Web) and what we could do with it to help with automated testing. There is plenty of documentation out there for TestGen4Web, but let me explain how this thing works and where I think we can leverage it for our own needs. TestGen4Web is a FireFox plugin that let's you record a session (as mentioned below). Once you push record, everything you do is remembered and you can save the file locally right from the toolbar. You can even load an existing script and play it. If you have downloaded the translator, you can generate an HttpUnit java source from the saved .xml file that implements everything you did with the mouse and keyboard that is a ready to run HttpUnit test class. For example, this is what is generated when I recorded a session for the struts-blank application: ?xml version='1.0' encoding='UTF-8'? tg4w version=0.38.1 actions action type=goto refresh=true step=0 xpath![CDATA[window.location.href]]/xpath value![CDATA[http://localhost:8080/struts-blank/]]/ value /action action type=verify-title refresh=true step=1 xpath![CDATA[*]]/xpath value![CDATA[Struts Blank Application]]/value /action action type=assert-text-exists step=2 xpath![CDATA[*]]/xpath value![CDATA[Welcome!]]/value /action /actions /tg4w You can see in step 0 I hit the url for the struts-blank app. Then in step 1 the title is verified and step 2 verifies that the text Welcome! is in the page. I did step 2 (assert-text-exists) by highlighting the Welcome! text and right click - Assert - Verify Text. The same .xml file can also be used to generate a Selenium configuration xml file that does the same test. Currently there isn't a Maven 2 plugin for the process of translating from xml to java and that's where I come in. What comes with TestGen4Web is an ant build that does the job. I'm currently working on a new Maven 2 plugin that handles this and it should be finished by the time you read this. Configuring the test to run as part of the build will be the tricky (Wendy? help?!?!) part. With this plugin, you won't have to download the translator. Like most things in Maven, you aren't forced into a directory structure, but for this purpose I propose we do this: struts + action + apps + blank + cookbook + ... + ... + TestGen4Web + src + main + scripts - new (.xml) scripts go here + target + gensrc - scripts get gen'd to here (.java) + classes- the plugin would compile the test to here (Just to be clear, the only thing we should be keeping in svn is the xml scripts) This plugin will generate both HttpUnit (.java) and Selenium (.xml) sources, and this would is where Patrick's process could pick up the Selenium config. And so, going forward with this plugin, all we would need to do is create a new script (e.g. blank-validate-welcome.xml) and put it under TestGen4Web/src/main/scripts and you are done. Obviously we would want to be consistent with script naming, personally I would prefer that they be verbose and in the event that a new script is needed to verify a specific bug, the bug number might also be in the name. One other option we might consider is keeping a README.txt in that directory if any further details are needed. These scripts would greatly speed up the release process since it would basically do the equivalent of about 2 or more hours of what used to be done manually. However, the fun doesn't stop here. I'm not sure how clear
Re: [action1] Improving our tests -- long [was Re: Standalone Tiles: Status and Call for Help]
On 4/28/06, James Mitchell [EMAIL PROTECTED] wrote: So, after a long winded discussion on the WW Chat line(no need to rehash, it's in your inbox -- or will be)... I've seen people make this type of comment before, but I have yet to see any of these chat transcripts show up in my mailbox. I'd appreciate it if someone could fix that, because right now, there are development discussions going on that are not visible to everyone on this list, which is not good. -- Martin Cooper several ideas were mentioned, and so I thought I'd throw it out there for comment. Patrick has graciously offered to provide the Struts community with free beer and back rubs woo hooo!!! - ok, just kidding about the beer. Patrick has offered us free resources for automated testing. I'll let Patrick explain in greater detail as I'm not that familiar with what he has to offer and I don't want to misquote. The discussion (as you've read from the chat) originally started just as chatter about the below plugin (TestGen4Web) and what we could do with it to help with automated testing. There is plenty of documentation out there for TestGen4Web, but let me explain how this thing works and where I think we can leverage it for our own needs. TestGen4Web is a FireFox plugin that let's you record a session (as mentioned below). Once you push record, everything you do is remembered and you can save the file locally right from the toolbar. You can even load an existing script and play it. If you have downloaded the translator, you can generate an HttpUnit java source from the saved .xml file that implements everything you did with the mouse and keyboard that is a ready to run HttpUnit test class. For example, this is what is generated when I recorded a session for the struts-blank application: ?xml version='1.0' encoding='UTF-8'? tg4w version=0.38.1 actions action type=goto refresh=true step=0 xpath![CDATA[window.location.href]]/xpath value![CDATA[http://localhost:8080/struts-blank/]]/ value /action action type=verify-title refresh=true step=1 xpath![CDATA[*]]/xpath value![CDATA[Struts Blank Application]]/value /action action type=assert-text-exists step=2 xpath![CDATA[*]]/xpath value![CDATA[Welcome!]]/value /action /actions /tg4w You can see in step 0 I hit the url for the struts-blank app. Then in step 1 the title is verified and step 2 verifies that the text Welcome! is in the page. I did step 2 (assert-text-exists) by highlighting the Welcome! text and right click - Assert - Verify Text. The same .xml file can also be used to generate a Selenium configuration xml file that does the same test. Currently there isn't a Maven 2 plugin for the process of translating from xml to java and that's where I come in. What comes with TestGen4Web is an ant build that does the job. I'm currently working on a new Maven 2 plugin that handles this and it should be finished by the time you read this. Configuring the test to run as part of the build will be the tricky (Wendy? help?!?!) part. With this plugin, you won't have to download the translator. Like most things in Maven, you aren't forced into a directory structure, but for this purpose I propose we do this: struts + action + apps + blank + cookbook + ... + ... + TestGen4Web + src + main + scripts - new (.xml) scripts go here + target + gensrc - scripts get gen'd to here (.java) + classes- the plugin would compile the test to here (Just to be clear, the only thing we should be keeping in svn is the xml scripts) This plugin will generate both HttpUnit (.java) and Selenium (.xml) sources, and this would is where Patrick's process could pick up the Selenium config. And so, going forward with this plugin, all we would need to do is create a new script (e.g. blank-validate-welcome.xml) and put it under TestGen4Web/src/main/scripts and you are done. Obviously we would want to be consistent with script naming, personally I would prefer that they be verbose and in the event that a new script is needed to verify a specific bug, the bug number might also be in the name. One other option we might consider is keeping a README.txt in that directory if any further details are needed. These scripts would greatly speed up the release process since it would basically do the equivalent of about 2 or more hours of what used to be done manually. However, the fun doesn't stop here. I'm not sure how clear I was in the chat log about my idea for a linear set of pages to test the taglibs, but what I had in mind would be to extend the cookbook and provide many more examples that demonstrate the various ways some tags can be used. I'm not sure if we want to go the same route as the taglib tests by providing a test for every
[action1] Improving our tests -- long [was Re: Standalone Tiles: Status and Call for Help]
So, after a long winded discussion on the WW Chat line(no need to rehash, it's in your inbox -- or will be)... several ideas were mentioned, and so I thought I'd throw it out there for comment. Patrick has graciously offered to provide the Struts community with free beer and back rubs woo hooo!!! - ok, just kidding about the beer. Patrick has offered us free resources for automated testing. I'll let Patrick explain in greater detail as I'm not that familiar with what he has to offer and I don't want to misquote. The discussion (as you've read from the chat) originally started just as chatter about the below plugin (TestGen4Web) and what we could do with it to help with automated testing. There is plenty of documentation out there for TestGen4Web, but let me explain how this thing works and where I think we can leverage it for our own needs. TestGen4Web is a FireFox plugin that let's you record a session (as mentioned below). Once you push record, everything you do is remembered and you can save the file locally right from the toolbar. You can even load an existing script and play it. If you have downloaded the translator, you can generate an HttpUnit java source from the saved .xml file that implements everything you did with the mouse and keyboard that is a ready to run HttpUnit test class. For example, this is what is generated when I recorded a session for the struts-blank application: ?xml version='1.0' encoding='UTF-8'? tg4w version=0.38.1 actions action type=goto refresh=true step=0 xpath![CDATA[window.location.href]]/xpath value![CDATA[http://localhost:8080/struts-blank/]]/ value /action action type=verify-title refresh=true step=1 xpath![CDATA[*]]/xpath value![CDATA[Struts Blank Application]]/value /action action type=assert-text-exists step=2 xpath![CDATA[*]]/xpath value![CDATA[Welcome!]]/value /action /actions /tg4w You can see in step 0 I hit the url for the struts-blank app. Then in step 1 the title is verified and step 2 verifies that the text Welcome! is in the page. I did step 2 (assert-text-exists) by highlighting the Welcome! text and right click - Assert - Verify Text. The same .xml file can also be used to generate a Selenium configuration xml file that does the same test. Currently there isn't a Maven 2 plugin for the process of translating from xml to java and that's where I come in. What comes with TestGen4Web is an ant build that does the job. I'm currently working on a new Maven 2 plugin that handles this and it should be finished by the time you read this. Configuring the test to run as part of the build will be the tricky (Wendy? help?!?!) part. With this plugin, you won't have to download the translator. Like most things in Maven, you aren't forced into a directory structure, but for this purpose I propose we do this: struts + action + apps + blank + cookbook + ... + ... + TestGen4Web + src + main + scripts - new (.xml) scripts go here + target + gensrc - scripts get gen'd to here (.java) + classes- the plugin would compile the test to here (Just to be clear, the only thing we should be keeping in svn is the xml scripts) This plugin will generate both HttpUnit (.java) and Selenium (.xml) sources, and this would is where Patrick's process could pick up the Selenium config. And so, going forward with this plugin, all we would need to do is create a new script (e.g. blank-validate-welcome.xml) and put it under TestGen4Web/src/main/scripts and you are done. Obviously we would want to be consistent with script naming, personally I would prefer that they be verbose and in the event that a new script is needed to verify a specific bug, the bug number might also be in the name. One other option we might consider is keeping a README.txt in that directory if any further details are needed. These scripts would greatly speed up the release process since it would basically do the equivalent of about 2 or more hours of what used to be done manually. However, the fun doesn't stop here. I'm not sure how clear I was in the chat log about my idea for a linear set of pages to test the taglibs, but what I had in mind would be to extend the cookbook and provide many more examples that demonstrate the various ways some tags can be used. I'm not sure if we want to go the same route as the taglib tests by providing a test for every possible combination of attributes for every tag and some with multiple scenarios for tags that supported different ways to provide values for a given attribute. Whatever the case, I'm excited about being able to add new tests that are also directly adding to the demo apps / tutorials.
Re: [action1] Improving our tests -- long [was Re: Standalone Tiles: Status and Call for Help]
One thing that I might have blurred the lines on is that there are basically 2 things being proposed here with respect to TestGen4Web. #1 - The ability to easily create test scripts for cases that can only be proven and solved via a deployed app (and without cactus) #2 - A set of scripts (that happen to be executed just like #1, yet serve a little different purpose) that recursively run through the cookbook links and assert success on the page after every click. While the existing cookbook pages usually offer some kind of interactivity (enter some data and see it work differently for different values), the new pages that demonstrate the taglibs (or test them, depending on your purpose) would be required to show some kind of success message that can be asserted. That value should not appear if the test fails. The success message will need to be consistent if the tests are to be recursive. Meaning that adding new tests (recipes) to an existing page (chapter) would only require that a new description and link be added and when clicked on will end up showing a page where the text could be asserted. Again, that applies to the pages that test (and demonstrate) taglibs. So, to mentally reorg the cookbook. Instead of this: -- Simple Form using ActionForm Execute - link that gets clicked and tested Simple Form using DynaActionForm Execute - same Select and Options tags Execute - same ... (and on down the page) ... -- We might have a Table of contents as an index that links to pages like the one above. If no one objects, I'll proceed on it. -- James Mitchell On Apr 28, 2006, at 11:36 PM, James Mitchell wrote: So, after a long winded discussion on the WW Chat line(no need to rehash, it's in your inbox -- or will be)... several ideas were mentioned, and so I thought I'd throw it out there for comment. Patrick has graciously offered to provide the Struts community with free beer and back rubs woo hooo!!! - ok, just kidding about the beer. Patrick has offered us free resources for automated testing. I'll let Patrick explain in greater detail as I'm not that familiar with what he has to offer and I don't want to misquote. The discussion (as you've read from the chat) originally started just as chatter about the below plugin (TestGen4Web) and what we could do with it to help with automated testing. There is plenty of documentation out there for TestGen4Web, but let me explain how this thing works and where I think we can leverage it for our own needs. TestGen4Web is a FireFox plugin that let's you record a session (as mentioned below). Once you push record, everything you do is remembered and you can save the file locally right from the toolbar. You can even load an existing script and play it. If you have downloaded the translator, you can generate an HttpUnit java source from the saved .xml file that implements everything you did with the mouse and keyboard that is a ready to run HttpUnit test class. For example, this is what is generated when I recorded a session for the struts-blank application: ?xml version='1.0' encoding='UTF-8'? tg4w version=0.38.1 actions action type=goto refresh=true step=0 xpath![CDATA[window.location.href]]/xpath value![CDATA[http://localhost:8080/struts-blank/]]/ value /action action type=verify-title refresh=true step=1 xpath![CDATA[*]]/xpath value![CDATA[Struts Blank Application]]/value /action action type=assert-text-exists step=2 xpath![CDATA[*]]/xpath value![CDATA[Welcome!]]/value /action /actions /tg4w You can see in step 0 I hit the url for the struts-blank app. Then in step 1 the title is verified and step 2 verifies that the text Welcome! is in the page. I did step 2 (assert-text-exists) by highlighting the Welcome! text and right click - Assert - Verify Text. The same .xml file can also be used to generate a Selenium configuration xml file that does the same test. Currently there isn't a Maven 2 plugin for the process of translating from xml to java and that's where I come in. What comes with TestGen4Web is an ant build that does the job. I'm currently working on a new Maven 2 plugin that handles this and it should be finished by the time you read this. Configuring the test to run as part of the build will be the tricky (Wendy? help?!?!) part. With this plugin, you won't have to download the translator. Like most things in Maven, you aren't forced into a directory structure, but for this purpose I propose we do this: struts + action + apps + blank + cookbook + ... + ... + TestGen4Web + src + main + scripts - new
