Re: [Wonder-disc] ERSelenium without ERExtensions
On Jun 30, 2007, at 3:08 PM, Michael Bushkov wrote: How do you handle "clicking" on component action links? We try to use "link" locators and simple xpath where possible: |clickAndWait|link=Return Hat|| and |clickAndWait|//[EMAIL PROTECTED]'Next']|| Here is also the ugly, but powerful example from the BugTracker's set of tests (I guess it's the only way there to emulate clicking on the 'Comment' link in the '__SeleniumBugSubject' bug's row): |clickAndWait|//td[contains(text(), '__SeleniumBugSubject')]/ parent::*/preceding-sibling::tr[position() = 1]//a[text() = 'Comment']|| Here is a somewhat cleaner example of xpath which will locate a link in the cell of the row with '__SeleniumBugSubject' text in it (we assume here that there only one link in a row): //tr[contains(string(), '__SeleniumBugSubject')]/td/a Not that ugly, huh? Xpath is not as clean as id locator, but it is pretty powerful. I tend to think that it is a better choice for developer than applying ids to all the needed elements. -- Denis ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Re: [Wonder-disc] ERSelenium without ERExtensions
Hi Chuck, Chuck Hill wrote: The advantage of this approach is that you don't need to do test source code postprocessing. The same source code will be equally processed in selenium-ide and on any host where ERSelenium is used. Besides Selenium-RC looks like a kind of overkill solution to us, so we try to avoid it and use only selenium-core, which is much more simple and easier to install and use. Why does it look like overkill to you? It requires installation of the Selenium-RC server on the test platform. This means that I can't run the test from the dozen of different browsers on the dozen of different platforms - even If I want to, I'd need to install selenium-rc everywhere. Selenium-rc allows the tests to be actually written in the real programming language. I'm sure that most of the test cases don't use any flexibility allowed by generic programming language. Selenium-rc allows a workaround over same-origin policy. But while developing WO app you actually don't have to search for such a workaround - because as you need to host the app somewhere, you can easily host the selenium-core in the same place. So, while allowing the set of features, mostly equal to the selenium-core, selenium-rc introduces a new layer of complexity above it. Of course the ERSelenium's approach (mostly similar to selenium-on-rails) introduces it too, but it's idea is to rely solely on selenium-core while adding some tests-sources-processing features and handling testing results. And I'm not an XPath admirer, but it can sometimes be extremely useful: |select|xpath=//label[contains(string(),'Payment type')]/following-sibling::div/select|By cash*| This will select "By cash" option in the listbox next to the "Payment type" label. But that assumes that you are using the HTML label tag. In my experience that is not a commonly used tag. So you end up either having to retrofit label tags or IDs in existing applications (or using really hard to read Xpath expressions). Label is in fact a semantic W3C recommended way of dealing with form labels, but anyway this is a specific test for a specific website. As 'label' tag was used in that website, it's ok (IMHO) to use it in the test. I mean that each test should be written individually for the project that it aims to test. So if that website used another 'div' instead of the label tag, then we'd write the test as follows: |select|xpath=//div[contains(string(),'Payment type')]/following-sibling::div/select|By cash*| How do you handle "clicking" on component action links? We try to use "link" locators and simple xpath where possible: |clickAndWait|link=Return Hat|| and |clickAndWait|//[EMAIL PROTECTED]'Next']|| Here is also the ugly, but powerful example from the BugTracker's set of tests (I guess it's the only way there to emulate clicking on the 'Comment' link in the '__SeleniumBugSubject' bug's row): |clickAndWait|//td[contains(text(), '__SeleniumBugSubject')]/parent::*/preceding-sibling::tr[position() = 1]//a[text() = 'Comment']|| Such kind of xpath is actually common when testing D2WList-driven pages with no additional code for generating ID attribute for links. In my tests, I often have a series of 3 - 7 tests that need to be run in order. I also have larger "groups" where the tests in each group can be run in any order but the groups have an order (e.g. need to setup the users before running the user login tests). To do this kind of thing we have: - A separate test, that tests the whole process of user registration - An action method that ensures that the 'test' user is registered - The test that tests the login procedure that uses the above action method to ensure that the needed user exists - A setup method that ensures that the 'test' user is registered and already logged in. All tests that need the 'test' user to be logged in use this method. With this approach each test checks some independent bit of application's logic and can be run without dependency on other tests. We used approach similar to the one that you describe, but later decided that the required ordering of tests is driven by the lack of setup methods. Besides, look at the beginning of the typical test: |open|/wa/SeleniumAction/resetSession|| |assertTextPresent|Action command succeeded.|| |open|/wa/SeleniumAction/ensureTestAdmin|| |assertTextPresent|Action command succeeded.|| These lines ensure that the user with admin rights is registered in the system. With this style of writing test, not only they become independent, but their sources become self-explanatory. Setup/teardown actions are also helpful when you need to clean up after your testing activity. Sometimes we run our tests against application in productioni mode (note, that we don't have to install anything other than application itself and ERSelenium framework to do that). In this case cleaning up is a must. Testing in production may sound dirty but in fact it is
Re: [Wonder-disc] ERSelenium without ERExtensions
Hi Chuck, On Jun 27, 2007, at 10:58 PM, Chuck Hill wrote: On Jun 27, 2007, at 5:53 AM, Michael Bushkov wrote: Chuck Hill wrote: One of my goals is to _not_ be the person who writes and maintains the functional tests. Because of this, it is important to keep the tests separate from the applications and frameworks that it tests. We think of selenium tests as about useful addition to Unit tests - and therefore we use different approach - the person who writes a part of the applications should prepare tests for this part. And that's why we treat selenium tests as a part of the application and store them in the application's folder. Yes, that is different from my use / intention. I try to keep my UI components very simple. So simple that they don't need tests. Obviously, using D2W, you have very different concerns and very different test needs. I have never worked on automated testing for a D2W app, I can see that would be quite a challenge. I don't really think that there is any difference between testing D2W or non-D2W apps since we focus on testing user scenarios and not components. One can easily break non-D2W app by renaming an attribute of an entity and you really want an easy way to check for broken bindings even if all of your unit tests are ok. Also, it is you as a developer (not QA team) who needs to check the scenarios right after refactoring and before commiting. Another thing is that tests maintainence leads to additional work so we try to write as few tests as possible and check only things that tend to break or are mission-critical. In our practice this helps discovering 90% bugs with 10% effort. Remaining 10% are normally discovered via log4j email error alerts or customer bug reports. And since these 10% bugs are not mission-critical our customers are still happy and we don't have to spend much time mainaining tests. -- Denis ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Re: [Wonder-disc] ERSelenium without ERExtensions
Hi Michael,
On Jun 27, 2007, at 5:53 AM, Michael Bushkov wrote:
Hi Chuck,
Chuck Hill wrote:
One of my goals is to _not_ be the person who writes and maintains
the functional tests. Because of this, it is important to keep the
tests separate from the applications and frameworks that it tests.
We think of selenium tests as about useful addition to Unit tests -
and
therefore we use different approach - the person who writes a part of
the applications should prepare tests for this part. And that's why we
treat selenium tests as a part of the application and store them in
the
application's folder.
Yes, that is different from my use / intention. I try to keep my UI
components very simple. So simple that they don't need tests.
Obviously, using D2W, you have very different concerns and very
different test needs. I have never worked on automated testing for a
D2W app, I can see that would be quite a challenge.
As my projects have functional tests anyway (part of our process), I
prefer to have only one set of Selenium tests rather than a developer
level set of tests and a domain level set of tests.
Anyway, to let a particular person edit a set of tests you need only
grant him access to Resources/Selenium subfolder of your application's
SVN/CVS repository and I don't see how ERSelenium-driven tests layout
can affect that.
- Will open/click commands in your test work ok if you run them on
the host with different WO base URL - for example if you switch
from "http://localhost/cgi-bin/WebObjects"; to "http://somehost/";?
Here is what I do. There are very few references to the URL. I
leave it as localhost for development as that is more convenient when
testing locally. For integration testing, I use Selenium RC (remote
control) from an Ant script. Before launching the suite, the Ant
script does a search and replace on the host and application names.
As I said earlier, that is only in a few places (like the log in test
case as the start of a string of tests cases).
Well this is some kind of workaround. Here is what we do. We develop
tests mostly in Selenium IDE. We specify Base URL there as:
http://localhost/cgi-bin/WebObjects/BugTracker.woa/-42421/
And then in tests we use only relative links of the following form:
|open|/wa/SeleniumAction/resetSession||
|openAndWait|/||
Yes, I use the full URL to call direct actions. I don't recall why I
don't use the Base URL, maybe an oversight! Component actions are
called by ID.
Selenium-ide handles base url and processes the test correctly.
ERSelenium transforms open/openAndWait/etc commands on the fly by
converting them to absolute URLs - and this allows them to be
correctly
processed by selenium-core.
I think a base URL can be used with Selenium RC as well. I should
look into that again.
The advantage of this approach is that you don't need to do test
source
code postprocessing. The same source code will be equally processed in
selenium-ide and on any host where ERSelenium is used.
Besides Selenium-RC looks like a kind of overkill solution to us,
so we
try to avoid it and use only selenium-core, which is much more simple
and easier to install and use.
Why does it look like overkill to you?
Most of the tests don't refer to the URL. Instead, I make extensive
use of HTML ID attributes. So a test for login would look like:
type userName chill2
type password chill2
clickAndWait loginButton
The use of IDs serves two purposes. One, it isolates the tests from
changes to the host, application name, etc. Two, it makes the tests
easier to read and understand. Again, this supports my goal of
having functional (domain) experts write and maintain the functional
tests. This is supported by a Selenium locator ordering that gives a
higher priority to the ID than to XPath etc.
Anjo points out that the use of ID is a Bad Idea (tm). Anjo, of
course, is dead wrong in this. :-)Anjo's objection was that ID
has to be unique in a page. This is true. For items in repetitions,
I append an index to the ID. For example, instead of id = "view", it
is rendered as id = "view.1"; or id = "view.row.1"; This does
require some code in the component that uses the repetition, but I do
not object to that as it supports my goal of readable tests. There
is probably a way to do this automatically; so far it has not seemed
worth the effort.
Another potential problem area is components that are used multiple
times on a page. Consider a text input that wraps up a WOTextField,
a required marker, and a validation message. The WOD of page using
this might look like:
FirstName: TextInputWithValidation { value = firstName; }
LastName: TextInputWithValidation { value = lastName; }
Obviously, if I implement TextInputWithValidation with a WOTextField
like this:
Text: WOTextField { value = ^value; id = "text"; }
the ID will get duplicated and bad things will happen. Instead, I
put the ID up a level:
FirstName: Tex
Re: [Wonder-disc] ERSelenium without ERExtensions
Hi Chuck,
Chuck Hill wrote:
One of my goals is to _not_ be the person who writes and maintains
the functional tests. Because of this, it is important to keep the
tests separate from the applications and frameworks that it tests.
We think of selenium tests as about useful addition to Unit tests - and
therefore we use different approach - the person who writes a part of
the applications should prepare tests for this part. And that's why we
treat selenium tests as a part of the application and store them in the
application's folder.
Anyway, to let a particular person edit a set of tests you need only
grant him access to Resources/Selenium subfolder of your application's
SVN/CVS repository and I don't see how ERSelenium-driven tests layout
can affect that.
- Will open/click commands in your test work ok if you run them on
the host with different WO base URL - for example if you switch
from "http://localhost/cgi-bin/WebObjects"; to "http://somehost/";?
Here is what I do. There are very few references to the URL. I
leave it as localhost for development as that is more convenient when
testing locally. For integration testing, I use Selenium RC (remote
control) from an Ant script. Before launching the suite, the Ant
script does a search and replace on the host and application names.
As I said earlier, that is only in a few places (like the log in test
case as the start of a string of tests cases).
Well this is some kind of workaround. Here is what we do. We develop
tests mostly in Selenium IDE. We specify Base URL there as:
http://localhost/cgi-bin/WebObjects/BugTracker.woa/-42421/
And then in tests we use only relative links of the following form:
|open|/wa/SeleniumAction/resetSession||
|openAndWait|/||
Selenium-ide handles base url and processes the test correctly.
ERSelenium transforms open/openAndWait/etc commands on the fly by
converting them to absolute URLs - and this allows them to be correctly
processed by selenium-core.
The advantage of this approach is that you don't need to do test source
code postprocessing. The same source code will be equally processed in
selenium-ide and on any host where ERSelenium is used.
Besides Selenium-RC looks like a kind of overkill solution to us, so we
try to avoid it and use only selenium-core, which is much more simple
and easier to install and use.
Most of the tests don't refer to the URL. Instead, I make extensive
use of HTML ID attributes. So a test for login would look like:
type userName chill2
type password chill2
clickAndWait loginButton
The use of IDs serves two purposes. One, it isolates the tests from
changes to the host, application name, etc. Two, it makes the tests
easier to read and understand. Again, this supports my goal of
having functional (domain) experts write and maintain the functional
tests. This is supported by a Selenium locator ordering that gives a
higher priority to the ID than to XPath etc.
Anjo points out that the use of ID is a Bad Idea (tm). Anjo, of
course, is dead wrong in this. :-)Anjo's objection was that ID
has to be unique in a page. This is true. For items in repetitions,
I append an index to the ID. For example, instead of id = "view", it
is rendered as id = "view.1"; or id = "view.row.1"; This does
require some code in the component that uses the repetition, but I do
not object to that as it supports my goal of readable tests. There
is probably a way to do this automatically; so far it has not seemed
worth the effort.
Another potential problem area is components that are used multiple
times on a page. Consider a text input that wraps up a WOTextField,
a required marker, and a validation message. The WOD of page using
this might look like:
FirstName: TextInputWithValidation { value = firstName; }
LastName: TextInputWithValidation { value = lastName; }
Obviously, if I implement TextInputWithValidation with a WOTextField
like this:
Text: WOTextField { value = ^value; id = "text"; }
the ID will get duplicated and bad things will happen. Instead, I
put the ID up a level:
FirstName: TextInputWithValidation { value = firstName; id =
"FirstName"; }
LastName: TextInputWithValidation { value = lastName; id =
"LastName"; }
Text: WOTextField { value = ^value; id = ^id; }
This has the advantage of being more readable as well as eliminating
the duplicate ID problem. Passing the ID down has avoided the
problem of duplicate IDs for me so far. Sometimes you need to do a
bit more work and concatenate the ID from multiple levels, but I have
not run into a situation where it required much work or thought to
come up with a unique ID that was also sensible to someone reading
the test.
I fully agree - using IDs is probably the best approach in writing
selenium-tests. And it's great if you can support proper id generation
in your application from the very start. But it's not always possible -
there are a lot of applications, that don't have such support, but
should be tested. And
Re: [Wonder-disc] ERSelenium without ERExtensions
On Jun 26, 2007, at 3:19 PM, Steven Mark McCraw wrote: ERSelenium's idea is to facilitate a lot of common tasks that are usually solved manually to organize Selenium testing. Among these tasks are: - Automatic test suites generation from your folder structure. I am curious as to what you are doing for this. To me, Selenium tests are for functional testing and so have no relation at all to project structure. The reason for doing it this way (says the guy who's used ERSelenium a few times now) is that is a piece of cake to create test suites. There is no maintaining an external test suite. You just create a folder called Selenium (or whatever you configure the name to be) in the Properties folder of the project you want to test, and any folder you embed within it becomes a test suite when you run everything from a simple URL. You drop your selenium tests in child folders of the Selenium folder and you are done. Are the test cases ordered? It is ridiculously easy, as opposed to the whole chrome:// test suite thing, which I never got to work, although I did not spend much time trying because ERSelenium became available just as I began getting good and familiar with using Selenium. The up side to user ERSelenium test suites (besides how easy it is to create them) is that they run in any browser. All you have to do is point to the 'StartSeleniumTesting' direct action class in your application url, and away you go. I believe I read that if you use chrome:// as your URL, you are stuck with Firefox as a testing platform, because only Firefox supports that protocol. Well, up to a point. Have you tried file uploads yet? :-) There are security restrictions on JavaScript and running in chrome mode is the only way (that I know of) to get around them. - Writing tests in the preferred format - wiki/html. Is that different from Selenese? The difference is that instead of using html tags to delimit things, you use pipe symbols, and it's a lot more readable (says me). Instead of type userName chill2 type password chill2 clickAndWait loginButton You have |type|userName|chill2| |type|password|chill2| |clickAndWait|loginButton|| But I believe you can use this format with or without ERSelenium. I use a WYSIWG HTML editor. :-) Chuck - Unified format of setup/tear down methods with appropriate error messages. I don't see the need for these. For me, the Selenium tests should be from the point of view of the user of the application. So any setup / tear down outside of loading the bootstrap data should be done through the UI and be part of the test. In some rare occasions (mimicking interactions from other systems), I do need some setup that is not possible from the UI. In that case, I create a direct action to do this and call it from the test in a new window. The direct action is only enabled during testing runs. I agree with you here. I've just been using direct actions. Not sure what ERSelenium brings to the table here or how to use it. -- --- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Wonder-disc mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/wonder-disc -- Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solve specific problems. http://www.global-village.net/products/practical_webobjects ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Re: [Wonder-disc] ERSelenium without ERExtensions
ERSelenium's idea is to facilitate a lot of common tasks that are usually solved manually to organize Selenium testing. Among these tasks are: - Automatic test suites generation from your folder structure. I am curious as to what you are doing for this. To me, Selenium tests are for functional testing and so have no relation at all to project structure. The reason for doing it this way (says the guy who's used ERSelenium a few times now) is that is a piece of cake to create test suites. There is no maintaining an external test suite. You just create a folder called Selenium (or whatever you configure the name to be) in the Properties folder of the project you want to test, and any folder you embed within it becomes a test suite when you run everything from a simple URL. You drop your selenium tests in child folders of the Selenium folder and you are done. It is ridiculously easy, as opposed to the whole chrome:// test suite thing, which I never got to work, although I did not spend much time trying because ERSelenium became available just as I began getting good and familiar with using Selenium. The up side to user ERSelenium test suites (besides how easy it is to create them) is that they run in any browser. All you have to do is point to the 'StartSeleniumTesting' direct action class in your application url, and away you go. I believe I read that if you use chrome:// as your URL, you are stuck with Firefox as a testing platform, because only Firefox supports that protocol. - Writing tests in the preferred format - wiki/html. Is that different from Selenese? The difference is that instead of using html tags to delimit things, you use pipe symbols, and it's a lot more readable (says me). Instead of type userName chill2 type password chill2 clickAndWait loginButton You have |type|userName|chill2| |type|password|chill2| |clickAndWait|loginButton|| But I believe you can use this format with or without ERSelenium. - Unified format of setup/tear down methods with appropriate error messages. I don't see the need for these. For me, the Selenium tests should be from the point of view of the user of the application. So any setup / tear down outside of loading the bootstrap data should be done through the UI and be part of the test. In some rare occasions (mimicking interactions from other systems), I do need some setup that is not possible from the UI. In that case, I create a direct action to do this and call it from the test in a new window. The direct action is only enabled during testing runs. I agree with you here. I've just been using direct actions. Not sure what ERSelenium brings to the table here or how to use it. ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Re: [Wonder-disc] ERSelenium without ERExtensions
Hi Michael,
Right, we were supposed to discuss this when I returned from WWDC. I
am still working through a page of "to do" tasks that I accumulated
that week.
On Jun 26, 2007, at 2:38 AM, Michael Bushkov wrote:
Chuck, I've got several questions:
- How do you prepare a test suite?
I am not certain what you are asking. I use the Selenium IDE to
create the individual test cases. I create the suite by hand, in an
HTML editor, so that I can control the order of the tests and add
HTML comments on what is happening and why. I maintain this
externally to the project in a directory that looks like this:
Project/
Code/
WOApp/
WOFramework1/
WOFramework2/
Testing/
selenium/
Suites/
TestCases/
TestData/
One of my goals is to _not_ be the person who writes and maintains
the functional tests. Because of this, it is important to keep the
tests separate from the applications and frameworks that it tests.
- What format do you use to store the selenium-tests?
Selenese (HTML). Again, this supports test creation and maintenance
by non-developers. The people doing the test creation and
maintenance do need some technical ability, but not to the extent of
knowing Java or XPath.
- Will open/click commands in your test work ok if you run them on
the host with different WO base URL - for example if you switch
from "http://localhost/cgi-bin/WebObjects"; to "http://somehost/";?
Here is what I do. There are very few references to the URL. I
leave it as localhost for development as that is more convenient when
testing locally. For integration testing, I use Selenium RC (remote
control) from an Ant script. Before launching the suite, the Ant
script does a search and replace on the host and application names.
As I said earlier, that is only in a few places (like the log in test
case as the start of a string of tests cases).
Most of the tests don't refer to the URL. Instead, I make extensive
use of HTML ID attributes. So a test for login would look like:
type
userName
chill2
type
password
chill2
clickAndWait
loginButton
The use of IDs serves two purposes. One, it isolates the tests from
changes to the host, application name, etc. Two, it makes the tests
easier to read and understand. Again, this supports my goal of
having functional (domain) experts write and maintain the functional
tests. This is supported by a Selenium locator ordering that gives a
higher priority to the ID than to XPath etc.
Anjo points out that the use of ID is a Bad Idea (tm). Anjo, of
course, is dead wrong in this. :-) Anjo's objection was that ID
has to be unique in a page. This is true. For items in repetitions,
I append an index to the ID. For example, instead of id = "view", it
is rendered as id = "view.1"; or id = "view.row.1"; This does
require some code in the component that uses the repetition, but I do
not object to that as it supports my goal of readable tests. There
is probably a way to do this automatically; so far it has not seemed
worth the effort.
Another potential problem area is components that are used multiple
times on a page. Consider a text input that wraps up a WOTextField,
a required marker, and a validation message. The WOD of page using
this might look like:
FirstName: TextInputWithValidation {
value = firstName;
}
LastName: TextInputWithValidation {
value = lastName;
}
Obviously, if I implement TextInputWithValidation with a WOTextField
like this:
Text: WOTextField {
value = ^value;
id = "text";
}
the ID will get duplicated and bad things will happen. Instead, I
put the ID up a level:
FirstName: TextInputWithValidation {
value = firstName;
id = "FirstName";
}
LastName: TextInputWithValidation {
value = lastName;
id = "LastName";
}
Text: WOTextField {
value = ^value;
id = ^id;
}
This has the advantage of being more readable as well as eliminating
the duplicate ID problem. Passing the ID down has avoided the
problem of duplicate IDs for me so far. Sometimes you need to do a
bit more work and concatenate the ID from multiple levels, but I have
not run into a situation where it required much work or thought to
come up with a unique ID that was also sensible to someone reading
the test.
ERSelenium's idea is to facilitate a lot of common tasks that are
usually solved manually to organize Selenium testing. Among these
tasks are:
- Automatic test suites generation from your folder structure.
I am curious as to what you are doing for this. To me, Selenium
tests are for functional testing and so have no relation at all to
project structure.
