inline On 02-Apr-2014, at 11:06 AM, Sowmya Krishnan <sowmya.krish...@citrix.com> wrote:
> Thanks for the FS Koushik. I think these enhancements would greatly help in > enhancing our test suite, including performance tests. > I have few questions/comments: > > 1. You mention about simulating a fixed fault/ fixed failure. Does this mean > that when I configure simulator to fail it always keeps failing for that > command until I re-configure it to succeed? Or is there a way I can say, "a > command should fail X % of the time"? [Koushik] Depends on how you configure it. If count is specified, it will fail for that many times. If it is not specified then it will keep on failing till the time you clean it up. > 2. Can I have different delays configured across different hosts/Clusters for > the same command? [Koushik] Yes. You need to create a separate configuration/mock for each specific host/cluster as required. > 3. I believe I can configure a responseType to be a fail/fault and at the > same time define a responseJSON as well in tandem with the fault/failure. [Koushik] Once you specify the JSON, response is already fixed. The logic is to simply deserialize the JSON and return it back to the upper layers. > 4. IIRC, delay is in ms presently. For certain commands like PingCommand, > even 1 sec might be a high value. For some of my performance tests, I've used > much lesser delay for such commands of the order of ms. [Koushik] Yes. > 5. Is there an ability to simulate fault for a specific storage/host? For > example, I have 5 storages in my cluster but I want to simulate failure for 3 > of the storage pools. Is that possible? > [Koushik] Good point. I will add storage pool also into the scope. > >> -----Original Message----- >> From: Koushik Das [mailto:koushik....@citrix.com] >> Sent: Tuesday, April 01, 2014 5:48 PM >> To: <dev@cloudstack.apache.org> >> Subject: [PROPOSAL] Simulator enhancements >> >> The current simulator provides a way to test various Cloudstack features >> without using actual resources. But its current capabilities are limited >> since it >> only allows success response for all agent commands and some ability to >> introduce delays. I am proposing some enhancements to the simulator [1] based >> on which test developers can write end-to-end tests for scenarios which were >> earlier not possible. Some examples would be VM deployment failure (due to >> storage/network failure), VM deployment retry logic, user VM HA etc. >> >> In the current set of automation tests mostly positive scenarios are tested >> as it is >> very complex to test negative scenarios with real hardware. With the ability >> to >> simulate failure, negative scenarios can be tested in an effective manner. >> Negative test scenarios will also help in improving the overall code >> coverage of >> the tests. >> >> Please share your feedback/comments. >> >> Thanks, >> Koushik >> >> [1] >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Simulator+enhance >> ments
