Re: [Standards] XSF Server Test Suite
This is the direction things are heading. The syntax is just something that we came up with at the last summit but I've exptended it. Perhaps is over complicated for the task. Perhaps these test scripts should be written in XML in order to avoid specialized parsers. Are there any decent C-syntax-style parsers out there already? In any case, I'd like to hear how people think this should be done. It is not a major issue but something to keep in mind designing the test suite. From my experience from working on the test suite for XMPP the most difficult part is to test cases when there are 2 or more user connections needed at the same time. As an example let's take presence subscription. You would like to test it for a different response from the other user - subscription approved, rejected. What about the case when the second user is offline at the time of the request sending and so on These are the cases which require a bit more complex logic/engine than just simple send some packet, receive response and compare it with pattern. Artur -- Artur Hefczyc http://www.tigase.org/ http://artur.hefczyc.net/
Re: [Standards] XSF Server Test Suite
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 9/29/09 10:20 AM, Nathan Fritz wrote: > Peter, I notice the interop group is not on the discussions page. That's because it hasn't been active. I'll add it now. If folks are interested, please continue this discussion on the inte...@xmpp.org list, to which you can subscribe here: http://mail.jabber.org/mailman/listinfo/interop Peter - -- Peter Saint-Andre https://stpeter.im/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.8 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkrCPqkACgkQNL8k5A2w/vy6nQCgzuZ0Y2k6EMiqaLtoEnMPFjek nnwAn3Vh+XX3a2qqwhnIBLcOKt/s4O/3 =bwKh -END PGP SIGNATURE-
Re: [Standards] XSF Server Test Suite
> > For the past 6 months, we've been discussing on creating a standardized > test suite as a tool for XMPP Server developers, and perhaps in the future > for further purposes. There was discussion of having a hosted solution that > can execute scripts from a list of > > --- > > What about using python and xmppy for the test scripts? Python is very easy > to pick up for C programmers (and others)? > > > Python is my go-to language as well, however, the point is to generate simple expect style scripts -- the tests need to avoid doing anything more complex than building xml, expecting, asserting, and sending. These points and counterpoints have been raised and hashed through already (though if you guys want to have it again, that's fine). Arguably, foreach and if statements are overkill as those use cases could be tested by applying a dtd. A simple interpreter is all that is needed to run the suite of tests that anyone can contribute to. In fact, scratch my foreach/if statements. Building stanzas, sending, expecting with XPath, getting with XPath, and asserting with DTDs and XPath are all that is necessary. -Fritz
Re: [Standards] XSF Server Test Suite
-Original Message- From: standards-boun...@xmpp.org [mailto:standards-boun...@xmpp.org] On Behalf Of Nathan Fritz Sent: 09/29/2009 12:21 PM To: inte...@xmpp.org; XMPP Extension Discussion List Subject: [Standards] XSF Server Test Suite I'm double posting this to standards and interop, but generally this is a conversation for interop that I thought you should all be aware of. Peter, I notice the interop group is not on the discussions page. For the past 6 months, we've been discussing on creating a standardized test suite as a tool for XMPP Server developers, and perhaps in the future for further purposes. There was discussion of having a hosted solution that can execute scripts from a list of --- What about using python and xmppy for the test scripts? Python is very easy to pick up for C programmers (and others)? Thanks
[Standards] XSF Server Test Suite
I'm double posting this to standards and interop, but generally this is a
conversation for interop that I thought you should all be aware of. Peter,
I notice the interop group is not on the discussions page.
For the past 6 months, we've been discussing on creating a standardized test
suite as a tool for XMPP Server developers, and perhaps in the future for
further purposes. There was discussion of having a hosted solution that can
execute scripts from a list of supported language, but this plan was
scrapped at the last summit. What we came up with instead is a simple
XPath-Expect script of our own design that we can easily build an
interpreter for.
There will be some basic commands: expect, get, send, assert, foreach_sub,
if_xpath, include, etc as well as built-in helper functions to jump the
script to the right state in the connection without having to be explicit.
There also needs to be some functions for building out stanzas and payload
element trees. Any xml argument could be a stanza object or string.
stream = connect('u...@xmpp.org/resource', 'password');
start_session(stream); # this command gets us through all of the stream
features, authenticated, and bound. This of course, could be done with the
send/expect commands as well. There may be other such commands that get us
to the state we need.
id = unique_id();
result = send(stream, ("", id), ("[xpath statement with %s for id]",
id));
#OR
iq_roster = makeIq('get', 'xmpp.org');
query = makeElement('{jabber:iq:roster}query');
append_xml(iq_roster, query);
id = get(iq_roster, '@id');
assert(result, ([xpath], [replacement values],));
foreach_sub(result, sub, ([xpath],)) {
assert("Testing item structure", sub, ([xpath]));
group = get(sub, ([xpath],));
# do something with groups here
}
This is the direction things are heading. The syntax is just something that
we came up with at the last summit but I've exptended it. Perhaps is over
complicated for the task. Perhaps these test scripts should be written in
XML in order to avoid specialized parsers. Are there any decent
C-syntax-style parsers out there already? In any case, I'd like to hear how
people think this should be done.
Thanks,
Nathan Fritz
