On 25 September 2012 21:57, Chuck Rolke <cro...@redhat.com> wrote: > > > ----- Original Message ----- > > From: "Rob Godfrey" <rob.j.godf...@gmail.com> > > To: proton@qpid.apache.org > > Sent: Tuesday, September 25, 2012 3:19:23 PM > > Subject: Re: Engine API > > > > On 25 September 2012 19:05, Rafael Schloming <r...@alum.mit.edu> > > wrote: > > > > > Ken already wrote a bunch of documentation on this, perhaps he > > > could post a > > > link? > > > > > > Also, if you run 'make docs' you can see all the doxygen for the > > > engine > > > APIs. > > > > > > While I agree we need more documentation, keeping the C and Java > > > impls in > > > sync is a separate matter. Documentation as a functional spec is > > > really > > > only useful if it reaches the level of a formal specification that > > > never > > > changes, however that's just not realistic for us. > > > > > > Besides, we already have a much much more powerful tool for keeping > > > the > > > implementations in sync. We have the python test suite that will > > > run > > > against both implementations and tell us precisely where they > > > differ. I > > > really would like this test suite to be the formal spec, and like > > > everything else it could certainly use more documentation, but it's > > > quite > > > capable of keeping the implementations in sync without that, and > > > more > > > importantly unlike a static document, we can drive changes in the > > > functional spec that might be motivated by either the C or Java or > > > JS impl > > > back into this test suite and spread them to the other > > > implementations much > > > more quickly than we can with just a document. > > > > > > If there are problems with that model, or if the test suite is > > > somehow > > > insufficient, I'd really like to know. I know Rob has mentioned > > > that the > > > tests need to be more granular, some of them are a bit monolithic > > > because > > > they are trying to reproduce a very specific set of behaviour. For > > > the new > > > stuff I've been adding I've been trying to keep things as granular > > > as > > > possible. > > > > > > > > To expand here a little... I think defining the functional spec via > > the > > tests is both sensible and powerful. However, for this to work, each > > test > > need to clearly correspond to a single piece of functionality that > > should > > be common across all implementations; and the tests must be *much* > > more > > comprehensive than they are currently. Moreover we should be > > developing the > > tests in advance of actually implementing in either C or Java (or - > > when we > > get to it - JavaScript). > > > > In general I feel that we should first be agreeing the scope of > > functionality on the list, then agreeing the API / detailed > > functional spec > > via the definition of tests in the Python test suite (with reasonably > > comprehensive documentation as to the functional requirement behind > > each > > test), and only then attempting to implement across the various > > languages. > > I feel like we are currently defining behaviour via implementation > > which > > may be sustainable in the short term, but probably doesn't scale if > > we are > > expecting a larger community to be contributing to the project. > > > > -- Rob > > > > > > The reason the python tests *won't* work as a spec are precisely > because they cannot "correspond to a piece of functionality". > > I'm not sure that you have put forward a case that they "can not" only that they "do not". I agree that the current tests are a fair way from meeting the requirements of a functional spec, but fundamentally there should be a 1:1 correspondence between a functional spec and a test suite. Whether one defines the spec explicitly (and then writes an implicit test suite) or implicitly (via the mechanism of a well defined set of tests) is really a style issue I think.
> The tests are really good at what they do: verifying that functions > work as expected. Keep them coming. > > Actually I disagree. The tests currently aren't all that great. When trying to get the tests to pass against the Java codebase there's really no help in trying to work out what the tests are supposed to do (though some - such as against the message API are much better by having a much clearer definition of what it is that they are testing). > But the tests are lousy at specifying what can be done. I pulled up > messenger.py as a random example. > 1. Not a single comment. > +1 > 2. Magic numbers. > +1 > 3. Magic process flow (setup, teardown part of some framework > unrelated to messenger. What's with assert? What's the thread model?). > +1 > 4. No enumeration of functions to call, function arg lists, arg > defaults, arg units, and so on. > Not sure exactly what you mean here, though if you are saying that the tests should in some way be categorized by which API function they are testing, then I completely agree. > 5. What are prerequisites? > > As a counterexample to using tests as specs, take a look at > cpp/src/tests/acl.py and tell me how C++ broker ACLs work. > No amount of ACL python test code will be a decent spec from > which users can understand and write ACL rules. > > I'm not sure that one can prove a negative by counter-example. My personal view is that we *can* define a spec implicitly through tests, but that we haven't yet proved that conjecture. Ultimately a (functional) test suite and a functional spec should have some sort of one-to-one correspondence and if they are defined in separate forms then there is a risk of divergence (and code is generally more precise than text). However for many people (probably including myself) it is easier to first write the positive statement (the spec) and then define the negative (via the test). Ultimately the means (test vs. doc) is less important than just ensuring that there is an adequate definition somewhere, which - to Rajith's original point - I'm not sure there is yet (at least not written down). -- Rob > -Chuck > > > > > > > > > --Rafael > > > > > > On Thu, Sep 20, 2012 at 3:45 PM, Rajith Attapattu > > > <rajit...@gmail.com > > > >wrote: > > > > > > > We should seriously consider documenting the Engine API and it's > > > > expected behaviour. > > > > This has several advantages. > > > > > > > > 1. Use that as a basis for a functional spec, which we can then > > > > write > > > > tests to verify the implementations. > > > > > > > > 2. Helps developers to navigate and understand the code more > > > > easily. > > > > > > > > 3. Finally we need this documentation sooner than later to > > > > accompany > > > > our releases to for the end users > > > > > > > > 4. Helps to keep both C and Java impls in sync. We should > > > > probably > > > > modify the doc before we change the code to ensure the doc is > > > > kept > > > > current. > > > > > > > > Any takers ? (Looking at Rafi :D) > > > > > > > > Regards, > > > > > > > > Rajith > > > > > > > > > >
