Swagger is meant to be used to design the API which can then spit out a spec (as far as I can tell). I don't know how well that integrates with our existing API, or even our existing documentation. But supposing it's fairly easy to do, it means creating a separate model for every API response we have, and that means maintaining all of those models twice. If we're going to integrate Swagger I think it's better to wait until API v2 so that we can use it to make the model and maintain it exactly once. ________________________________________ From: Gray, Jonathan <[email protected]> Sent: Monday, February 4, 2019 1:09 PM To: [email protected] Subject: Re: [EXTERNAL] Traffic Ops Mock for testing
What do you think about instead of rolling a separate tool we just finish the swagger documentation that's partially integrated. Then you can just feed that swagger catalog into any languages stub code generator. Bonus is that you get client stubs too for free. Jonathan G On 2/4/19, 1:03 PM, "Fieck, Brennan" <[email protected]> wrote: Just to be perfectly clear: there should be no database, just a single service - an HTTP server - that responds to requests for the Traffic Ops API request paths with responses that are structurally and syntactically valid for the respective request path. ________________________________________ From: Rawlin Peters <[email protected]> Sent: Monday, February 4, 2019 10:35 AM To: [email protected] Subject: Re: [EXTERNAL] Traffic Ops Mock for testing The idea is that Mock TO will use the same exact Go structs as TO, and by default Mock TO will dynamically generate a response from the given endpoint using reflection on the endpoint's corresponding Go struct. No dataset would need to be maintained for Mock TO unless you wanted to write specific test cases in which you provide a predefined static response for Mock TO to return (be it via predefined Go structs or just JSON files). The mechanism for choosing dynamic vs static responses would have to be designed still, but that's the general idea. - Rawlin On Mon, Feb 4, 2019 at 10:25 AM Gray, Jonathan <[email protected]> wrote: > > How do you plan to build/maintain the dataset behind such a tool? Do you intend to use a fixed TO dataset backed by postgres, or just fixed responses to various combinations of parameters. > > Jonathan G > > On 2/4/19, 9:41 AM, "Rawlin Peters" <[email protected]> wrote: > > Hey all, > > @ocket8888 and I would like to propose implementing a minimal Traffic > Ops "mock" that could be used for integration testing things like > Traffic Ops clients (basically, anything that interacts with the > Traffic Ops API but doesn't need to verify the actual Traffic Ops > implementation, just the external interface) without having to stand > up a real instance of Traffic Ops. > > This Mock TO would basically just be a minimal Go HTTP server that > receives requests against the TO API endpoints, unmarshalls JSON > request data into the endpoint's corresponding Go TC struct, marshalls > the Go TC struct back into JSON, and returns it back to the client. > > So for most POST/PUT endpoints, the Mock TO would basically just be an > echo server, and for DELETEs it could just return some > statically-defined response that matches the API implementation. > Ideally, the DELETE could just reuse the corresponding DELETE success > response from TO, but we don't know how tightly we should integrate > this Mock TO with the Real TO just yet. For GET endpoints, we could > have Mock TO return some statically-defined JSON response for the > endpoint (be it via literal JSON files read off the filesystem or just > predefined TC Go structs that would marshal into the expected JSON), > but it would be nice to also have the Mock TO be able to dynamically > generate a GET response using reflection against the Go struct. With > dynamic generation of the response, we wouldn't have to maintain > static JSON files against the current version of the API for > everything, but we'd still be able to define static JSON responses for > specific test scenarios. > > For example, in order to test ORT, you might want to hit the Mock TO > but define a static response for a configfile endpoint in order to > verify that ORT can create the file from a given response. > > In order to test the Python and Java TO clients, we don't necessarily > need to define static JSON responses from Mock TO, since for those we > don't care about the actual values but more so about the structure of > the request and response through the various stages between the client > and the server (marshalling, unmarshalling, etc). > > We're currently getting great test coverage of the Go TO client due to > being used for the Go API integration tests, but the Python and Java > TO clients are getting left behind. It would be nice to at least be > able to verify that there haven't been any breaking-API changes that > would actually break TO clients in the repo. For example, if there is > an API regression in a Go-rewrite of a Perl endpoint that breaks the > Python TO client, we can easily find and fix that regression before > the PR is merged. Currently, the only way we can find out if the > Python or Java TO clients have been broken is through normal usage > after breaking-PRs have been merged, which is not great. > > A Mock TO would enable us to test all these things, keep the in-repo > TO clients from breaking, and help us prevent breaking the Traffic Ops > API contract accidentally. Mock TO would use the same Go structs as > the Real TO, so Mock TO wouldn't have to maintain its own separate > structs and would automatically stay in sync with changes to the > official TO structs. It would essentially always stay up to date with > the latest version of the structs, which we think is important for > something like this. Otherwise, it could quickly become out of sync > with what it's supposed to mock, and its value as a test tool would be > degraded. > > A major advantage of this Mock TO is that it could be used for easy > integration testing where you don't necessarily want to set up and > configure a real instance of Traffic Ops, which is currently a major > pain and honestly a bit overkill for things like just verifying that a > TO client can properly handle requests/responses using the current > version of the API. > > Some goals we would have for Mock TO: > - be integrated enough with Real TO so that it "automatically" stays > up to date with Real TO's interface as much as possible > - require very little maintenance (i.e. as new endpoints are added or > existing endpoints are modified, those changes would be picked up > "automatically" in Mock TO) > - virtually no required configuration (probably just a port to listen on) > - require no prerequisite data in order to run and serve mock requests > > Please let us know if there's something specific you'd also like to > see in a Mock TO or have other ideas for mocking TO that you'd like to > share. We'd like for Mock TO to be as pain-free and useful as > possible. > > - @ocket8888 and Rawlin > >
