I agree. I think Swagger is a nice goal for api 2.x. Would probably be pretty messy integrating with 1.x. In my experience, tools like consistency and api 1.x is hardly consistent.
On Mon, Feb 4, 2019 at 1:53 PM Rawlin Peters <[email protected]> wrote: > Swagger would be a good tool for this, but the amount of work involved > in Swagger-izing the existing TO API would mean that we most likely > wouldn't be able to get a Mock TO for quite a long time, and we could > benefit from something like this in the meantime. > > I agree that we should probably integrate Swagger into TO API 2.0 from > the start, but I'm not sure we should really spend much effort on > Swagger-izing the existing TO API since that would mean less capacity > to work on a better version of the API with Swagger built-in from the > start. > > - Rawlin > > On Mon, Feb 4, 2019 at 1:31 PM Fieck, Brennan <[email protected]> > wrote: > > > > Swagger is meant to be used to design the API which can then spit out a > spec (as far as I can tell). I don't know how well that integrates with our > existing API, or even our existing documentation. But supposing it's fairly > easy to do, it means creating a separate model for every API response we > have, and that means maintaining all of those models twice. If we're going > to integrate Swagger I think it's better to wait until API v2 so that we > can use it to make the model and maintain it exactly once. > > ________________________________________ > > From: Gray, Jonathan <[email protected]> > > Sent: Monday, February 4, 2019 1:09 PM > > To: [email protected] > > Subject: Re: [EXTERNAL] Traffic Ops Mock for testing > > > > What do you think about instead of rolling a separate tool we just > finish the swagger documentation that's partially integrated. Then you can > just feed that swagger catalog into any languages stub code generator. > Bonus is that you get client stubs too for free. > > > > Jonathan G > > > > > > On 2/4/19, 1:03 PM, "Fieck, Brennan" <[email protected]> wrote: > > > > Just to be perfectly clear: there should be no database, just a > single service - an HTTP server - that responds to requests for the Traffic > Ops API request paths with responses that are structurally and > syntactically valid for the respective request path. > > ________________________________________ > > From: Rawlin Peters <[email protected]> > > Sent: Monday, February 4, 2019 10:35 AM > > To: [email protected] > > Subject: Re: [EXTERNAL] Traffic Ops Mock for testing > > > > The idea is that Mock TO will use the same exact Go structs as TO, > and > > by default Mock TO will dynamically generate a response from the > given > > endpoint using reflection on the endpoint's corresponding Go struct. > > No dataset would need to be maintained for Mock TO unless you wanted > > to write specific test cases in which you provide a predefined static > > response for Mock TO to return (be it via predefined Go structs or > > just JSON files). The mechanism for choosing dynamic vs static > > responses would have to be designed still, but that's the general > > idea. > > > > - Rawlin > > > > On Mon, Feb 4, 2019 at 10:25 AM Gray, Jonathan > > <[email protected]> wrote: > > > > > > How do you plan to build/maintain the dataset behind such a tool? > Do you intend to use a fixed TO dataset backed by postgres, or just fixed > responses to various combinations of parameters. > > > > > > Jonathan G > > > > > > On 2/4/19, 9:41 AM, "Rawlin Peters" <[email protected]> > wrote: > > > > > > Hey all, > > > > > > @ocket8888 and I would like to propose implementing a minimal > Traffic > > > Ops "mock" that could be used for integration testing things > like > > > Traffic Ops clients (basically, anything that interacts with > the > > > Traffic Ops API but doesn't need to verify the actual Traffic > Ops > > > implementation, just the external interface) without having to > stand > > > up a real instance of Traffic Ops. > > > > > > This Mock TO would basically just be a minimal Go HTTP server > that > > > receives requests against the TO API endpoints, unmarshalls > JSON > > > request data into the endpoint's corresponding Go TC struct, > marshalls > > > the Go TC struct back into JSON, and returns it back to the > client. > > > > > > So for most POST/PUT endpoints, the Mock TO would basically > just be an > > > echo server, and for DELETEs it could just return some > > > statically-defined response that matches the API > implementation. > > > Ideally, the DELETE could just reuse the corresponding DELETE > success > > > response from TO, but we don't know how tightly we should > integrate > > > this Mock TO with the Real TO just yet. For GET endpoints, we > could > > > have Mock TO return some statically-defined JSON response for > the > > > endpoint (be it via literal JSON files read off the filesystem > or just > > > predefined TC Go structs that would marshal into the expected > JSON), > > > but it would be nice to also have the Mock TO be able to > dynamically > > > generate a GET response using reflection against the Go > struct. With > > > dynamic generation of the response, we wouldn't have to > maintain > > > static JSON files against the current version of the API for > > > everything, but we'd still be able to define static JSON > responses for > > > specific test scenarios. > > > > > > For example, in order to test ORT, you might want to hit the > Mock TO > > > but define a static response for a configfile endpoint in > order to > > > verify that ORT can create the file from a given response. > > > > > > In order to test the Python and Java TO clients, we don't > necessarily > > > need to define static JSON responses from Mock TO, since for > those we > > > don't care about the actual values but more so about the > structure of > > > the request and response through the various stages between > the client > > > and the server (marshalling, unmarshalling, etc). > > > > > > We're currently getting great test coverage of the Go TO > client due to > > > being used for the Go API integration tests, but the Python > and Java > > > TO clients are getting left behind. It would be nice to at > least be > > > able to verify that there haven't been any breaking-API > changes that > > > would actually break TO clients in the repo. For example, if > there is > > > an API regression in a Go-rewrite of a Perl endpoint that > breaks the > > > Python TO client, we can easily find and fix that regression > before > > > the PR is merged. Currently, the only way we can find out if > the > > > Python or Java TO clients have been broken is through normal > usage > > > after breaking-PRs have been merged, which is not great. > > > > > > A Mock TO would enable us to test all these things, keep the > in-repo > > > TO clients from breaking, and help us prevent breaking the > Traffic Ops > > > API contract accidentally. Mock TO would use the same Go > structs as > > > the Real TO, so Mock TO wouldn't have to maintain its own > separate > > > structs and would automatically stay in sync with changes to > the > > > official TO structs. It would essentially always stay up to > date with > > > the latest version of the structs, which we think is important > for > > > something like this. Otherwise, it could quickly become out of > sync > > > with what it's supposed to mock, and its value as a test tool > would be > > > degraded. > > > > > > A major advantage of this Mock TO is that it could be used for > easy > > > integration testing where you don't necessarily want to set up > and > > > configure a real instance of Traffic Ops, which is currently a > major > > > pain and honestly a bit overkill for things like just > verifying that a > > > TO client can properly handle requests/responses using the > current > > > version of the API. > > > > > > Some goals we would have for Mock TO: > > > - be integrated enough with Real TO so that it "automatically" > stays > > > up to date with Real TO's interface as much as possible > > > - require very little maintenance (i.e. as new endpoints are > added or > > > existing endpoints are modified, those changes would be picked > up > > > "automatically" in Mock TO) > > > - virtually no required configuration (probably just a port to > listen on) > > > - require no prerequisite data in order to run and serve mock > requests > > > > > > Please let us know if there's something specific you'd also > like to > > > see in a Mock TO or have other ideas for mocking TO that you'd > like to > > > share. We'd like for Mock TO to be as pain-free and useful as > > > possible. > > > > > > - @ocket8888 and Rawlin > > > > > > > > > > >
