On Mon, Apr 4, 2011 at 9:25 AM, Michael Foord <mfo...@python.org> wrote:
> On 04/04/2011 15:14, Carl Karsten wrote: > > > > On Mon, Apr 4, 2011 at 4:34 AM, Michael Foord <mfo...@python.org> wrote: > >> On 04/04/2011 03:01, Richard Jones wrote: >> >>> On Mon, Apr 4, 2011 at 9:14 AM, "Martin v. Löwis"<mar...@v.loewis.de> >>> wrote: >>> >>>> That sounds awesome, especially for testing tools that have to interact >>>>> with pypi. >>>>> >>>> Actually, for testing tools, I think the requirements are quite >>>> different: you want concurrent runs of the same test case on multiple >>>> machines, so you need to make sure they don't collide. >>>> >>>> So for that use case, I rather suggest to setup an installation that >>>> allows dynamic creation of PyPI instances, each one good for only an >>>> hour. Their URLs would be test.pypi.python.org/<generatedid>. >>>> >>> I believe such tests should be run against a mocked PyPI so the test >>> has complete control over responses and error cases. >>> >> >> For integration level tests running against a mocked server is fine. >> However if pypi behaviour changes your mocks won't reflect that, so your >> tests could be passing but your code broken. For functional level tests you >> ideally want to run against a *real* implementation of pypi - preferably >> local (and under your control) but having a dev server to run against would >> be the next best thing. >> > > > Can you give an example of "pypi behaviour changes" - I am not gripping > what you mean. > > If the API changes, or even corner cases where your mocks behave > differently, then testing against a mock instead of the real thing will bite > you. > "Ahh, Mock." Got it. > For testing there are two competing principles that you have to balance - > don't depend on external services for your tests, but also do end-to-end > tests against *real* APIs / services / clients. > > For this particular case having a locally running pypi would be ideal. > Having configurable online instances is next best, a single online dev test > bed the third best. All are better than *just* testing against mocks. > > agreed. -- Carl K
_______________________________________________ pydotorg-www mailing list pydotorg-www@python.org http://mail.python.org/mailman/listinfo/pydotorg-www