Preinstalling dependencies may affect the dependency resolution, and we may end up testing a different configuration than a user would have after installing beam into a clean environment.
I do think pip uses cache, unless one specifies "--no-cache-dir". By default the cache is ~/.cache/pip. Looking up the log message in OP, we can see several "Using cached..." log entries. Not sure why futures was not fetched from cache or PyPi. Perhaps it is also a pip flake. I would be against wiping flakes under the rug by rerunning the whole suite after an error, but re-rerunning parts of the test environment set up, that are prone to environmental flakes, such as setupVirtualEnv seems reasonable. I agree with Udi that care should be taken to not overload Jenkins (e.g. retries should be limited)
