On Tue, Jun 13, 2017 at 2:31 AM, Michael Gottesman <mgottes...@apple.com> wrote: > > I don't think we can get more consistent test results just from re-running > tests that were detected as changes in the first pass, as described in > SR-4669 <https://bugs.swift.org/browse/SR-4669>, because that improves > accuracy only for one side of the comparison - the branch. When the > measurement error is with the baseline from the master, re-running the > branch would not help. > > > When we are benchmarking, we can always have access to the baseline > compiler by stashing the build directory. So we can always take more > samples (in fact when I was talking about re-running I always assumed we > would). >
Well, if I understand correctly how the swift-CI builds perf-PR, then switching between master and branch from Benchmark_Driver is not possible... Or are you thinking about manual benchmarking scenario? --Pavol
_______________________________________________ swift-dev mailing list swift-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-dev
