> On Jun 12, 2017, at 5:55 PM, Pavol Vaskovic <[email protected]> wrote: > > > > On Tue, Jun 13, 2017 at 2:31 AM, Michael Gottesman <[email protected] > <mailto:[email protected]>> wrote: >> I don't think we can get more consistent test results just from re-running >> tests that were detected as changes in the first pass, as described in >> SR-4669 <https://bugs.swift.org/browse/SR-4669>, because that improves >> accuracy only for one side of the comparison - the branch. When the >> measurement error is with the baseline from the master, re-running the >> branch would not help. > > When we are benchmarking, we can always have access to the baseline compiler > by stashing the build directory. So we can always take more samples (in fact > when I was talking about re-running I always assumed we would). > > Well, if I understand correctly how the swift-CI builds perf-PR, then > switching between master and branch from Benchmark_Driver is not possible... > > Or are you thinking about manual benchmarking scenario? > > --Pavol
I was thinking (hoping) Benchmark_Driver would support this and we could ask for support from CI to call the driver that way. -Andy
_______________________________________________ swift-dev mailing list [email protected] https://lists.swift.org/mailman/listinfo/swift-dev
