StephanDollberg commented on PR #127: URL: https://github.com/apache/otava/pull/127#issuecomment-3922601702
> --ignore-direction This is essentially always the case now Sounds good > that a lot of the positive changes turn out to be bugs upon closer analysis 100% > whether there's a change point on the last comit or not, and ignore everything else. Right, that's a fine workaround though having it builtin would be convenient. > You want to compare two different versions/builds. A feature branch vs main branch, or the main branch against a stable branch (previous release). My opinion on this one is that it is not completely wrong approach, but this is not what Otava is designed to do. It is not change point detection, it is more like A/B testing or something. Right, we definitley use it for A/B testing. For us the advantage of that is that like this (A: dev/master/main/mergebase, B: PR) we don't have re-run A at all. We just use our existing master data and use otava/change-point-detection to find the correct data to compare to (which the is the point the PR/branch forked from master. Otherwise you might get differences based on changes that are just further ahead on master) We also don't really care about old changepoints for the PR as the latest version is all that matters. But sure we could do some filtering on our side for only the last dataset on the branch/PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
