henrikingo commented on PR #127: URL: https://github.com/apache/otava/pull/127#issuecomment-3920606782
Partial answer, as I never really used regressions myself, but otoh I'm probably the one with strongest/most confident opinions on these topics, so you can always blame me for bad decisions... `--ignore-direction` This is essentially always the case now (and always was in `analyze`). If you wanted to get only negative change points, you'd have to filter out positive changes by checking the direction field. Like you, I strongly believe that a lot of the positive changes turn out to be bugs upon closer analysis, so we should not even allow the option of ignoring them. `--last 1` If analyze doesn't have this, it could be added. In nyrkio we use otava in PRs, and it works great. We simply run it normally, then check whether there's a change point on the last comit or not, and ignore everything else. `--since-commit` This can be broken down into two use cases: * You want to compare two different versions/builds. A feature branch vs main branch, or the main branch against a stable branch (previous release). My opinion on this one is that it is not completely wrong approach, but this is not what Otava is designed to do. It is not change point detection, it is more like A/B testing or something. * Alternative syntax to --last N, IMO this can be supported if there's demand. It is just a convenience method to filter out some of the results, jst like last 1 or min threshold. But note that comparing against the merge-base is actually not necessarily what you want to do: In a feature branch, it is not necessarily wrong or a problem to find change points. The question is just have you fixed or otherwise addressed all of them. So using --since or --last with a requirement/expectation that no change points should be found, isn't correct in general case.' -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
