> > This is a valid concern but since the tool remains under active > development, I would caution against making an assessment based on the > number of issues currently reported in the tracker. >
Fair point! I'll reserve judgement in that case then. On Wed, Dec 30, 2020 at 9:57 PM K. Alex Mills <k.alex.mi...@gmail.com> wrote: > On Wed, Dec 30, 2020, 10:34 PM Tyler Compton <xavi...@gmail.com> wrote: > >> This is a very interesting project! I went ahead and labeled a few of the >> issues. However, much as I hate to pour cold water on the idea, I have some >> doubts about how effective this approach can be. The githubvet bot has >> reported over ten thousand issues so far, which is quite a lot. >> > > This is a valid concern but since the tool remains under active > development, I would caution against making an assessment based on the > number of issues currently reported in the tracker. > > Part of the problem is that the current tracker contains issues from > multiple runs, since I've rescanned during testing, as I find and fixing > issues. > > There are still some things that can be done to bring down the > false-positive rate, but before addressing that I need to spend time on > some of the false-negatives that I am currently aware of. > > My gut feeling is that if code does exist that correctly relies on the >> current behavior, it would be extremely subtle. If we're going to get >> through all of these issues, we won't be able to give each example a >> complete deep dive which I think means we'll likely miss very subtle cases. >> Not that I have any better ideas :) >> > > Part of the hope has been that a large number of community participants > would help parallelize the effort and (perhaps) even get multiple pairs of > eyes on each issue. Whether that will work or not remains to be seen. > > I will also say that many of the issues I have looked at have seemed to be > classifiable rather quickly. OTOH, that could also be an indication that > the false-positive rate remains a little high. > > All that said, so far, the issues reported only focus on race-conditions > or incorrectly storing a reference to a range-loop pointer. Tim King has > just pointed out a few other concerns like pointer comparison and > finalizers for which analyzers have yet to be written. My perception is > that these two features aren't very widely used, so I wouldn't expect to > find a large number of instances like these. > > >> On Sun, Dec 20, 2020 at 12:38 PM K. Alex Mills <k.alex.mi...@gmail.com> >> wrote: >> >>> Hello Gophers! >>> >>> During a Q&A session at this year's GopherCon, some members of the Go >>> language team indicated a willingness to modify the behavior of range >>> loops <https://github.com/golang/go/issues/20733> *if* we can be >>> reasonably certain that the change will not cause incorrect behavior in >>> current programs. To make that determination, a large collection of >>> real-world go programs would need to be vetted. If we can find that, in >>> every case, modifying the compiler behavior does not lead to an undesirable >>> outcome, we will have strong evidence that the change may only have a small >>> impact. >>> >>> I've been working on a project to gather and crowdsource data for that >>> sort of analysis. It has reached a point where I think that it's time to >>> share it with the Go community. >>> >>> The GitHub Vet Project <https://github.com/github-vet> performs static >>> analysis on public Go repositories hosted on GitHub in order to better >>> understand the impact of this proposed language change >>> <https://github.com/golang/go/issues/20733>. It consists of two bots. >>> VetBot crawls GitHub to snippets of code it thinks could be interesting, it >>> reports it as an issue in a separate repository >>> <https://github.com/github-vet/rangeloop-pointer-findings>, for humans >>> to analyze via crowdsourcing. TrackBot >>> <https://github.com/github-vet/bots/tree/main/cmd/track-bot> manages >>> the crowdsourcing workflow. >>> >>> At this point, all of the features I think are necessary for the >>> project's success have been implemented. But I am only one Gopher, and so I >>> would like to a) make the community aware of the project and b) ask the >>> community to help review it in three ways: >>> >>> 1) Test out TrackBot and the project instructions by actually >>> crowdsourcing through the issues found so far >>> <https://github.com/github-vet/rangeloop-pointer-findings>. >>> 2) Review the output >>> <https://github.com/github-vet/rangeloop-pointer-findings/issues> and >>> raise concern if anything that I claim ought to be excluded >>> <https://github.com/github-vet/bots/tree/main/cmd/vet-bot#2-run-static-analysis> >>> is somehow making it through. >>> 3) Static analysis experts: review the analyzers >>> <https://github.com/github-vet/bots/tree/main/cmd/vet-bot> and >>> ruthlessly question anything that could lead to a false negative. I don't >>> think anything exists, but one pair of eyes is often wrong, and there are >>> many folks on this list with (much) more expertise than me. >>> >>> One final thing. The crowd-sourcing workflow I've designed relies on the >>> opinion of experts to help estimate the reliability of the community >>> assessment. In order for that to work, I need the help of some Go experts >>> who are willing to commit some time to reviewing the findings. If you'd >>> like to participate in this way, first, read the TrackBot documentation >>> <https://github.com/github-vet/bots/tree/main/cmd/track-bot> to >>> understand what being an expert entails. If you're still interested, email >>> me with the subject line 'GitHub Vet Expert' and include your GitHub >>> username and a brief outline of your experience with Go. >>> >>> It's my hope that this project can provide some data that can help to >>> move Go forward. To that end, I'm also interested in any and all feedback >>> and suggestions. Contributions are also welcome >>> <https://github.com/github-vet/bots/issues>. >>> >>> Thanks for reading, >>> >>> K. Alex Mills, Ph.D >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "golang-nuts" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to golang-nuts+unsubscr...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/golang-nuts/CALJzkY86ZgQMQBqiAyDOKHHionjLZJbB6f%2BEDcMp82ZaPCKmXw%40mail.gmail.com >>> <https://groups.google.com/d/msgid/golang-nuts/CALJzkY86ZgQMQBqiAyDOKHHionjLZJbB6f%2BEDcMp82ZaPCKmXw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAA%3DXfu0rdbJ7gA9K%3DLzLKs%3DLCUtOW6p3USJm%2BaT3DrJBoZYe8Q%40mail.gmail.com.
