On Wed, Jan 09, 2019 at 01:29:02PM +0100, Dmitry Vyukov wrote: > On Wed, Jan 9, 2019 at 1:11 PM Andrea Parri > <andrea.pa...@amarulasolutions.com> wrote: > > > > On Wed, Jan 09, 2019 at 12:55:27PM +0100, Dmitry Vyukov wrote: > > > On Wed, Jan 9, 2019 at 12:24 PM Andrea Parri > > > <andrea.pa...@amarulasolutions.com> wrote: > > > > > > > > On Tue, Jan 08, 2019 at 04:36:46PM -0800, Anatol Pomozov wrote: > > > > > Hello > > > > > > > > > > On Tue, Jan 8, 2019 at 4:02 PM Andrea Parri > > > > > <andrea.pa...@amarulasolutions.com> wrote: > > > > > > > > > > > > Hi Anatol, > > > > > > > > > > > > On Tue, Jan 08, 2019 at 11:33:39AM -0800, Anatol Pomozov wrote: > > > > > > > Hello folks, > > > > > > > > > > > > > > A bit of context what I am doing. I am trying to port KTSAN > > > > > > > (Kernel > > > > > > > Thread Sanitizer) tool to v4.20. That tool tracks shared data > > > > > > > usage > > > > > > > and makes sure it is accessed in a thread-safe manner. > > > > > > > > > > > > Interesting! FYI, some LKMM's maintainers (Paul included) had and > > > > > > continued to have some "fun" discussing topics related to "thread- > > > > > > safe memory accesses": I'm sure that they'll be very interested in > > > > > > such work of yours and eager to discuss your results. > > > > > > > > > > Thread Sanitizer is a great tool to find thread-safety issues with > > > > > user-space code. The tool been developed by a team of smart people > > > > > from Google [1]. > > > > > > > > > > KTSAN is an attempt to bring the same ideas to Linux kernel [2]. A > > > > > bunch of work been done there but the project is still at > > > > > proof-of-concept point. > > > > > > > > Yes, I have been aware of these tools since at least ;-) > > > > > > > > https://groups.google.com/forum/#!msg/ktsan/bVZ1c6H2NE0/Dxrw55bfBAAJ > > > > > > > > > > > > > > > > > > I am not a part of Google's dynamic tools team. But I've decided to > > > > > pick something to do during the New Year holidays so started porting > > > > > KTSAN from v4.2 to v4.20. The work is "almost completed" but I need to > > > > > fix a few crashes [3]. > > > > > > > > I guess my first reaction would remain > > > > > > > > "it's kind of hard (to use an euphemism) to review 7,582 additions > > > > or so for a data race detector without a clear/an accepted (by the > > > > community) notion of data race..." > > > > > > Tsan's notion of a data race is basically the C/C++'s notion: > > > concurrent/unsynchronized non-atomic access in different threads at > > > least one of which is a write. > > > > Yeah, I think that this notion needs to be detailed, discussed, > > documented, and discussed again. ;-) > > > > > > > Tremendous (for such a project) benefits of automatic data race > > > detection is a good motivation to finally agree on and accept a > > > practically useful notion of a data race. > > > > Agreed. > > While having a 100% formal definition of a data race upfront would be > useful, I don't think this is a hard requirement for deployment of > KTSAN. What I think is required is: > 1. Agree that the overall direction is right. > 2. Agree that we want to enable data race detection and resolve > problems as they appear in a practical manner (rather than block whole > effort on every small thing). > We deployed TSAN in user-space in much larger code bases than kernel, > and while we had the C/C++ formal definition of a data race, practical > and legacy matters were similar to that of the kernel (lots of legacy > code, different opinions, etc). Doing both things in tandem (defining > a memory model and deploying a data race detector) can actually have > benefits as a race detector may point to under-defined or > impractically defined areas, and will otherwise help to validate that > the model works and is useful. > KTSAN is not fixed as well. We adopted it as we gathered more > knowledge and understanding of the kernel. So it's not that we have to > commit to something upfront.
In any case, there might well be some differences in approach between KTSAN and LKMM due to input size differences: One would expect LKMM to be able to tolerate a more computationally intensive definition as a consequence of KTSAN's ability to process much larger code bases. But I nevertheless believe that it would be good to have these differences be a matter of conscious choice rather than a matter of chance. ;-) My guess is that LKMM picks its starting point (which might take some additional time), then KTSAN critiques it, and then we work out what differences should result in a change to one or the other (or both) and which differences are inherent in the different workloads that LKMM and KTSAN are presented with. Seem reasonable? Thanx, Paul