Re: use-after-free in __perf_install_in_context

2016-01-05 Thread Peter Zijlstra
On Thu, Dec 31, 2015 at 06:15:41PM +0100, Dmitry Vyukov wrote: > On Thu, Dec 17, 2015 at 3:43 PM, Peter Zijlstra wrote: > > On Thu, Dec 17, 2015 at 03:35:32PM +0100, Dmitry Vyukov wrote: > >> In short, I did not see use-after-frees but perf_event_open fuzzing > >> started hanging VMs very frequent

Re: use-after-free in __perf_install_in_context

2015-12-31 Thread Dmitry Vyukov
On Thu, Dec 17, 2015 at 3:43 PM, Peter Zijlstra wrote: > On Thu, Dec 17, 2015 at 03:35:32PM +0100, Dmitry Vyukov wrote: >> In short, I did not see use-after-frees but perf_event_open fuzzing >> started hanging VMs very frequently, so testing is inconclusive. > > Right, I'll get back to staring at

Re: use-after-free in __perf_install_in_context

2015-12-17 Thread Peter Zijlstra
On Thu, Dec 17, 2015 at 03:35:32PM +0100, Dmitry Vyukov wrote: > In short, I did not see use-after-frees but perf_event_open fuzzing > started hanging VMs very frequently, so testing is inconclusive. Right, I'll get back to staring at that one. Running your test case on bare metal got my machine

Re: use-after-free in __perf_install_in_context

2015-12-17 Thread Dmitry Vyukov
On Thu, Dec 17, 2015 at 3:28 PM, Peter Zijlstra wrote: >> > >> I'm not sure I can explain your problem with this, but I figure its >> > >> worth a try. >> > > >> > > Did it make a difference? >> > > >> > > Also, does the syzkaller thing do cpu-hotplug during its runs? >> > >> > See my email on Dec

Re: use-after-free in __perf_install_in_context

2015-12-17 Thread Peter Zijlstra
On Thu, Dec 17, 2015 at 03:26:17PM +0100, Peter Zijlstra wrote: > On Thu, Dec 17, 2015 at 03:08:49PM +0100, Dmitry Vyukov wrote: > > On Thu, Dec 17, 2015 at 3:06 PM, Peter Zijlstra > > wrote: > > > On Thu, Dec 10, 2015 at 08:57:40PM +0100, Peter Zijlstra wrote: > > >> I'm not sure I can explain y

Re: use-after-free in __perf_install_in_context

2015-12-17 Thread Peter Zijlstra
On Thu, Dec 17, 2015 at 03:08:49PM +0100, Dmitry Vyukov wrote: > On Thu, Dec 17, 2015 at 3:06 PM, Peter Zijlstra wrote: > > On Thu, Dec 10, 2015 at 08:57:40PM +0100, Peter Zijlstra wrote: > >> I'm not sure I can explain your problem with this, but I figure its > >> worth a try. > > > > Did it make

Re: use-after-free in __perf_install_in_context

2015-12-17 Thread Dmitry Vyukov
On Thu, Dec 17, 2015 at 3:06 PM, Peter Zijlstra wrote: > On Thu, Dec 10, 2015 at 08:57:40PM +0100, Peter Zijlstra wrote: >> I'm not sure I can explain your problem with this, but I figure its >> worth a try. > > Did it make a difference? > > Also, does the syzkaller thing do cpu-hotplug during its

Re: use-after-free in __perf_install_in_context

2015-12-17 Thread Peter Zijlstra
On Thu, Dec 10, 2015 at 08:57:40PM +0100, Peter Zijlstra wrote: > I'm not sure I can explain your problem with this, but I figure its > worth a try. Did it make a difference? Also, does the syzkaller thing do cpu-hotplug during its runs? > --- > kernel/events/core.c | 15 +-- > 1 fi

Re: use-after-free in __perf_install_in_context

2015-12-15 Thread Dmitry Vyukov
On Fri, Dec 11, 2015 at 9:14 AM, Ingo Molnar wrote: > > * Alexei Starovoitov wrote: > >> On Thu, Dec 10, 2015 at 10:02:51AM +0100, Peter Zijlstra wrote: >> > On Wed, Dec 09, 2015 at 07:54:35PM -0800, Alexei Starovoitov wrote: >> > > Freeing memory is a requirement regardless. >> > > Even when ker

Re: use-after-free in __perf_install_in_context

2015-12-15 Thread Dmitry Vyukov
On Thu, Dec 10, 2015 at 8:57 PM, Peter Zijlstra wrote: > On Tue, Dec 08, 2015 at 08:14:58PM +0100, Dmitry Vyukov wrote: >> Tested with your patches. >> The additional WARNING does not fire. >> For the rcu stacks, I had to change two more 2's to TRACK_NR and also >> moved memorization from call_rcu

Re: use-after-free in __perf_install_in_context

2015-12-11 Thread Ingo Molnar
* Alexei Starovoitov wrote: > On Thu, Dec 10, 2015 at 10:02:51AM +0100, Peter Zijlstra wrote: > > On Wed, Dec 09, 2015 at 07:54:35PM -0800, Alexei Starovoitov wrote: > > > Freeing memory is a requirement regardless. > > > Even when kernel running with kasan, there must be a way to stop > > > sta

Re: use-after-free in __perf_install_in_context

2015-12-10 Thread Peter Zijlstra
On Tue, Dec 08, 2015 at 08:14:58PM +0100, Dmitry Vyukov wrote: > Tested with your patches. > The additional WARNING does not fire. > For the rcu stacks, I had to change two more 2's to TRACK_NR and also > moved memorization from call_rcu to __call_rcu, but now it is working. > Two reports with indi

Re: use-after-free in __perf_install_in_context

2015-12-10 Thread Alexei Starovoitov
On Thu, Dec 10, 2015 at 10:02:51AM +0100, Peter Zijlstra wrote: > On Wed, Dec 09, 2015 at 07:54:35PM -0800, Alexei Starovoitov wrote: > > Freeing memory is a requirement regardless. > > Even when kernel running with kasan, there must be a way to stop > > stack collection and free that memory. > > Y

Re: use-after-free in __perf_install_in_context

2015-12-10 Thread Peter Zijlstra
On Wed, Dec 09, 2015 at 07:54:35PM -0800, Alexei Starovoitov wrote: > Freeing memory is a requirement regardless. > Even when kernel running with kasan, there must be a way to stop > stack collection and free that memory. > You cannot treat kernel as your test program or 'device under test'. Relax

Re: use-after-free in __perf_install_in_context

2015-12-09 Thread Alexei Starovoitov
On Wed, Dec 09, 2015 at 10:17:17AM +0100, Dmitry Vyukov wrote: > > We would happily share this code with other subsystems, or even better > reuse an existing solutions. But to the best of my knowledge there is > no such existing solution, and I still know basically nothing about > what you were ha

Re: use-after-free in __perf_install_in_context

2015-12-09 Thread Dmitry Vyukov
On Tue, Dec 8, 2015 at 8:56 PM, Alexei Starovoitov wrote: > On Tue, Dec 08, 2015 at 07:35:20PM +0100, Dmitry Vyukov wrote: >> On Tue, Dec 8, 2015 at 7:05 PM, Alexei Starovoitov >> wrote: >> > On Tue, Dec 08, 2015 at 06:56:23PM +0100, Dmitry Vyukov wrote: >> >> On Tue, Dec 8, 2015 at 6:54 PM, Alex

Re: use-after-free in __perf_install_in_context

2015-12-09 Thread Peter Zijlstra
On Tue, Dec 08, 2015 at 07:57:38PM +0100, Ingo Molnar wrote: > Btw., could we add more redundancy / debug code to the refcounting code? It > seems > to be a frequent source of very hard to find/fix races/bugs - so it should be > ripe > for some extra debug infrastructure ... I'll try, but its

Re: use-after-free in __perf_install_in_context

2015-12-08 Thread Alexei Starovoitov
On Tue, Dec 08, 2015 at 07:35:20PM +0100, Dmitry Vyukov wrote: > On Tue, Dec 8, 2015 at 7:05 PM, Alexei Starovoitov > wrote: > > On Tue, Dec 08, 2015 at 06:56:23PM +0100, Dmitry Vyukov wrote: > >> On Tue, Dec 8, 2015 at 6:54 PM, Alexei Starovoitov > >> wrote: > >> > On Tue, Dec 08, 2015 at 05:12:

Re: use-after-free in __perf_install_in_context

2015-12-08 Thread Dmitry Vyukov
On Tue, Dec 8, 2015 at 5:44 PM, Peter Zijlstra wrote: > On Mon, Dec 07, 2015 at 05:09:21PM +0100, Dmitry Vyukov wrote: >> If your audit does not give any results, can you give me a patch that >> prints rcu callback submission stacks in KASAN reports? > > Just because my brain is fried for today, I

Re: use-after-free in __perf_install_in_context

2015-12-08 Thread Ingo Molnar
* Peter Zijlstra wrote: > On Fri, Dec 04, 2015 at 09:04:35PM +0100, Dmitry Vyukov wrote: > > Hello, > > > > While running syzkaller fuzzer I am seeing lots of the following > > use-after-free reports. Unfortunately all my numerous attempts to > > reproduce them in a controlled environment faile

Re: use-after-free in __perf_install_in_context

2015-12-08 Thread Dmitry Vyukov
On Tue, Dec 8, 2015 at 7:05 PM, Alexei Starovoitov wrote: > On Tue, Dec 08, 2015 at 06:56:23PM +0100, Dmitry Vyukov wrote: >> On Tue, Dec 8, 2015 at 6:54 PM, Alexei Starovoitov >> wrote: >> > On Tue, Dec 08, 2015 at 05:12:04PM +0100, Dmitry Vyukov wrote: >> >> On Tue, Dec 8, 2015 at 4:24 AM, Alex

Re: use-after-free in __perf_install_in_context

2015-12-08 Thread Alexei Starovoitov
On Tue, Dec 08, 2015 at 06:56:23PM +0100, Dmitry Vyukov wrote: > On Tue, Dec 8, 2015 at 6:54 PM, Alexei Starovoitov > wrote: > > On Tue, Dec 08, 2015 at 05:12:04PM +0100, Dmitry Vyukov wrote: > >> On Tue, Dec 8, 2015 at 4:24 AM, Alexei Starovoitov > >> wrote: > >> > On Mon, Dec 07, 2015 at 05:09:

Re: use-after-free in __perf_install_in_context

2015-12-08 Thread Dmitry Vyukov
On Tue, Dec 8, 2015 at 6:54 PM, Alexei Starovoitov wrote: > On Tue, Dec 08, 2015 at 05:12:04PM +0100, Dmitry Vyukov wrote: >> On Tue, Dec 8, 2015 at 4:24 AM, Alexei Starovoitov >> wrote: >> > On Mon, Dec 07, 2015 at 05:09:21PM +0100, Dmitry Vyukov wrote: >> >> > So it would be _awesome_ if we cou

Re: use-after-free in __perf_install_in_context

2015-12-08 Thread Alexei Starovoitov
On Tue, Dec 08, 2015 at 05:12:04PM +0100, Dmitry Vyukov wrote: > On Tue, Dec 8, 2015 at 4:24 AM, Alexei Starovoitov > wrote: > > On Mon, Dec 07, 2015 at 05:09:21PM +0100, Dmitry Vyukov wrote: > >> > So it would be _awesome_ if we could somehow extend this callchain to > >> > include the site that

Re: use-after-free in __perf_install_in_context

2015-12-08 Thread Dmitry Vyukov
On Tue, Dec 8, 2015 at 5:27 PM, Peter Zijlstra wrote: > On Fri, Dec 04, 2015 at 09:04:35PM +0100, Dmitry Vyukov wrote: > >> I was able to reproduce it by >> restricting syscalls only to perf_event_open, perf ioctls and bpf >> syscall. > > Does that include cpu hotplug? I have CONFIG_HOTPLUG_CPU e

Re: use-after-free in __perf_install_in_context

2015-12-08 Thread Peter Zijlstra
On Mon, Dec 07, 2015 at 05:09:21PM +0100, Dmitry Vyukov wrote: > If your audit does not give any results, can you give me a patch that > prints rcu callback submission stacks in KASAN reports? Just because my brain is fried for today, I figured I'd give it a go. Completely untested.. --- includ

Re: use-after-free in __perf_install_in_context

2015-12-08 Thread Peter Zijlstra
On Fri, Dec 04, 2015 at 09:04:35PM +0100, Dmitry Vyukov wrote: > I was able to reproduce it by > restricting syscalls only to perf_event_open, perf ioctls and bpf > syscall. Does that include cpu hotplug? Sasha just reminded me there is a swevent use-after-free issue on hotplug. -- To unsubscrib

Re: use-after-free in __perf_install_in_context

2015-12-08 Thread Peter Zijlstra
On Fri, Dec 04, 2015 at 09:04:35PM +0100, Dmitry Vyukov wrote: > Hello, > > While running syzkaller fuzzer I am seeing lots of the following > use-after-free reports. Unfortunately all my numerous attempts to > reproduce them in a controlled environment failed. They pop up during > fuzzing periodi

Re: use-after-free in __perf_install_in_context

2015-12-08 Thread Dmitry Vyukov
On Tue, Dec 8, 2015 at 4:24 AM, Alexei Starovoitov wrote: > On Mon, Dec 07, 2015 at 05:09:21PM +0100, Dmitry Vyukov wrote: >> > So it would be _awesome_ if we could somehow extend this callchain to >> > include the site that calls call_rcu(). >> >> We have a patch for KASAN in works that adds so-c

Re: use-after-free in __perf_install_in_context

2015-12-07 Thread Alexei Starovoitov
On Mon, Dec 07, 2015 at 05:09:21PM +0100, Dmitry Vyukov wrote: > > So it would be _awesome_ if we could somehow extend this callchain to > > include the site that calls call_rcu(). > > We have a patch for KASAN in works that adds so-called stack depot > which allows to map a stack trace onto uint3

Re: use-after-free in __perf_install_in_context

2015-12-07 Thread Dmitry Vyukov
On Mon, Dec 7, 2015 at 4:36 PM, Peter Zijlstra wrote: > On Fri, Dec 04, 2015 at 09:04:35PM +0100, Dmitry Vyukov wrote: >> == >> BUG: KASAN: use-after-free in __lock_acquire+0x4e99/0x5100 at addr >> 880038706e60 >> Read of size 8 b

Re: use-after-free in __perf_install_in_context

2015-12-07 Thread Peter Zijlstra
On Fri, Dec 04, 2015 at 09:04:35PM +0100, Dmitry Vyukov wrote: > == > BUG: KASAN: use-after-free in __lock_acquire+0x4e99/0x5100 at addr > 880038706e60 > Read of size 8 by task syzkaller_execu/6513 > ===

Re: use-after-free in __perf_install_in_context

2015-12-07 Thread Dmitry Vyukov
On Mon, Dec 7, 2015 at 12:06 PM, Dmitry Vyukov wrote: > On Mon, Dec 7, 2015 at 12:04 PM, Dmitry Vyukov wrote: >> On Fri, Dec 4, 2015 at 10:00 PM, Dmitry Vyukov wrote: >>> On Fri, Dec 4, 2015 at 9:32 PM, Alexei Starovoitov >>> wrote: On Fri, Dec 04, 2015 at 09:04:35PM +0100, Dmitry Vyukov w

Re: use-after-free in __perf_install_in_context

2015-12-07 Thread Dmitry Vyukov
On Mon, Dec 7, 2015 at 12:04 PM, Dmitry Vyukov wrote: > On Fri, Dec 4, 2015 at 10:00 PM, Dmitry Vyukov wrote: >> On Fri, Dec 4, 2015 at 9:32 PM, Alexei Starovoitov >> wrote: >>> On Fri, Dec 04, 2015 at 09:04:35PM +0100, Dmitry Vyukov wrote: Hello, While running syzkaller fuzzer I

Re: use-after-free in __perf_install_in_context

2015-12-07 Thread Dmitry Vyukov
On Fri, Dec 4, 2015 at 10:00 PM, Dmitry Vyukov wrote: > On Fri, Dec 4, 2015 at 9:32 PM, Alexei Starovoitov > wrote: >> On Fri, Dec 04, 2015 at 09:04:35PM +0100, Dmitry Vyukov wrote: >>> Hello, >>> >>> While running syzkaller fuzzer I am seeing lots of the following >>> use-after-free reports. Unf

Re: use-after-free in __perf_install_in_context

2015-12-04 Thread Dmitry Vyukov
On Fri, Dec 4, 2015 at 9:32 PM, Alexei Starovoitov wrote: > On Fri, Dec 04, 2015 at 09:04:35PM +0100, Dmitry Vyukov wrote: >> Hello, >> >> While running syzkaller fuzzer I am seeing lots of the following >> use-after-free reports. Unfortunately all my numerous attempts to >> reproduce them in a co

Re: use-after-free in __perf_install_in_context

2015-12-04 Thread Alexei Starovoitov
On Fri, Dec 04, 2015 at 09:04:35PM +0100, Dmitry Vyukov wrote: > Hello, > > While running syzkaller fuzzer I am seeing lots of the following > use-after-free reports. Unfortunately all my numerous attempts to > reproduce them in a controlled environment failed. They pop up during > fuzzing periodi

use-after-free in __perf_install_in_context

2015-12-04 Thread Dmitry Vyukov
Hello, While running syzkaller fuzzer I am seeing lots of the following use-after-free reports. Unfortunately all my numerous attempts to reproduce them in a controlled environment failed. They pop up during fuzzing periodically (once in several hours in a single VM), but whenever I try to stress-