Re: + ipc-convert-ipcs_idr-to-xarray-update.patch added to -mm tree

2020-06-09 Thread Matthew Wilcox
On Fri, Jun 05, 2020 at 02:20:39PM -0700, Andrew Morton wrote: > On Fri, 5 Jun 2020 13:11:34 -0700 Matthew Wilcox wrote: > > > On Fri, Jun 05, 2020 at 03:58:48PM -0400, Qian Cai wrote: > > > This will trigger, > > > > > > [ 8853.759549] LTP: starting semget05 > > > [ 8867.257088] BUG: sleeping f

Re: + ipc-convert-ipcs_idr-to-xarray-update.patch added to -mm tree

2020-06-05 Thread Andrew Morton
On Fri, 5 Jun 2020 13:11:34 -0700 Matthew Wilcox wrote: > On Fri, Jun 05, 2020 at 03:58:48PM -0400, Qian Cai wrote: > > This will trigger, > > > > [ 8853.759549] LTP: starting semget05 > > [ 8867.257088] BUG: sleeping function called from invalid context at > > mm/slab.h:567 > > [ 8867.270259]

Re: + ipc-convert-ipcs_idr-to-xarray-update.patch added to -mm tree

2020-06-05 Thread Matthew Wilcox
On Fri, Jun 05, 2020 at 03:58:48PM -0400, Qian Cai wrote: > This will trigger, > > [ 8853.759549] LTP: starting semget05 > [ 8867.257088] BUG: sleeping function called from invalid context at > mm/slab.h:567 > [ 8867.270259] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 22556, > name: s

Re: + ipc-convert-ipcs_idr-to-xarray-update.patch added to -mm tree

2020-06-05 Thread Qian Cai
On Thu, Apr 23, 2020 at 06:47:53PM -0700, Andrew Morton wrote: > -- > From: "Matthew Wilcox (Oracle)" > Subject: ipc-convert-ipcs_idr-to-xarray-update > > implemant change suggested by Manfred > > Link: http://lkml.kernel.org/r/20200421183342.g

Re: [ipc/sem.c] 6394de3b86: BUG:kernel_NULL_pointer_dereference,address

2019-10-23 Thread Manfred Spraul
Hello, On 10/21/19 10:35 AM, kernel test robot wrote: FYI, we noticed the following commit (built with gcc-7): commit: 6394de3b868537a90dd9128607192b0e97109f6b ("[PATCH 4/5] ipc/sem.c: Document and update memory barriers") url: https://github.com/0day-ci/linux/commits/Manfred-Spraul/wake_q-Cl

Re: ipc/sem: Three function calls less in do_semtimedop()

2019-07-07 Thread Markus Elfring
>> +list_add_tail(&queue.list, >> + alter >> + ? (sma->complex_count >> +? &sma->pending_alter >> +: &curr->pending_alter) >> + : &curr->pending_const); > >

Re: ipc/msg: zalloc struct msg_queue when creating a new msq

2018-07-04 Thread Dmitry Vyukov
On Wed, Jul 4, 2018 at 4:08 PM, Manfred Spraul wrote: > Hello Dmitry, > On 07/04/2018 12:03 PM, Dmitry Vyukov wrote: >> >> On Wed, Jul 4, 2018 at 11:18 AM, Manfred Spraul >> wrote: >>> >>> >>> There are 2 relevant values: kern_ipc_perm.id and kern_ipc_perm.seq. >>> >>> For kern_ipc_perm.id, it is

Re: ipc/msg: zalloc struct msg_queue when creating a new msq

2018-07-04 Thread Manfred Spraul
Hello Dmitry, On 07/04/2018 12:03 PM, Dmitry Vyukov wrote: On Wed, Jul 4, 2018 at 11:18 AM, Manfred Spraul wrote: There are 2 relevant values: kern_ipc_perm.id and kern_ipc_perm.seq. For kern_ipc_perm.id, it is possible to move the access to the codepath that hold the lock. For kern_ipc_perm

Re: ipc/msg: zalloc struct msg_queue when creating a new msq

2018-07-04 Thread Dmitry Vyukov
On Wed, Jul 4, 2018 at 11:18 AM, Manfred Spraul wrote: > Hello together, > > On 06/25/2018 11:21 AM, Dmitry Vyukov wrote: >> >> On Sun, Jun 24, 2018 at 4:56 AM, Davidlohr Bueso >> wrote: >>> >>> The following splat was reported around the msg_queue structure >>> which can have uninitialized field

Re: ipc/msg: zalloc struct msg_queue when creating a new msq

2018-07-04 Thread Manfred Spraul
Hello together, On 06/25/2018 11:21 AM, Dmitry Vyukov wrote: On Sun, Jun 24, 2018 at 4:56 AM, Davidlohr Bueso wrote: The following splat was reported around the msg_queue structure which can have uninitialized fields left over after newque(). Future syscalls which make use of the msq id (now v

Re: ipc/msg: zalloc struct msg_queue when creating a new msq

2018-06-25 Thread Dmitry Vyukov
On Sun, Jun 24, 2018 at 4:56 AM, Davidlohr Bueso wrote: > The following splat was reported around the msg_queue structure > which can have uninitialized fields left over after newque(). > Future syscalls which make use of the msq id (now valid) can thus > make KMSAN complain because not all fields

Re: ipc: BUG: sem_unlock unlocks non-locked lock

2016-12-18 Thread Manfred Spraul
On 12/18/2016 05:29 PM, Davidlohr Bueso wrote: On Sun, 18 Dec 2016, Bueso wrote: On Fri, 16 Dec 2016, Dmitry Vyukov wrote: [ BUG: bad unlock balance detected! ] 4.9.0+ #89 Not tainted Thanks for the report, I can reproduce the issue as of (which I obviously should have tested with lockdep

Re: ipc: BUG: sem_unlock unlocks non-locked lock

2016-12-18 Thread Davidlohr Bueso
On Sun, 18 Dec 2016, Bueso wrote: On Fri, 16 Dec 2016, Dmitry Vyukov wrote: [ BUG: bad unlock balance detected! ] 4.9.0+ #89 Not tainted Thanks for the report, I can reproduce the issue as of (which I obviously should have tested with lockdep): 370b262c896 (ipc/sem: avoid idr tree lookup fo

Re: ipc: BUG: sem_unlock unlocks non-locked lock

2016-12-18 Thread Davidlohr Bueso
On Fri, 16 Dec 2016, Dmitry Vyukov wrote: [ BUG: bad unlock balance detected! ] 4.9.0+ #89 Not tainted Thanks for the report, I can reproduce the issue as of (which I obviously should have tested with lockdep): 370b262c896 (ipc/sem: avoid idr tree lookup for interrupted semop) I need to thin

Re: ipc/sem.c: change perform_atomic_semop parameters

2014-06-10 Thread Manfred Spraul
Hi Dave, On 06/10/2014 04:19 PM, Dave Jones wrote: On Fri, Jun 06, 2014 at 11:50:06PM +, Linux Kernel wrote: > Gitweb: http://git.kernel.org/linus/;a=commit;h=d198cd6d6d02d0a335af2deacb60816ebb4719d1 > Commit: d198cd6d6d02d0a335af2deacb60816ebb4719d1 > Parent: 2f2ed41dcaec

Re: ipc/sem.c: change perform_atomic_semop parameters

2014-06-10 Thread Dave Jones
On Fri, Jun 06, 2014 at 11:50:06PM +, Linux Kernel wrote: > Gitweb: > http://git.kernel.org/linus/;a=commit;h=d198cd6d6d02d0a335af2deacb60816ebb4719d1 > Commit: d198cd6d6d02d0a335af2deacb60816ebb4719d1 > Parent: 2f2ed41dcaec34f2d6f224aa84efcc5a9dd8d5c3 > Refname:refs/heads

Re: [ipc] 5769cf63: LTP semget02 TFAILs

2014-01-17 Thread Andrew Morton
On Fri, 17 Jan 2014 18:24:28 +0800 Fengguang Wu wrote: > Hi Davidlohr, > > We noticed LTP test failures > > ltp.msgget02.1.TFAIL > ltp.semget02.2.TFAIL > ltp.semget02.3.TFAIL > > and the first bad commit is > > commit 5769cf6355d87f63906b3e51887eff7017c39217 > Author:

Re: [ipc] 5769cf63: LTP semget02 TFAILs

2014-01-17 Thread Stephen Rothwell
[ Just adding Andrew to the cc list ] On Fri, 17 Jan 2014 18:24:28 +0800 Fengguang Wu wrote: > > Hi Davidlohr, > > We noticed LTP test failures > > ltp.msgget02.1.TFAIL > ltp.semget02.2.TFAIL > ltp.semget02.3.TFAIL > > and the first bad commit is > > commit 5769cf6355d

Re: ipc whitespace cleanups

2013-12-23 Thread Rafael Aquini
On Mon, Dec 23, 2013 at 11:19:44AM +0100, Manfred Spraul wrote: > Hi Rafael, > > Is anything else in your IPC todo list? > For me, it seems as if the locking updates have finally stabilized, > no more wrong return codes, races and whatever else. > (/me knocks on wood). > > Therefore I would like

Re: ipc msg now works (was Re: ipc-msg broken again on 3.11-rc7?)

2013-09-03 Thread Sedat Dilek
On Tue, Sep 3, 2013 at 12:32 PM, Vineet Gupta wrote: > On 09/03/2013 03:47 PM, Manfred Spraul wrote: >> Hi Vineet, >> >> On 09/03/2013 11:51 AM, Vineet Gupta wrote: >>> On 09/03/2013 02:53 PM, Manfred Spraul wrote: The access to msq->q_cbytes is not protected. Vineet, could you try

ipc msg now works (was Re: ipc-msg broken again on 3.11-rc7?)

2013-09-03 Thread Vineet Gupta
On 09/03/2013 03:47 PM, Manfred Spraul wrote: > Hi Vineet, > > On 09/03/2013 11:51 AM, Vineet Gupta wrote: >> On 09/03/2013 02:53 PM, Manfred Spraul wrote: >>> The access to msq->q_cbytes is not protected. >>> >>> Vineet, could you try to move the test for free space after ipc_lock? >>> I.e. the lo

Re: ipc-msg broken again on 3.11-rc7?

2013-09-03 Thread Manfred Spraul
Hi Vineet, On 09/03/2013 11:51 AM, Vineet Gupta wrote: On 09/03/2013 02:53 PM, Manfred Spraul wrote: The access to msq->q_cbytes is not protected. Vineet, could you try to move the test for free space after ipc_lock? I.e. the lock must not get dropped between testing for free space and enqueu

Re: ipc-msg broken again on 3.11-rc7?

2013-09-03 Thread Vineet Gupta
On 09/03/2013 02:53 PM, Manfred Spraul wrote: > On 09/03/2013 11:16 AM, Vineet Gupta wrote: >> On 09/03/2013 02:27 PM, Manfred Spraul wrote: >>> On 09/03/2013 10:44 AM, Vineet Gupta wrote: > b) Could you check that it is not just a performance regression? >Does ./msgctl08 1000 16 ha

Re: ipc-msg broken again on 3.11-rc7?

2013-09-03 Thread Manfred Spraul
On 09/03/2013 11:16 AM, Vineet Gupta wrote: On 09/03/2013 02:27 PM, Manfred Spraul wrote: On 09/03/2013 10:44 AM, Vineet Gupta wrote: b) Could you check that it is not just a performance regression? Does ./msgctl08 1000 16 hang, too? Nope that doesn't hang. The minimal configuration tha

Re: ipc-msg broken again on 3.11-rc7?

2013-09-03 Thread Vineet Gupta
On 09/03/2013 02:27 PM, Manfred Spraul wrote: > On 09/03/2013 10:44 AM, Vineet Gupta wrote: >>> b) Could you check that it is not just a performance regression? >>> Does ./msgctl08 1000 16 hang, too? >> Nope that doesn't hang. The minimal configuration that hangs reliably is >> msgctl >> 500

Re: ipc-msg broken again on 3.11-rc7?

2013-09-03 Thread Manfred Spraul
On 09/03/2013 10:44 AM, Vineet Gupta wrote: b) Could you check that it is not just a performance regression? Does ./msgctl08 1000 16 hang, too? Nope that doesn't hang. The minimal configuration that hangs reliably is msgctl 5 2 With this config there are 3 processes. ... 555 554

Re: ipc-msg broken again on 3.11-rc7?

2013-09-03 Thread Vineet Gupta
On 09/02/2013 09:59 PM, Manfred Spraul wrote: > Hi, > > [forgot to cc everyone, thus I'll summarize some mails...] > On 09/02/2013 06:58 AM, Vineet Gupta wrote: >> On 08/31/2013 11:20 PM, Linus Torvalds wrote: >>> Vineet, actual patch for what Davidlohr suggests attached. Can you try it? >>> >>>

Re: ipc-msg broken again on 3.11-rc7?

2013-09-03 Thread Sedat Dilek
On Tue, Sep 3, 2013 at 9:49 AM, Manfred Spraul wrote: > Hi Vineet, > > > On 09/03/2013 09:34 AM, Vineet Gupta wrote: > > However assuming we are going ahead with debugging this - can you please > confirm whether you see the issue on x86 as well as I have not tested that ? > I vaguely remember one

Re: ipc-msg broken again on 3.11-rc7?

2013-09-03 Thread Vineet Gupta
On 09/03/2013 12:46 PM, Sedat Dilek wrote: > Just FYI: > > Linux Testing Project (LTP) will do a new release in the 1st September week. > Some IPC test-suites were reworked. > Manfred can you look at them ("...msgctl08 uses one queue for each > thread pair."). > ( Might be worth to throw some words

Re: ipc-msg broken again on 3.11-rc7?

2013-09-03 Thread Sedat Dilek
On Mon, Sep 2, 2013 at 6:29 PM, Manfred Spraul wrote: > Hi, > > [forgot to cc everyone, thus I'll summarize some mails...] > > On 09/02/2013 06:58 AM, Vineet Gupta wrote: >> >> On 08/31/2013 11:20 PM, Linus Torvalds wrote: >>> >>> Vineet, actual patch for what Davidlohr suggests attached. Can you

Re: ipc-msg broken again on 3.11-rc7?

2013-09-02 Thread Manfred Spraul
Hi, [forgot to cc everyone, thus I'll summarize some mails...] On 09/02/2013 06:58 AM, Vineet Gupta wrote: On 08/31/2013 11:20 PM, Linus Torvalds wrote: Vineet, actual patch for what Davidlohr suggests attached. Can you try it? Linus Apologies for late in getting back to this -

Re: ipc-msg broken again on 3.11-rc7?

2013-09-01 Thread Vineet Gupta
On 08/31/2013 11:20 PM, Linus Torvalds wrote: > Vineet, actual patch for what Davidlohr suggests attached. Can you try it? > > Linus Apologies for late in getting back to this - I was away from my computer for a bit. Unfortunately, with a quick test, this patch doesn't help. FWIW, t

Re: ipc-msg broken again on 3.11-rc7?

2013-08-31 Thread Linus Torvalds
Vineet, actual patch for what Davidlohr suggests attached. Can you try it? Linus On Fri, Aug 30, 2013 at 9:31 AM, Davidlohr Bueso wrote: > > After a quick glance, I suspect that the problem might be because we > are calling security_msg_queue_msgsnd() without taking the lock. This >

Re: ipc-msg broken again on 3.11-rc7?

2013-08-30 Thread Davidlohr Bueso
> From: Vineet Gupta > Date: Fri, Aug 30, 2013 at 4:46 AM > Subject: Re: ipc-msg broken again on 3.11-rc7? > To: "sedat.di...@gmail.com" > Cc: linus Torvalds , Davidlohr Bueso > , linux-next , LKML > , Stephen Rothwell , > Andrew Morton , linux-mm , >

Re: ipc-msg broken again on 3.11-rc7?

2013-08-30 Thread Vineet Gupta
On 08/30/2013 01:57 PM, Sedat Dilek wrote: > On Fri, Aug 30, 2013 at 10:19 AM, Vineet Gupta wrote: >> Ping ? >> >> It seems 3.11 is pretty close to releasing but we stil have LTP msgctl08 >> causing a >> hang (atleast on ARC) for both linux-next 20130829 as well as Linus tree. >> >> So far, I hav

Re: ipc-msg broken again on 3.11-rc7? (was Re: linux-next: Tree for Jun 21 [ BROKEN ipc/ipc-msg ])

2013-08-30 Thread Sedat Dilek
On Fri, Aug 30, 2013 at 10:19 AM, Vineet Gupta wrote: > Ping ? > > It seems 3.11 is pretty close to releasing but we stil have LTP msgctl08 > causing a > hang (atleast on ARC) for both linux-next 20130829 as well as Linus tree. > > So far, I haven't seemed to have drawn attention of people involv

Re: ipc-msg broken again on 3.11-rc7? (was Re: linux-next: Tree for Jun 21 [ BROKEN ipc/ipc-msg ])

2013-08-30 Thread Vineet Gupta
Ping ? It seems 3.11 is pretty close to releasing but we stil have LTP msgctl08 causing a hang (atleast on ARC) for both linux-next 20130829 as well as Linus tree. So far, I haven't seemed to have drawn attention of people involved. -Vineet On 08/29/2013 01:22 PM, Sedat Dilek wrote: > On Thu,

Re: ipc-msg broken again on 3.11-rc7? (was Re: linux-next: Tree for Jun 21 [ BROKEN ipc/ipc-msg ])

2013-08-29 Thread Sedat Dilek
On Thu, Aug 29, 2013 at 9:21 AM, Vineet Gupta wrote: > On 08/29/2013 08:34 AM, Sedat Dilek wrote: >> On Wed, Aug 28, 2013 at 1:58 PM, Vineet Gupta >> wrote: >>> Hi David, >>> >>> On 06/26/2013 04:59 AM, Davidlohr Bueso wrote: On Tue, 2013-06-25 at 23:41 +0200, Sedat Dilek wrote: > On Tue

Re: ipc-msg broken again on 3.11-rc7? (was Re: linux-next: Tree for Jun 21 [ BROKEN ipc/ipc-msg ])

2013-08-29 Thread Vineet Gupta
On 08/29/2013 08:34 AM, Sedat Dilek wrote: > On Wed, Aug 28, 2013 at 1:58 PM, Vineet Gupta > wrote: >> Hi David, >> >> On 06/26/2013 04:59 AM, Davidlohr Bueso wrote: >>> On Tue, 2013-06-25 at 23:41 +0200, Sedat Dilek wrote: On Tue, Jun 25, 2013 at 10:33 PM, Davidlohr Bueso wrote: >

Re: ipc-msg broken again on 3.11-rc7? (was Re: linux-next: Tree for Jun 21 [ BROKEN ipc/ipc-msg ])

2013-08-28 Thread Sedat Dilek
On Wed, Aug 28, 2013 at 1:58 PM, Vineet Gupta wrote: > Hi David, > > On 06/26/2013 04:59 AM, Davidlohr Bueso wrote: >> On Tue, 2013-06-25 at 23:41 +0200, Sedat Dilek wrote: >>> On Tue, Jun 25, 2013 at 10:33 PM, Davidlohr Bueso >>> wrote: On Tue, 2013-06-25 at 18:10 +0200, Sedat Dilek wrote:

Re: [ipc,shm] BUG: lock held when returning to user space!

2013-08-01 Thread Davidlohr Bueso
On Sat, 2013-07-27 at 21:34 +0800, Fengguang Wu wrote: > On Sat, Jul 20, 2013 at 09:46:45AM -0700, Davidlohr Bueso wrote: > > On Sun, 2013-07-21 at 00:02 +0800, Xiaotian Feng wrote: > > > On Sat, Jul 20, 2013 at 9:13 PM, Fengguang Wu > > > wrote: > > > > Greetings, > > > > > > > > I got the below

Re: [ipc,shm] BUG: lock held when returning to user space!

2013-07-27 Thread Fengguang Wu
On Sat, Jul 20, 2013 at 09:46:45AM -0700, Davidlohr Bueso wrote: > On Sun, 2013-07-21 at 00:02 +0800, Xiaotian Feng wrote: > > On Sat, Jul 20, 2013 at 9:13 PM, Fengguang Wu > > wrote: > > > Greetings, > > > > > > I got the below dmesg and the first bad commit is > > > > > > commit c5d0282a0405b0a

Re: [ipc,shm] BUG: lock held when returning to user space!

2013-07-20 Thread Davidlohr Bueso
On Sun, 2013-07-21 at 00:02 +0800, Xiaotian Feng wrote: > On Sat, Jul 20, 2013 at 9:13 PM, Fengguang Wu wrote: > > Greetings, > > > > I got the below dmesg and the first bad commit is > > > > commit c5d0282a0405b0a81fa3390e4230e4cbb3ced7a2 > > Author: Davidlohr Bueso > > Date: Fri Jul 19 09:56:

Re: [ipc,shm] BUG: lock held when returning to user space!

2013-07-20 Thread Xiaotian Feng
On Sat, Jul 20, 2013 at 9:13 PM, Fengguang Wu wrote: > Greetings, > > I got the below dmesg and the first bad commit is > > commit c5d0282a0405b0a81fa3390e4230e4cbb3ced7a2 > Author: Davidlohr Bueso > Date: Fri Jul 19 09:56:58 2013 +1000 > > ipc,shm: shorten critical region for shmat > >

Re: [IPC] INFO: suspicious RCU usage.

2013-06-11 Thread Davidlohr Bueso
On Mon, 2013-06-10 at 15:56 -0700, Andrew Morton wrote: > On Thu, 06 Jun 2013 10:35:22 -0700 Davidlohr Bueso > wrote: > > > > [ 51.524946] > > > [ 51.525983] === > > > [ 51.532875] [ INFO: suspicious RCU usage. ] > > > [ 51.535385] 3.10.0-rc4-next-20130606 #6

Re: [IPC] INFO: suspicious RCU usage.

2013-06-10 Thread Andrew Morton
On Thu, 06 Jun 2013 10:35:22 -0700 Davidlohr Bueso wrote: > > [ 51.524946] > > [ 51.525983] === > > [ 51.532875] [ INFO: suspicious RCU usage. ] > > [ 51.535385] 3.10.0-rc4-next-20130606 #6 Not tainted > > [ 51.538304] --- > > [

Re: [IPC] INFO: suspicious RCU usage.

2013-06-06 Thread Davidlohr Bueso
Ccing Andrew. On Thu, 2013-06-06 at 21:25 +0800, Fengguang Wu wrote: > Greetings, > > I got the below dmesg and the first bad commit is > > commit 1f6587114a689a5d7fdfb0d4abc818117e3182a5 > Author: Davidlohr Bueso > Date: Thu Jun 6 10:41:56 2013 +1000 > > ipc: move rcu lock out of ipc_ad

Re: ipc/shm.c:494:18: warning: unused variable ‘hs’ [-Wunused-variable]

2013-05-08 Thread Borislav Petkov
On Wed, May 08, 2013 at 12:12:32PM -0400, Naoya Horiguchi wrote: > Thank you for the report. > I believe we can fix it with this one. > --- > From: Naoya Horiguchi > Date: Wed, 8 May 2013 11:48:01 -0400 > Subject: [PATCH] ipc/shm.c: don't use auto variable hs in newseg() > > This patch fixes "war

Re: ipc/shm.c:494:18: warning: unused variable ‘hs’ [-Wunused-variable]

2013-05-08 Thread Naoya Horiguchi
Hi Borislav, On Wed, May 08, 2013 at 04:34:11PM +0200, Borislav Petkov wrote: > Hi, > > looks like this warning came in with > af73e4d9506d3b797509f3c030e7dcd554f7d9c4 and fires in the > > # CONFIG_HUGETLB_PAGE is not set > > case because > > struct hstate *hs = hstate_sizelog((shmflg >>

Re: ipc,sem: sysv semaphore scalability

2013-05-06 Thread Jörn Engel
On Sat, 4 May 2013 20:12:45 +0200, Borislav Petkov wrote: > On Sat, May 04, 2013 at 11:55:58AM -0400, Jörn Engel wrote: > > Blockconsole currently lives here: > > https://git.kernel.org/cgit/linux/kernel/git/joern/bcon2.git/ > > Tja, if only that were upstream... Linus has a pull request. If he

Re: ipc,sem: sysv semaphore scalability

2013-05-04 Thread Borislav Petkov
On Sat, May 04, 2013 at 11:55:58AM -0400, Jörn Engel wrote: > Blockconsole currently lives here: > https://git.kernel.org/cgit/linux/kernel/git/joern/bcon2.git/ Tja, if only that were upstream... -- Regards/Gruss, Boris. Sent from a fat crate under my desk. Formatting is fine. -- -- To unsu

Re: ipc,sem: sysv semaphore scalability

2013-05-04 Thread Jörn Engel
On Sat, 23 March 2013 10:19:16 +0700, Emmanuel Benisty wrote: > > I could reproduce it but could you please let me know what would be > the right tools I should use to catch the original oops? > This is what I got but I doubt it will be helpful: > http://i.imgur.com/Mewi1hC.jpg You could use eith

Re: ipc,sem: sysv semaphore scalability

2013-05-03 Thread Peter Hurley
On 03/29/2013 03:01 PM, Dave Jones wrote: On Tue, Mar 26, 2013 at 12:43:09PM -0700, Andrew Morton wrote: > On Tue, 26 Mar 2013 15:28:52 -0400 Dave Jones wrote: > > > On Thu, Mar 21, 2013 at 02:10:58PM -0700, Andrew Morton wrote: > > > > > Whichever way we go, we should get a wiggle on

Re: ipc,sem: sysv semaphore scalability

2013-04-16 Thread Andrew Morton
On Tue, 26 Mar 2013 11:35:33 -0700 Andrew Morton wrote: > Do we need the locking at all? What does it actually do? > > sem_lock_and_putref(sma); > if (sma->sem_perm.deleted) { > sem_unlock(sma, -1); >

Re: ipc,sem: sysv semaphore scalability

2013-04-02 Thread Dave Jones
On Tue, Apr 02, 2013 at 03:53:01PM -0400, Sasha Levin wrote: > On 04/02/2013 01:52 PM, Linus Torvalds wrote: > > On Tue, Apr 2, 2013 at 9:08 AM, Sasha Levin wrote: > >> > >> By just playing with the 'msgsz' parameter with MSG_COPY set. > > > > Hmm. Looking closer, I suspect you're testing w

Re: ipc,sem: sysv semaphore scalability

2013-04-02 Thread Sasha Levin
On 04/02/2013 01:52 PM, Linus Torvalds wrote: > On Tue, Apr 2, 2013 at 9:08 AM, Sasha Levin wrote: >> >> By just playing with the 'msgsz' parameter with MSG_COPY set. > > Hmm. Looking closer, I suspect you're testing without commit > 88b9e456b164 ("ipc: don't allocate a copy larger than max"). Th

Re: ipc,sem: sysv semaphore scalability

2013-04-02 Thread Linus Torvalds
On Tue, Apr 2, 2013 at 9:08 AM, Sasha Levin wrote: > > By just playing with the 'msgsz' parameter with MSG_COPY set. Hmm. Looking closer, I suspect you're testing without commit 88b9e456b164 ("ipc: don't allocate a copy larger than max"). That should limit the size passed in to prepare_copy -> lo

Re: ipc,sem: sysv semaphore scalability

2013-04-02 Thread Linus Torvalds
On Tue, Apr 2, 2013 at 9:08 AM, Sasha Levin wrote: > > If you guys are already looking at this, the conversions between size_t, > long and int in the do_msgrcv/load_msg/alloc_msg code are a mess. You could > trigger anything from: Good catch. Let's just change the "(long)bufsz < 0" into "bufsz >

Re: ipc,sem: sysv semaphore scalability

2013-04-02 Thread Sasha Levin
On 03/29/2013 03:36 PM, Peter Hurley wrote: > On Fri, 2013-03-29 at 12:26 -0700, Linus Torvalds wrote: >> On Fri, Mar 29, 2013 at 12:06 PM, Dave Jones wrote: >>> >>> Here's an oops I just hit.. >>> >>> BUG: unable to handle kernel NULL pointer dereference at 000f >>> IP: [] testmsg.isr

Re: ipc,sem: sysv semaphore scalability

2013-04-01 Thread Stanislav Kinsbursky
29.03.2013 22:43, Linus Torvalds пишет: On Fri, Mar 29, 2013 at 9:17 AM, Dave Jones wrote: > >Now that I have that reverted, I'm not seeing msgrcv traces any more, but >I've started seeing this.. > >general protection fault: [#1] PREEMPT SMP DEBUG_PAGEALLOC Do you have CONFIG_CHECKPOINT_R

Re: ipc,sem: sysv semaphore scalability

2013-03-31 Thread Linus Torvalds
On Sun, Mar 31, 2013 at 6:45 AM, Rik van Riel wrote: > > Should we use "semid" here, like Linus suggested, instead of "un->semid"? As Davidlohr noted, in linux-next the rcu read-lock is held over the whole thing, so no, un->semid should be stable once "un" has been re-looked-up under the semaphor

Re: ipc,sem: sysv semaphore scalability

2013-03-31 Thread Emmanuel Benisty
Hi Davidlohr, On Sun, Mar 31, 2013 at 12:01 PM, Davidlohr Bueso wrote: > Specially dropping the rcu read lock before the continue statement > (sorry for not mentioning this in the last email). I was missing this indeed, thanks. Still the same issues however... I'll do some more testing on the sa

Re: ipc,sem: sysv semaphore scalability

2013-03-31 Thread Rik van Riel
On 03/31/2013 01:01 AM, Davidlohr Bueso wrote: diff --git a/ipc/sem.c b/ipc/sem.c index f257afe..74cedfe 100644 --- a/ipc/sem.c +++ b/ipc/sem.c @@ -1867,8 +1867,7 @@ void exit_sem(struct task_struct *tsk) struct sem_array *sma; struct sem_undo *un;

Re: ipc,sem: sysv semaphore scalability

2013-03-30 Thread Davidlohr Bueso
On Sat, 2013-03-30 at 11:33 +0700, Emmanuel Benisty wrote: > On Sat, Mar 30, 2013 at 10:46 AM, Linus Torvalds > wrote: > > On Fri, Mar 29, 2013 at 8:02 PM, Emmanuel Benisty > > wrote: > >> > >> Then I start building a random package and the problems start. They > >> may also happen without compi

Re: ipc,sem: sysv semaphore scalability

2013-03-30 Thread Emmanuel Benisty
Hi Linus, On Sun, Mar 31, 2013 at 12:22 AM, Linus Torvalds wrote: > On Fri, Mar 29, 2013 at 10:57 PM, Emmanuel Benisty > wrote: >> On Sat, Mar 30, 2013 at 12:10 PM, Linus Torvalds >>> >>> This came from the gcc build? >> >> yes, very early in the build process, IIRC this line was repeated a >>

Re: ipc,sem: sysv semaphore scalability

2013-03-30 Thread Linus Torvalds
On Fri, Mar 29, 2013 at 10:57 PM, Emmanuel Benisty wrote: > On Sat, Mar 30, 2013 at 12:10 PM, Linus Torvalds >> >> This came from the gcc build? > > yes, very early in the build process, IIRC this line was repeated a > few times and the build just stalled. Ok, we're bringing out the crazy hacks n

Re: ipc,sem: sysv semaphore scalability

2013-03-29 Thread Emmanuel Benisty
On Sat, Mar 30, 2013 at 12:10 PM, Linus Torvalds wrote: >> Another shot in >> the dark, I had this weird message when trying to build gcc: >> semop(2): encountered an error: Identifier removed > > This came from the gcc build? yes, very early in the build process, IIRC this line was repeated a fe

Re: ipc,sem: sysv semaphore scalability

2013-03-29 Thread Linus Torvalds
On Fri, Mar 29, 2013 at 9:33 PM, Emmanuel Benisty wrote: > > I just tried the 7 original patches + the 2 one liners from -next + > modified Linus' patch (attached) .. that patch looks fine. > on the top of 3.9-rc4 using > PREEMPT_NONE and after moving sem_lock(sma, NULL, -1) as explained > above

Re: ipc,sem: sysv semaphore scalability

2013-03-29 Thread Emmanuel Benisty
On Sat, Mar 30, 2013 at 10:46 AM, Linus Torvalds wrote: > On Fri, Mar 29, 2013 at 8:02 PM, Emmanuel Benisty wrote: >> >> Then I start building a random package and the problems start. They >> may also happen without compiling but this seems to trigger the bug >> quite quickly. > > I suspect it's

Re: ipc,sem: sysv semaphore scalability

2013-03-29 Thread Linus Torvalds
On Fri, Mar 29, 2013 at 8:02 PM, Emmanuel Benisty wrote: > > Then I start building a random package and the problems start. They > may also happen without compiling but this seems to trigger the bug > quite quickly. I suspect it's about preemption, and the build just results in enough scheduling

Re: ipc,sem: sysv semaphore scalability

2013-03-29 Thread Emmanuel Benisty
Hi Davidlohr, On Sat, Mar 30, 2013 at 9:08 AM, Davidlohr Bueso wrote: > Not sure which one liner you refer to, but, if you haven't already done > so, please try with these fixes (queued in linux-next): > > http://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git/commit/?id=a9cead0347283f3e

Re: ipc,sem: sysv semaphore scalability

2013-03-29 Thread Davidlohr Bueso
On Fri, 2013-03-29 at 19:09 -0700, Linus Torvalds wrote: > On Fri, Mar 29, 2013 at 6:36 PM, Emmanuel Benisty wrote: > > > > I had to slightly modify the patch since it wouldn't match the changes > > introduced by 7-7-ipc-sem-fine-grained-locking-for-semtimedop.patch, > > hope that was the right th

Re: ipc,sem: sysv semaphore scalability

2013-03-29 Thread Linus Torvalds
On Fri, Mar 29, 2013 at 6:36 PM, Emmanuel Benisty wrote: > > I had to slightly modify the patch since it wouldn't match the changes > introduced by 7-7-ipc-sem-fine-grained-locking-for-semtimedop.patch, > hope that was the right thing to do. So, what I tried was: original 7 > patches + the one lin

Re: ipc,sem: sysv semaphore scalability

2013-03-29 Thread Davidlohr Bueso
On Sat, 2013-03-30 at 08:36 +0700, Emmanuel Benisty wrote: > Hi Linus, > > On Sat, Mar 30, 2013 at 6:16 AM, Linus Torvalds > wrote: > > Emmanuel, can you try the attached patch? I think it applies cleanly > > on top of the scalability series too without any changes, but I didn't > > check if the

Re: ipc,sem: sysv semaphore scalability

2013-03-29 Thread Emmanuel Benisty
Hi Linus, On Sat, Mar 30, 2013 at 6:16 AM, Linus Torvalds wrote: > Emmanuel, can you try the attached patch? I think it applies cleanly > on top of the scalability series too without any changes, but I didn't > check if the patches perhaps changed some of the naming or something. I had to slight

Re: ipc,sem: sysv semaphore scalability

2013-03-29 Thread Linus Torvalds
On Fri, Mar 29, 2013 at 2:12 PM, Linus Torvalds wrote: > > I dunno. I'm still not sure this is triggerable, but it looks bad. But > both the semaphore case and the msg cases seem to be solvable by > moving the unlock down, and shm seem to have no getref/putref users to > race with, so this (whites

Re: ipc,sem: sysv semaphore scalability

2013-03-29 Thread Linus Torvalds
On Fri, Mar 29, 2013 at 1:41 PM, Linus Torvalds wrote: > > The alternative would be to make sure the thing is always locked (and > in a rcu-read-safe region) before putref/getref. The only place (apart > from the initial allocation, which doesn't matter, because nothing can > see if itf that path

Re: ipc,sem: sysv semaphore scalability

2013-03-29 Thread Linus Torvalds
On Fri, Mar 29, 2013 at 9:17 AM, Dave Jones wrote: > > Now that I have that reverted, I'm not seeing msgrcv traces any more, but > I've started seeing this.. > > general protection fault: [#1] PREEMPT SMP DEBUG_PAGEALLOC > RIP: 0010:[] [] free_msg+0x2b/0x40 > Call Trace: > [] freeque+0xcf/0

Re: ipc,sem: sysv semaphore scalability

2013-03-29 Thread Linus Torvalds
On Fri, Mar 29, 2013 at 12:33 PM, Peter Hurley wrote: > On Fri, 2013-03-29 at 11:43 -0700, Linus Torvalds wrote: >> I think I foud at least one bug in the MSG_COPY stuff: it leaks the >> "copy" allocation if >> >> mode == SEARCH_LESSEQUAL >> >> but maybe I'm misreading it. > > Yes, you're misr

Re: ipc,sem: sysv semaphore scalability

2013-03-29 Thread Peter Hurley
On Fri, 2013-03-29 at 12:26 -0700, Linus Torvalds wrote: > On Fri, Mar 29, 2013 at 12:06 PM, Dave Jones wrote: > > > > Here's an oops I just hit.. > > > > BUG: unable to handle kernel NULL pointer dereference at 000f > > IP: [] testmsg.isra.5+0x1a/0x60 > > Btw, looking at the code lea

Re: ipc,sem: sysv semaphore scalability

2013-03-29 Thread Peter Hurley
On Fri, 2013-03-29 at 11:43 -0700, Linus Torvalds wrote: > I think I foud at least one bug in the MSG_COPY stuff: it leaks the > "copy" allocation if > > mode == SEARCH_LESSEQUAL > > but maybe I'm misreading it. Yes, you're misreading it. copy_msg() returns the 'copy' address when copying is

Re: ipc,sem: sysv semaphore scalability

2013-03-29 Thread Linus Torvalds
On Fri, Mar 29, 2013 at 12:06 PM, Dave Jones wrote: > > Here's an oops I just hit.. > > BUG: unable to handle kernel NULL pointer dereference at 000f > IP: [] testmsg.isra.5+0x1a/0x60 Btw, looking at the code leading up to this, what the f*ck is wrong with the IPC stuff? It's using t

Re: ipc,sem: sysv semaphore scalability

2013-03-29 Thread Linus Torvalds
On Fri, Mar 29, 2013 at 12:06 PM, Dave Jones wrote: > > Btw, something that's really bothering me is just how much bogus > 'follow-on' spew we have lately. I'm not sure what changed, but it > seems to have gotten worse. .. we have many more sanity checks, and they tend to trigger in the case of u

Re: ipc,sem: sysv semaphore scalability

2013-03-29 Thread Dave Jones
On Fri, Mar 29, 2013 at 11:43:25AM -0700, Linus Torvalds wrote: > On Fri, Mar 29, 2013 at 9:17 AM, Dave Jones wrote: > > > > Now that I have that reverted, I'm not seeing msgrcv traces any more, but > > I've started seeing this.. > > > > general protection fault: [#1] PREEMPT SMP DEBUG_

Re: ipc,sem: sysv semaphore scalability

2013-03-29 Thread Dave Jones
On Tue, Mar 26, 2013 at 12:43:09PM -0700, Andrew Morton wrote: > On Tue, 26 Mar 2013 15:28:52 -0400 Dave Jones wrote: > > > On Thu, Mar 21, 2013 at 02:10:58PM -0700, Andrew Morton wrote: > > > > > Whichever way we go, we should get a wiggle on - this has been hanging > > > around for too

Re: ipc,sem: sysv semaphore scalability

2013-03-29 Thread Linus Torvalds
On Fri, Mar 29, 2013 at 9:17 AM, Dave Jones wrote: > > Now that I have that reverted, I'm not seeing msgrcv traces any more, but > I've started seeing this.. > > general protection fault: [#1] PREEMPT SMP DEBUG_PAGEALLOC Do you have CONFIG_CHECKPOINT_RESTORE enabled? Does it go away if you d

Re: ipc,sem: sysv semaphore scalability

2013-03-29 Thread Linus Torvalds
On Fri, Mar 29, 2013 at 11:04 AM, Dave Jones wrote: > > mainline. Your current tree. Ok, that's what I thought you were generally testing, just wanted to verify. The subject kind of implied otherwise.. Linus -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

Re: ipc,sem: sysv semaphore scalability

2013-03-29 Thread Dave Jones
On Fri, Mar 29, 2013 at 11:00:27AM -0700, Linus Torvalds wrote: > On Fri, Mar 29, 2013 at 9:17 AM, Dave Jones wrote: > > > > Now that I have that reverted, I'm not seeing msgrcv traces any more, but > > I've started seeing this.. > > > > general protection fault: [#1] PREEMPT SMP DEBUG_

Re: ipc,sem: sysv semaphore scalability

2013-03-29 Thread Linus Torvalds
On Fri, Mar 29, 2013 at 9:17 AM, Dave Jones wrote: > > Now that I have that reverted, I'm not seeing msgrcv traces any more, but > I've started seeing this.. > > general protection fault: [#1] PREEMPT SMP DEBUG_PAGEALLOC > > Looks like seg was already kfree'd. Just to clarify: is this you te

Re: ipc,sem: sysv semaphore scalability

2013-03-29 Thread Dave Jones
On Tue, Mar 26, 2013 at 12:43:09PM -0700, Andrew Morton wrote: > On Tue, 26 Mar 2013 15:28:52 -0400 Dave Jones wrote: > > > On Thu, Mar 21, 2013 at 02:10:58PM -0700, Andrew Morton wrote: > > > > > Whichever way we go, we should get a wiggle on - this has been hanging > > > around for too

Re: ipc,sem: sysv semaphore scalability

2013-03-26 Thread Andrew Morton
On Tue, 26 Mar 2013 15:28:52 -0400 Dave Jones wrote: > On Thu, Mar 21, 2013 at 02:10:58PM -0700, Andrew Morton wrote: > > > Whichever way we go, we should get a wiggle on - this has been hanging > > around for too long. Dave, do you have time to determine whether > > reverting 88b9e456b16497

Re: ipc,sem: sysv semaphore scalability

2013-03-26 Thread Dave Jones
On Thu, Mar 21, 2013 at 02:10:58PM -0700, Andrew Morton wrote: > Whichever way we go, we should get a wiggle on - this has been hanging > around for too long. Dave, do you have time to determine whether > reverting 88b9e456b1649722673ff ("ipc: don't allocate a copy larger > than max") fixes t

Re: ipc,sem: sysv semaphore scalability

2013-03-26 Thread Andrew Morton
On Tue, 26 Mar 2013 10:59:27 -0700 Davidlohr Bueso wrote: > On Mon, 2013-03-25 at 20:47 +0700, Emmanuel Benisty wrote: > > On Mon, Mar 25, 2013 at 12:10 AM, Linus Torvalds > > wrote: > > > And you never see this problem without Rik's patches? > > > > No, never. > > > > > Could you bisect > >

Re: ipc,sem: sysv semaphore scalability

2013-03-26 Thread Rik van Riel
On 03/26/2013 02:07 PM, Sasha Levin wrote: On 03/26/2013 01:51 PM, Davidlohr Bueso wrote: On Tue, 2013-03-26 at 13:33 -0400, Sasha Levin wrote: On 03/20/2013 03:55 PM, Rik van Riel wrote: This series makes the sysv semaphore code more scalable, by reducing the time the semaphore lock is held,

Re: ipc,sem: sysv semaphore scalability

2013-03-26 Thread Rik van Riel
On 03/26/2013 01:59 PM, Davidlohr Bueso wrote: From: Davidlohr Bueso Subject: [PATCH] ipc, sem: prevent possible deadlock In semctl_main(), when cmd == GETALL, we're locking sma->sem_perm.lock (through sem_lock_and_putref), yet after the conditional, we lock it again. Unlock sma right after ex

Re: ipc,sem: sysv semaphore scalability

2013-03-26 Thread Sasha Levin
On 03/20/2013 03:55 PM, Rik van Riel wrote: > This series makes the sysv semaphore code more scalable, > by reducing the time the semaphore lock is held, and making > the locking more scalable for semaphore arrays with multiple > semaphores. Hi Rik, Another issue that came up is: [ 96.347341]

Re: ipc,sem: sysv semaphore scalability

2013-03-26 Thread Sasha Levin
On 03/26/2013 01:51 PM, Davidlohr Bueso wrote: > On Tue, 2013-03-26 at 13:33 -0400, Sasha Levin wrote: >> On 03/20/2013 03:55 PM, Rik van Riel wrote: >>> This series makes the sysv semaphore code more scalable, >>> by reducing the time the semaphore lock is held, and making >>> the locking more sca

Re: ipc,sem: sysv semaphore scalability

2013-03-26 Thread Davidlohr Bueso
On Mon, 2013-03-25 at 20:47 +0700, Emmanuel Benisty wrote: > On Mon, Mar 25, 2013 at 12:10 AM, Linus Torvalds > wrote: > > And you never see this problem without Rik's patches? > > No, never. > > > Could you bisect > > *which* patch it starts with? Are the first four ones ok (the moving > > of t

Re: ipc,sem: sysv semaphore scalability

2013-03-26 Thread Paul E. McKenney
On Tue, Mar 26, 2013 at 01:33:07PM -0400, Sasha Levin wrote: > On 03/20/2013 03:55 PM, Rik van Riel wrote: > > This series makes the sysv semaphore code more scalable, > > by reducing the time the semaphore lock is held, and making > > the locking more scalable for semaphore arrays with multiple >

Re: ipc,sem: sysv semaphore scalability

2013-03-26 Thread Davidlohr Bueso
On Tue, 2013-03-26 at 13:33 -0400, Sasha Levin wrote: > On 03/20/2013 03:55 PM, Rik van Riel wrote: > > This series makes the sysv semaphore code more scalable, > > by reducing the time the semaphore lock is held, and making > > the locking more scalable for semaphore arrays with multiple > > semap

  1   2   >