Re: [PATCH 0/3 v3] dcache: make it more scalable on large system

2013-06-05 Thread Dave Chinner
On Wed, May 29, 2013 at 10:37:00PM +0200, Andi Kleen wrote: > > As Dave said before, is the last path component sufficient? Or how > > about an inode number? > > Neither works, the profiler needs to find the file and read it. > > inode searching would be incredible expensive, unless the file sys

Re: [PATCH 0/3 v3] dcache: make it more scalable on large system

2013-05-30 Thread Jörn Engel
On Thu, 30 May 2013 11:48:50 -0400, Waiman Long wrote: > On 05/29/2013 05:19 PM, Jörn Engel wrote: > >On Wed, 29 May 2013 22:37:00 +0200, Andi Kleen wrote: > >>>As Dave said before, is the last path component sufficient? Or how > >>>about an inode number? > >>Neither works, the profiler needs to f

Re: [PATCH 0/3 v3] dcache: make it more scalable on large system

2013-05-30 Thread Waiman Long
On 05/29/2013 05:19 PM, Jörn Engel wrote: On Wed, 29 May 2013 22:37:00 +0200, Andi Kleen wrote: As Dave said before, is the last path component sufficient? Or how about an inode number? Neither works, the profiler needs to find the file and read it. Ignoring all the complexity this would caus

Re: [PATCH 0/3 v3] dcache: make it more scalable on large system

2013-05-29 Thread Jörn Engel
On Wed, 29 May 2013 22:37:00 +0200, Andi Kleen wrote: > > > As Dave said before, is the last path component sufficient? Or how > > about an inode number? > > Neither works, the profiler needs to find the file and read it. Ignoring all the complexity this would cause downstream, you could do the

Re: [PATCH 0/3 v3] dcache: make it more scalable on large system

2013-05-29 Thread Andi Kleen
On Wed, May 29, 2013 at 04:43:23PM -0400, J. Bruce Fields wrote: > On Wed, May 29, 2013 at 10:37:00PM +0200, Andi Kleen wrote: > > > As Dave said before, is the last path component sufficient? Or how > > > about an inode number? > > > > Neither works, the profiler needs to find the file and read

Re: [PATCH 0/3 v3] dcache: make it more scalable on large system

2013-05-29 Thread J. Bruce Fields
On Wed, May 29, 2013 at 10:37:00PM +0200, Andi Kleen wrote: > > As Dave said before, is the last path component sufficient? Or how > > about an inode number? > > Neither works, the profiler needs to find the file and read it. > > inode searching would be incredible expensive, How fast does it n

Re: [PATCH 0/3 v3] dcache: make it more scalable on large system

2013-05-29 Thread Waiman Long
On 05/29/2013 02:46 PM, J. Bruce Fields wrote: On Wed, May 29, 2013 at 11:55:14AM -0400, Waiman Long wrote: On 05/26/2013 10:09 PM, Dave Chinner wrote: On Thu, May 23, 2013 at 05:34:23PM -0400, Waiman Long wrote: On 05/23/2013 05:42 AM, Dave Chinner wrote: What was it I said about this patchs

Re: [PATCH 0/3 v3] dcache: make it more scalable on large system

2013-05-29 Thread Waiman Long
On 05/29/2013 12:56 PM, Andi Kleen wrote: On Wed, May 29, 2013 at 12:18:09PM -0400, Simo Sorce wrote: To be honest a workload base on /etc/passwd or /etc/group is completely artificial, in actual usage, if you really have such access you use nscd or sssd with their shared memory caches to compl

Re: [PATCH 0/3 v3] dcache: make it more scalable on large system

2013-05-29 Thread Andi Kleen
> As Dave said before, is the last path component sufficient? Or how > about an inode number? Neither works, the profiler needs to find the file and read it. inode searching would be incredible expensive, unless the file system provided a "open-by-inode" primitive In normal operation you only h

Re: [PATCH 0/3 v3] dcache: make it more scalable on large system

2013-05-29 Thread Waiman Long
On 05/29/2013 12:18 PM, Simo Sorce wrote: On Wed, 2013-05-29 at 11:55 -0400, Waiman Long wrote: My patch set consists of 2 different changes. The first one is to avoid taking the d_lock lock when updating the reference count in the dentries. This particular change also benefit some other worklo

Re: [PATCH 0/3 v3] dcache: make it more scalable on large system

2013-05-29 Thread Waiman Long
On 05/29/2013 12:13 PM, Andi Kleen wrote: The d_path() is called by perf_event_mmap_event() which translates VMA to its file path for memory segments backed by files. As perf is not just for sampling data within the kernel, it can also be used for checking access pattern in the user space. As a r

Re: [PATCH 0/3 v3] dcache: make it more scalable on large system

2013-05-29 Thread J. Bruce Fields
On Wed, May 29, 2013 at 11:55:14AM -0400, Waiman Long wrote: > On 05/26/2013 10:09 PM, Dave Chinner wrote: > >On Thu, May 23, 2013 at 05:34:23PM -0400, Waiman Long wrote: > >>On 05/23/2013 05:42 AM, Dave Chinner wrote: > >>> > >>>What was it I said about this patchset when you posted it to speed >

Re: [PATCH 0/3 v3] dcache: make it more scalable on large system

2013-05-29 Thread Simo Sorce
On Wed, 2013-05-29 at 18:56 +0200, Andi Kleen wrote: > On Wed, May 29, 2013 at 12:18:09PM -0400, Simo Sorce wrote: > > On Wed, 2013-05-29 at 11:55 -0400, Waiman Long wrote: > > > > > My patch set consists of 2 different changes. The first one is to avoid > > > taking the d_lock lock when updating

Re: [PATCH 0/3 v3] dcache: make it more scalable on large system

2013-05-29 Thread Andi Kleen
On Wed, May 29, 2013 at 12:18:09PM -0400, Simo Sorce wrote: > On Wed, 2013-05-29 at 11:55 -0400, Waiman Long wrote: > > > My patch set consists of 2 different changes. The first one is to avoid > > taking the d_lock lock when updating the reference count in the > > dentries. This particular chan

Re: [PATCH 0/3 v3] dcache: make it more scalable on large system

2013-05-29 Thread Simo Sorce
On Wed, 2013-05-29 at 11:55 -0400, Waiman Long wrote: > My patch set consists of 2 different changes. The first one is to avoid > taking the d_lock lock when updating the reference count in the > dentries. This particular change also benefit some other workloads that > are filesystem intensive.

Re: [PATCH 0/3 v3] dcache: make it more scalable on large system

2013-05-29 Thread Andi Kleen
> The d_path() is called by perf_event_mmap_event() which translates > VMA to its file path for memory segments backed by files. As perf is > not just for sampling data within the kernel, it can also be used > for checking access pattern in the user space. As a result, it needs > to map VMAs back t

Re: [PATCH 0/3 v3] dcache: make it more scalable on large system

2013-05-29 Thread Waiman Long
On 05/26/2013 10:09 PM, Dave Chinner wrote: On Thu, May 23, 2013 at 05:34:23PM -0400, Waiman Long wrote: On 05/23/2013 05:42 AM, Dave Chinner wrote: What was it I said about this patchset when you posted it to speed up an Oracle benchmark back in february? I'll repeat: "Nobody should be doing

Re: [PATCH 0/3 v3] dcache: make it more scalable on large system

2013-05-26 Thread Dave Chinner
On Thu, May 23, 2013 at 05:34:23PM -0400, Waiman Long wrote: > On 05/23/2013 05:42 AM, Dave Chinner wrote: > >On Wed, May 22, 2013 at 09:37:25PM -0400, Waiman Long wrote: > >>Change log: > >> > >>v2->v3 > >> - Fix the RCU lock problem found by Al Viro. > >> - Rebase the code to the latest v3.10-r

Re: [PATCH 0/3 v3] dcache: make it more scalable on large system

2013-05-23 Thread Waiman Long
On 05/23/2013 05:42 AM, Dave Chinner wrote: On Wed, May 22, 2013 at 09:37:25PM -0400, Waiman Long wrote: Change log: v2->v3 - Fix the RCU lock problem found by Al Viro. - Rebase the code to the latest v3.10-rc1 linux mainline. - Remove patch 4 which may be problematic if the dentry is del

Re: [PATCH 0/3 v3] dcache: make it more scalable on large system

2013-05-23 Thread Dave Chinner
On Wed, May 22, 2013 at 09:37:25PM -0400, Waiman Long wrote: > Change log: > > v2->v3 > - Fix the RCU lock problem found by Al Viro. > - Rebase the code to the latest v3.10-rc1 linux mainline. > - Remove patch 4 which may be problematic if the dentry is deleted. > - Rerun performance measureme

[PATCH 0/3 v3] dcache: make it more scalable on large system

2013-05-22 Thread Waiman Long
Change log: v2->v3 - Fix the RCU lock problem found by Al Viro. - Rebase the code to the latest v3.10-rc1 linux mainline. - Remove patch 4 which may be problematic if the dentry is deleted. - Rerun performance measurement using 3.10-rc1 kernel. v1->v2 - Include performance improvement in the