Re: [PATCH 0/3] i_ino uniqueness: alternate approach -- hash the inodes
Eric Dumazet wrote: > The problem is you wont see the overhead of insert/delete the inode in a > global tree, since you keep hot caches. > > To have a better estimate of the overhead, I suggest you try to use more > active pipes like : > Eric, thanks for the new program. With that, the situation looks slightly worse: hashing patch (pipebench): sys 1m15.329s sys 1m16.249s sys 1m17.169s unpatched (pipebench): sys 1m9.836s sys 1m12.541s sys 1m14.153s Which works out to 1.05642174294555027017. So ~5-6%. -- Jeff - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 0/3] i_ino uniqueness: alternate approach -- hash the inodes
On Wednesday 24 January 2007 15:22, Jeff Layton wrote:
> Andrew Morton wrote:
> > What is the additional overhead, expressed in relative terms? ie: as a
> > percentage?
>
> Short answer: ~3-4% in a not very scientific test.
>
> Long answer: I timed 3 different runs of a program that created and then
> closed a pipe 10 million times on a patched and unpatched kernel. I then
> added up the "system" times for each and divided them:
Do you mean this program ?
int count, pfd[2];
for (count = 0 ; count < 1000 ; count++) {
pipe(pfd);
close(pfd[0]);
close(pfd[1]);
}
The problem is you wont see the overhead of insert/delete the inode in a
global tree, since you keep hot caches.
To have a better estimate of the overhead, I suggest you try to use more
active pipes like :
#include
#define SIZE 16384
int fds[SIZE];
int main(int argc, char *argv[])
{
unsigned int i , count ;
for (i = 0 ; i < SIZE ; i += 2)
pipe(fds + i);
i = 0;
for (count = 0 ; count < 1000 ; count++) {
close(fds[i]);
close(fds[i + 1]);
pipe(fds + i);
i = (i + 2) % SIZE;
}
return 0;
}
# ulimit -n 2
# time ./pipebench
Eric
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 0/3] i_ino uniqueness: alternate approach -- hash the inodes
Andrew Morton wrote: What is the additional overhead, expressed in relative terms? ie: as a percentage? Short answer: ~3-4% in a not very scientific test. Long answer: I timed 3 different runs of a program that created and then closed a pipe 10 million times on a patched and unpatched kernel. I then added up the "system" times for each and divided them: unpatched: sys 1m53.959s sys 1m56.083s sys 1m48.055s patched: sys 1m56.899s sys 1m57.027s sys 1m57.031s The result was 1.03803642150033866020. -- Jeff - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 0/3] i_ino uniqueness: alternate approach -- hash the inodes
Andrew Morton wrote: What is the additional overhead, expressed in relative terms? ie: as a percentage? Short answer: ~3-4% in a not very scientific test. Long answer: I timed 3 different runs of a program that created and then closed a pipe 10 million times on a patched and unpatched kernel. I then added up the system times for each and divided them: unpatched: sys 1m53.959s sys 1m56.083s sys 1m48.055s patched: sys 1m56.899s sys 1m57.027s sys 1m57.031s The result was 1.03803642150033866020. -- Jeff - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 0/3] i_ino uniqueness: alternate approach -- hash the inodes
On Wednesday 24 January 2007 15:22, Jeff Layton wrote:
Andrew Morton wrote:
What is the additional overhead, expressed in relative terms? ie: as a
percentage?
Short answer: ~3-4% in a not very scientific test.
Long answer: I timed 3 different runs of a program that created and then
closed a pipe 10 million times on a patched and unpatched kernel. I then
added up the system times for each and divided them:
Do you mean this program ?
int count, pfd[2];
for (count = 0 ; count 1000 ; count++) {
pipe(pfd);
close(pfd[0]);
close(pfd[1]);
}
The problem is you wont see the overhead of insert/delete the inode in a
global tree, since you keep hot caches.
To have a better estimate of the overhead, I suggest you try to use more
active pipes like :
#include unistd.h
#define SIZE 16384
int fds[SIZE];
int main(int argc, char *argv[])
{
unsigned int i , count ;
for (i = 0 ; i SIZE ; i += 2)
pipe(fds + i);
i = 0;
for (count = 0 ; count 1000 ; count++) {
close(fds[i]);
close(fds[i + 1]);
pipe(fds + i);
i = (i + 2) % SIZE;
}
return 0;
}
# ulimit -n 2
# time ./pipebench
Eric
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 0/3] i_ino uniqueness: alternate approach -- hash the inodes
Eric Dumazet wrote: The problem is you wont see the overhead of insert/delete the inode in a global tree, since you keep hot caches. To have a better estimate of the overhead, I suggest you try to use more active pipes like : Eric, thanks for the new program. With that, the situation looks slightly worse: hashing patch (pipebench): sys 1m15.329s sys 1m16.249s sys 1m17.169s unpatched (pipebench): sys 1m9.836s sys 1m12.541s sys 1m14.153s Which works out to 1.05642174294555027017. So ~5-6%. -- Jeff - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 0/3] i_ino uniqueness: alternate approach -- hash the inodes
On Tue, 16 Jan 2007 13:57:38 -0500 Jeff Layton <[EMAIL PROTECTED]> wrote: > The questions are: > > 1) how much would this slow down lookups for these filesystems? > 2) is it enough to justify adding more infrastructure to avoid it? > > What might be best is to start with this approach and then only move to using > IDR or some other scheme if these extra inodes in the hashtable prove to be > problematic. > > I've done some cursory testing with this patch and the overhead of hashing > and unhashing the inodes with pipefs is pretty low -- just a few seconds of > system time added on to the creation and destruction of 10 million pipes (very > similar to the overhead that the IDR approach would add). What is the additional overhead, expressed in relative terms? ie: as a percentage? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 0/3] i_ino uniqueness: alternate approach -- hash the inodes
On Tue, 16 Jan 2007 13:57:38 -0500 Jeff Layton [EMAIL PROTECTED] wrote: The questions are: 1) how much would this slow down lookups for these filesystems? 2) is it enough to justify adding more infrastructure to avoid it? What might be best is to start with this approach and then only move to using IDR or some other scheme if these extra inodes in the hashtable prove to be problematic. I've done some cursory testing with this patch and the overhead of hashing and unhashing the inodes with pipefs is pretty low -- just a few seconds of system time added on to the creation and destruction of 10 million pipes (very similar to the overhead that the IDR approach would add). What is the additional overhead, expressed in relative terms? ie: as a percentage? - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH 0/3] i_ino uniqueness: alternate approach -- hash the inodes
Since there's been no further comment on these patches, I'm going to resend them one more time and ask Andrew to commit these to -mm. Once they're in place there, I'll start working on and sending out follow-on patches to clean up the other filesystems and ensure that they properly hash their inodes. --[snip]- Since Joern mentioned that he thought that hashing the inodes might be simpler and not have a drastic performance impact, I took the liberty of whipping up some patches that use that approach. They follow in the next set of emails. To reiterate, the problems are: 1) on filesystems w/o permanent inode numbers, i_ino values can be larger than 32 bits, which can cause problems for some 32 bit userspace programs on a 64 bit kernel. We can't do anything for filesystems that have actual >32-bit inode numbers, but on filesystems that generate i_ino values on the fly, we should try to have them fit in 32 bits. We could trivially fix this by making the static counters in new_inode and iunique 32 bits, but... 2) many filesystems call new_inode and assume that the i_ino values they are given are unique. They are not guaranteed to be so, since the static counter can wrap. This problem is exacerbated by the fix for #1. 3) after allocating a new inode, some filesystems call iunique to try to get a unique i_ino value, but they don't actually add their inodes to the hashtable, and so they're still not guaranteed to be unique if that counter wraps. This patch set takes the simpler approach of simply using iunique and hashing the inodes afterward. Christoph H. previously mentioned that he thought that this approach may slow down lookups for filesystems that currently hash their inodes. The questions are: 1) how much would this slow down lookups for these filesystems? 2) is it enough to justify adding more infrastructure to avoid it? What might be best is to start with this approach and then only move to using IDR or some other scheme if these extra inodes in the hashtable prove to be problematic. I've done some cursory testing with this patch and the overhead of hashing and unhashing the inodes with pipefs is pretty low -- just a few seconds of system time added on to the creation and destruction of 10 million pipes (very similar to the overhead that the IDR approach would add). The hard thing to measure is what effect this has on other filesystems. I'm open to ways to try and gauge this. Again, I've only converted pipefs as an example. If this approach is acceptable then I'll start work on patches to convert other filesystems. Comments and suggestions welcome... -- Jeff - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH 0/3] i_ino uniqueness: alternate approach -- hash the inodes
Since there's been no further comment on these patches, I'm going to resend them one more time and ask Andrew to commit these to -mm. Once they're in place there, I'll start working on and sending out follow-on patches to clean up the other filesystems and ensure that they properly hash their inodes. --[snip]- Since Joern mentioned that he thought that hashing the inodes might be simpler and not have a drastic performance impact, I took the liberty of whipping up some patches that use that approach. They follow in the next set of emails. To reiterate, the problems are: 1) on filesystems w/o permanent inode numbers, i_ino values can be larger than 32 bits, which can cause problems for some 32 bit userspace programs on a 64 bit kernel. We can't do anything for filesystems that have actual 32-bit inode numbers, but on filesystems that generate i_ino values on the fly, we should try to have them fit in 32 bits. We could trivially fix this by making the static counters in new_inode and iunique 32 bits, but... 2) many filesystems call new_inode and assume that the i_ino values they are given are unique. They are not guaranteed to be so, since the static counter can wrap. This problem is exacerbated by the fix for #1. 3) after allocating a new inode, some filesystems call iunique to try to get a unique i_ino value, but they don't actually add their inodes to the hashtable, and so they're still not guaranteed to be unique if that counter wraps. This patch set takes the simpler approach of simply using iunique and hashing the inodes afterward. Christoph H. previously mentioned that he thought that this approach may slow down lookups for filesystems that currently hash their inodes. The questions are: 1) how much would this slow down lookups for these filesystems? 2) is it enough to justify adding more infrastructure to avoid it? What might be best is to start with this approach and then only move to using IDR or some other scheme if these extra inodes in the hashtable prove to be problematic. I've done some cursory testing with this patch and the overhead of hashing and unhashing the inodes with pipefs is pretty low -- just a few seconds of system time added on to the creation and destruction of 10 million pipes (very similar to the overhead that the IDR approach would add). The hard thing to measure is what effect this has on other filesystems. I'm open to ways to try and gauge this. Again, I've only converted pipefs as an example. If this approach is acceptable then I'll start work on patches to convert other filesystems. Comments and suggestions welcome... -- Jeff - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 0/3] i_ino uniqueness: alternate approach -- hash the inodes
Jeff Layton wrote: > Resending this set of patches to the list per Al Viro's request. I've gotten > no comments so far, so I'll presume that denotes consent and just ask Andrew > to merge them if I don't hear anything after this ;-). > > --[snip]- > > Since Joern mentioned that he thought that hashing the inodes might be simpler > and not have a drastic performance impact, I took the liberty of whipping up > some patches that use that approach. They follow in the next set of emails. > > To reiterate, the problems are: > > 1) on filesystems w/o permanent inode numbers, i_ino values can be > larger than 32 bits, which can cause problems for some 32 bit userspace > programs on a 64 bit kernel. We can't do anything for filesystems that have > actual >32-bit inode numbers, but on filesystems that generate i_ino > values on the fly, we should try to have them fit in 32 bits. We could > trivially fix this by making the static counters in new_inode and iunique > 32 bits, but... > > 2) many filesystems call new_inode and assume that the i_ino values they > are given are unique. They are not guaranteed to be so, since the static > counter can wrap. This problem is exacerbated by the fix for #1. > > 3) after allocating a new inode, some filesystems call iunique to try to > get a unique i_ino value, but they don't actually add their inodes to > the hashtable, and so they're still not guaranteed to be unique if that > counter wraps. > > This patch set takes the simpler approach of simply using iunique and > hashing the inodes afterward. Christoph H. previously mentioned that he > thought that this approach may slow down lookups for filesystems that > currently hash their inodes. > > The questions are: > > 1) how much would this slow down lookups for these filesystems? > 2) is it enough to justify adding more infrastructure to avoid it? > > What might be best is to start with this approach and then only move to using > IDR or some other scheme if these extra inodes in the hashtable prove to be > problematic. > > I've done some cursory testing with this patch and the overhead of hashing > and unhashing the inodes with pipefs is pretty low -- just a few seconds of > system time added on to the creation and destruction of 10 million pipes (very > similar to the overhead that the IDR approach would add). > > The hard thing to measure is what effect this has on other filesystems. I'm > open to ways to try and gauge this. > > Again, I've only converted pipefs as an example. If this approach is > acceptable then I'll start work on patches to convert other filesystems. > > Comments and suggestions welcome... The first two seem fine to me; I'm still thinking about how the un-hashing works in the 3rd one. Thanks, -Eric - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 0/3] i_ino uniqueness: alternate approach -- hash the inodes
Jeff Layton wrote: Resending this set of patches to the list per Al Viro's request. I've gotten no comments so far, so I'll presume that denotes consent and just ask Andrew to merge them if I don't hear anything after this ;-). --[snip]- Since Joern mentioned that he thought that hashing the inodes might be simpler and not have a drastic performance impact, I took the liberty of whipping up some patches that use that approach. They follow in the next set of emails. To reiterate, the problems are: 1) on filesystems w/o permanent inode numbers, i_ino values can be larger than 32 bits, which can cause problems for some 32 bit userspace programs on a 64 bit kernel. We can't do anything for filesystems that have actual 32-bit inode numbers, but on filesystems that generate i_ino values on the fly, we should try to have them fit in 32 bits. We could trivially fix this by making the static counters in new_inode and iunique 32 bits, but... 2) many filesystems call new_inode and assume that the i_ino values they are given are unique. They are not guaranteed to be so, since the static counter can wrap. This problem is exacerbated by the fix for #1. 3) after allocating a new inode, some filesystems call iunique to try to get a unique i_ino value, but they don't actually add their inodes to the hashtable, and so they're still not guaranteed to be unique if that counter wraps. This patch set takes the simpler approach of simply using iunique and hashing the inodes afterward. Christoph H. previously mentioned that he thought that this approach may slow down lookups for filesystems that currently hash their inodes. The questions are: 1) how much would this slow down lookups for these filesystems? 2) is it enough to justify adding more infrastructure to avoid it? What might be best is to start with this approach and then only move to using IDR or some other scheme if these extra inodes in the hashtable prove to be problematic. I've done some cursory testing with this patch and the overhead of hashing and unhashing the inodes with pipefs is pretty low -- just a few seconds of system time added on to the creation and destruction of 10 million pipes (very similar to the overhead that the IDR approach would add). The hard thing to measure is what effect this has on other filesystems. I'm open to ways to try and gauge this. Again, I've only converted pipefs as an example. If this approach is acceptable then I'll start work on patches to convert other filesystems. Comments and suggestions welcome... The first two seem fine to me; I'm still thinking about how the un-hashing works in the 3rd one. Thanks, -Eric - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH 0/3] i_ino uniqueness: alternate approach -- hash the inodes
Resending this set of patches to the list per Al Viro's request. I've gotten no comments so far, so I'll presume that denotes consent and just ask Andrew to merge them if I don't hear anything after this ;-). --[snip]- Since Joern mentioned that he thought that hashing the inodes might be simpler and not have a drastic performance impact, I took the liberty of whipping up some patches that use that approach. They follow in the next set of emails. To reiterate, the problems are: 1) on filesystems w/o permanent inode numbers, i_ino values can be larger than 32 bits, which can cause problems for some 32 bit userspace programs on a 64 bit kernel. We can't do anything for filesystems that have actual >32-bit inode numbers, but on filesystems that generate i_ino values on the fly, we should try to have them fit in 32 bits. We could trivially fix this by making the static counters in new_inode and iunique 32 bits, but... 2) many filesystems call new_inode and assume that the i_ino values they are given are unique. They are not guaranteed to be so, since the static counter can wrap. This problem is exacerbated by the fix for #1. 3) after allocating a new inode, some filesystems call iunique to try to get a unique i_ino value, but they don't actually add their inodes to the hashtable, and so they're still not guaranteed to be unique if that counter wraps. This patch set takes the simpler approach of simply using iunique and hashing the inodes afterward. Christoph H. previously mentioned that he thought that this approach may slow down lookups for filesystems that currently hash their inodes. The questions are: 1) how much would this slow down lookups for these filesystems? 2) is it enough to justify adding more infrastructure to avoid it? What might be best is to start with this approach and then only move to using IDR or some other scheme if these extra inodes in the hashtable prove to be problematic. I've done some cursory testing with this patch and the overhead of hashing and unhashing the inodes with pipefs is pretty low -- just a few seconds of system time added on to the creation and destruction of 10 million pipes (very similar to the overhead that the IDR approach would add). The hard thing to measure is what effect this has on other filesystems. I'm open to ways to try and gauge this. Again, I've only converted pipefs as an example. If this approach is acceptable then I'll start work on patches to convert other filesystems. Comments and suggestions welcome... -- Jeff - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH 0/3] i_ino uniqueness: alternate approach -- hash the inodes
Resending this set of patches to the list per Al Viro's request. I've gotten no comments so far, so I'll presume that denotes consent and just ask Andrew to merge them if I don't hear anything after this ;-). --[snip]- Since Joern mentioned that he thought that hashing the inodes might be simpler and not have a drastic performance impact, I took the liberty of whipping up some patches that use that approach. They follow in the next set of emails. To reiterate, the problems are: 1) on filesystems w/o permanent inode numbers, i_ino values can be larger than 32 bits, which can cause problems for some 32 bit userspace programs on a 64 bit kernel. We can't do anything for filesystems that have actual 32-bit inode numbers, but on filesystems that generate i_ino values on the fly, we should try to have them fit in 32 bits. We could trivially fix this by making the static counters in new_inode and iunique 32 bits, but... 2) many filesystems call new_inode and assume that the i_ino values they are given are unique. They are not guaranteed to be so, since the static counter can wrap. This problem is exacerbated by the fix for #1. 3) after allocating a new inode, some filesystems call iunique to try to get a unique i_ino value, but they don't actually add their inodes to the hashtable, and so they're still not guaranteed to be unique if that counter wraps. This patch set takes the simpler approach of simply using iunique and hashing the inodes afterward. Christoph H. previously mentioned that he thought that this approach may slow down lookups for filesystems that currently hash their inodes. The questions are: 1) how much would this slow down lookups for these filesystems? 2) is it enough to justify adding more infrastructure to avoid it? What might be best is to start with this approach and then only move to using IDR or some other scheme if these extra inodes in the hashtable prove to be problematic. I've done some cursory testing with this patch and the overhead of hashing and unhashing the inodes with pipefs is pretty low -- just a few seconds of system time added on to the creation and destruction of 10 million pipes (very similar to the overhead that the IDR approach would add). The hard thing to measure is what effect this has on other filesystems. I'm open to ways to try and gauge this. Again, I've only converted pipefs as an example. If this approach is acceptable then I'll start work on patches to convert other filesystems. Comments and suggestions welcome... -- Jeff - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH 0/3] i_ino uniqueness: alternate approach -- hash the inodes
Since Joern mentioned that he thought that hashing the inodes might be simpler and not have a drastic performance impact, I took the liberty of whipping up some patches that use that approach. They follow in the next set of emails. To reiterate, the problems are: 1) on filesystems w/o permanent inode numbers, i_ino values can be larger than 32 bits, which can cause problems for some 32 bit userspace programs on a 64 bit kernel. We can't do anything for filesystems that have actual >32-bit inode numbers, but on filesystems that generate i_ino values on the fly, we should try to have them fit in 32 bits. We could trivially fix this by making the static counters in new_inode and iunique 32 bits, but... 2) many filesystems call new_inode and assume that the i_ino values they are given are unique. They are not guaranteed to be so, since the static counter can wrap. This problem is exacerbated by the fix for #1. 3) after allocating a new inode, some filesystems call iunique to try to get a unique i_ino value, but they don't actually add their inodes to the hashtable, and so they're still not guaranteed to be unique if that counter wraps. This patch set takes the simpler approach of simply using iunique and hashing the inodes afterward. Christoph H. previously mentioned that he thought that this approach may slow down lookups for filesystems that currently hash their inodes. The questions are: 1) how much would this slow down lookups for these filesystems? 2) is it enough to justify adding more infrastructure to avoid it? What might be best is to start with this approach and then only move to using IDR or some other scheme if these extra inodes in the hashtable prove to be problematic. I've done some cursory testing with this patch and the overhead of hashing and unhashing the inodes with pipefs is pretty low -- just a few seconds of system time added on to the creation and destruction of 10 million pipes (very similar to the overhead that the IDR approach would add). The hard thing to measure is what effect this has on other filesystems. I'm open to ways to try and gauge this. Again, I've only converted pipefs as an example. If this approach is acceptable then I'll start work on patches to convert other filesystems. Comments and suggestions welcome... -- Jeff - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH 0/3] i_ino uniqueness: alternate approach -- hash the inodes
Since Joern mentioned that he thought that hashing the inodes might be simpler and not have a drastic performance impact, I took the liberty of whipping up some patches that use that approach. They follow in the next set of emails. To reiterate, the problems are: 1) on filesystems w/o permanent inode numbers, i_ino values can be larger than 32 bits, which can cause problems for some 32 bit userspace programs on a 64 bit kernel. We can't do anything for filesystems that have actual 32-bit inode numbers, but on filesystems that generate i_ino values on the fly, we should try to have them fit in 32 bits. We could trivially fix this by making the static counters in new_inode and iunique 32 bits, but... 2) many filesystems call new_inode and assume that the i_ino values they are given are unique. They are not guaranteed to be so, since the static counter can wrap. This problem is exacerbated by the fix for #1. 3) after allocating a new inode, some filesystems call iunique to try to get a unique i_ino value, but they don't actually add their inodes to the hashtable, and so they're still not guaranteed to be unique if that counter wraps. This patch set takes the simpler approach of simply using iunique and hashing the inodes afterward. Christoph H. previously mentioned that he thought that this approach may slow down lookups for filesystems that currently hash their inodes. The questions are: 1) how much would this slow down lookups for these filesystems? 2) is it enough to justify adding more infrastructure to avoid it? What might be best is to start with this approach and then only move to using IDR or some other scheme if these extra inodes in the hashtable prove to be problematic. I've done some cursory testing with this patch and the overhead of hashing and unhashing the inodes with pipefs is pretty low -- just a few seconds of system time added on to the creation and destruction of 10 million pipes (very similar to the overhead that the IDR approach would add). The hard thing to measure is what effect this has on other filesystems. I'm open to ways to try and gauge this. Again, I've only converted pipefs as an example. If this approach is acceptable then I'll start work on patches to convert other filesystems. Comments and suggestions welcome... -- Jeff - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
