Re: [GSoC][RFC] Case Insensitive Filesystem
Hi, Just wanted to let you know that i have finally found the time to set up a little page for ciopfs: http://www.brain-dump.org/projects/ciopfs/ There are certainly a few things which could be improved but it in my tests it worked quite well in it's current form. As it seems that the idea of a fuse mounted .wine directory isn't that popular i will probably leave the code as it is and move on to other things. Anyway it was fun to do some work with fuse and in case someone is interested in it you now know where you can find it. Regards, Marc -- Marc Andre Tanner http://www.brain-dump.org/ GPG key: CF7D56C0
Re: [GSoC][RFC] Case Insensitive Filesystem
On Wed, Mar 26, 2008 at 04:17:57PM -0700, Scott Ritchie wrote:
Alexandre Julliard wrote:
Marc Andre Tanner [EMAIL PROTECTED] writes:
Yes that's true the case information is lost but is it really needed?
If you create the files with the original mixed case you will have to
scan the whole directory in order to match it to a given filename.
And if i am not mistaken this is exactly what wine does.
Yes, and that's precisely because Wine wants to preserve case, so that
tools that look at the filesystem directly see the right thing. If you
are going to store everything lowercase in the filesystem, then we could
just as well do that inside Wine, without a FUSE module.
It seems like we have a few goals:
* Prevent having to read the whole directory when looking for a
case-insensitive file
So what are the different method's to achieve this? Apart from
translating every thing to lower case which i did.
* Allow non-Wine programs to work with the cased filenames
* Prevent the creation of conflicting files
With my proposal it basically works as long as your accessing the files
via the mount point of the fuse filesystem, however when you start messing
around with the underlying directory directly things may go wrong.
The last one is important, as we need to have a 1:1 correspondence
between cased and uncased files in the .wine folder to prevent breakages
when someone, say, installs a patch with different casing from a zip file.
I don't see any reason why it would be bad to have the FUSE module
mounted at login rather than when Wine starts. This would make your
Wine folder no different to other apps than, say, a vfat disk mounted
there. I should be able to extract a zip with file-roller containing
foo.txt and have it overwrite Foo.txt even if Wine isn't running.
Again this will work if you do it via the mountpoint:
mkdir -p ~/ciopfs/{data,mountpoint} cd ~/ciopfs
ciopfs -d data mountpoint
echo Testdata mountpoint/Foo.txt
at this point your directories should look like this:
|-- data
| `-- foo.txt
`-- mountpoint
`-- Foo.txt
and 'getfattr -dR data' should show the original file name of foo.txt:
# file: data/foo.txt
user.filename=Foo.txt
now you extract your archive in mountpoint and should get:
|-- data
| `-- foo.txt
`-- mountpoint
`-- foo.txt
What i am trying to say is that as long as you interact with your data
via mountpoint everything is fine.
When you start creating files directly in data which are not all
lowercase they will currently simply be ignored.
So the question is whether it is acceptable to say: don't touch data
directly do everything via mountpoint?
Regards,
Marc
--
Marc Andre Tanner http://www.brain-dump.org/ GPG key: CF7D56C0
Re: [GSoC][RFC] Case Insensitive Filesystem
On Tue, 25 Mar 2008, Marc Andre Tanner wrote: [...] The filesystem converts every path to lower case before further operations take place. On file creation the original filename is stored in an extended attribute and later returned upon request. Shouldn't it be the other way around? I guess the way you've done it is simpler because you can simply rely on the standard kernel code to ensure filename uniqueness, but it also means that anyone accessing the underlying files directly will lose the case information. -- Francois Gouget [EMAIL PROTECTED] http://fgouget.free.fr/ Any sufficiently advanced Operating System is indistinguishable from Linux
Re: [GSoC][RFC] Case Insensitive Filesystem
On Wed, Mar 26, 2008 at 12:49:16PM +0100, Francois Gouget wrote: On Tue, 25 Mar 2008, Marc Andre Tanner wrote: [...] The filesystem converts every path to lower case before further operations take place. On file creation the original filename is stored in an extended attribute and later returned upon request. Shouldn't it be the other way around? I guess the way you've done it is simpler because you can simply rely on the standard kernel code to ensure filename uniqueness, but it also means that anyone accessing the underlying files directly will lose the case information. Yes that's true the case information is lost but is it really needed? If you create the files with the original mixed case you will have to scan the whole directory in order to match it to a given filename. And if i am not mistaken this is exactly what wine does. Note also that in the current form having files with upper case letters in the underlying directory will cause problems (i will probably just ignore them in readdir for now). Another quite crazy idea is to transform the whole directory to lowercase on mount and then on unmount back to the original mixed case representation. That would preserve the case information, but the mount/unmount will be quite expensive and probably cause some problems if someone is messing around with de underlying directory directly. By the way there is now a git repository available for those who want to play with it. git clone git://repo.or.cz/ciopfs.git Regards, Marc -- Marc Andre Tanner http://www.brain-dump.org/ GPG key: CF7D56C0
Re: [GSoC][RFC] Case Insensitive Filesystem
Marc Andre Tanner [EMAIL PROTECTED] writes: Yes that's true the case information is lost but is it really needed? If you create the files with the original mixed case you will have to scan the whole directory in order to match it to a given filename. And if i am not mistaken this is exactly what wine does. Yes, and that's precisely because Wine wants to preserve case, so that tools that look at the filesystem directly see the right thing. If you are going to store everything lowercase in the filesystem, then we could just as well do that inside Wine, without a FUSE module. -- Alexandre Julliard [EMAIL PROTECTED]
Re: [GSoC][RFC] Case Insensitive Filesystem
On Wed, Mar 26, 2008 at 06:10:16PM +0100, Alexandre Julliard wrote: Marc Andre Tanner [EMAIL PROTECTED] writes: Yes that's true the case information is lost but is it really needed? If you create the files with the original mixed case you will have to scan the whole directory in order to match it to a given filename. And if i am not mistaken this is exactly what wine does. Yes, and that's precisely because Wine wants to preserve case, so that tools that look at the filesystem directly see the right thing. If you are going to store everything lowercase in the filesystem, then we could just as well do that inside Wine, without a FUSE module. I am not quite sure whether i understand what you mean. Let's say we mount the case insensitive filesystem at ~/.wine/drive_c this will then preserve the case of the filenames the actual data which is all lowercase is in ~/.wine/.drive_c. So as long as the applications are looking for the files in ~/.wine/drive_c everything is fine. Now when you unmount the filesystem or simply look into ~/.wine/.drive_c the case information is 'lost' (it's still there in the extended attributes but no one cares about it). If this is all wrong, then what is the FUSE module supposed to do? Thanks, Marc -- Marc Andre Tanner http://www.brain-dump.org/ GPG key: CF7D56C0
Re: [GSoC][RFC] Case Insensitive Filesystem
Alexandre Julliard wrote: Marc Andre Tanner [EMAIL PROTECTED] writes: Yes that's true the case information is lost but is it really needed? If you create the files with the original mixed case you will have to scan the whole directory in order to match it to a given filename. And if i am not mistaken this is exactly what wine does. Yes, and that's precisely because Wine wants to preserve case, so that tools that look at the filesystem directly see the right thing. If you are going to store everything lowercase in the filesystem, then we could just as well do that inside Wine, without a FUSE module. It seems like we have a few goals: * Prevent having to read the whole directory when looking for a case-insensitive file * Allow non-Wine programs to work with the cased filenames * Prevent the creation of conflicting files The last one is important, as we need to have a 1:1 correspondence between cased and uncased files in the .wine folder to prevent breakages when someone, say, installs a patch with different casing from a zip file. I don't see any reason why it would be bad to have the FUSE module mounted at login rather than when Wine starts. This would make your Wine folder no different to other apps than, say, a vfat disk mounted there. I should be able to extract a zip with file-roller containing foo.txt and have it overwrite Foo.txt even if Wine isn't running. Thanks, Scott Ritchie
Re: [GSoC][RFC] Case Insensitive Filesystem
On Wed, Mar 26, 2008 at 6:17 PM, Scott Ritchie [EMAIL PROTECTED] wrote: Alexandre Julliard wrote: Marc Andre Tanner [EMAIL PROTECTED] writes: Yes that's true the case information is lost but is it really needed? If you create the files with the original mixed case you will have to scan the whole directory in order to match it to a given filename. And if i am not mistaken this is exactly what wine does. Yes, and that's precisely because Wine wants to preserve case, so that tools that look at the filesystem directly see the right thing. If you are going to store everything lowercase in the filesystem, then we could just as well do that inside Wine, without a FUSE module. It seems like we have a few goals: * Prevent having to read the whole directory when looking for a case-insensitive file * Allow non-Wine programs to work with the cased filenames * Prevent the creation of conflicting files We could do this for the drive_c directory but what about /? you cannot enforce that there. Maybe this part project should be divided into two different problems. 1. How to achieve that for drive_c, and 2. how to mount a proxy filesystem for / and apply some rules when conflicting files exist, as keep the newest conflicting file with the original name and rename other files like it was proposed on another thread on this list. The last one is important, as we need to have a 1:1 correspondence between cased and uncased files in the .wine folder to prevent breakages when someone, say, installs a patch with different casing from a zip file. I don't see any reason why it would be bad to have the FUSE module mounted at login rather than when Wine starts. This would make your Wine folder no different to other apps than, say, a vfat disk mounted there. I should be able to extract a zip with file-roller containing foo.txt and have it overwrite Foo.txt even if Wine isn't running. Thanks, Scott Ritchie
Re: [GSoC][RFC] Case Insensitive Filesystem
On Wed, Mar 26, 2008 at 6:39 PM, Cesar Izurieta [EMAIL PROTECTED] wrote: Alexandre Julliard wrote: Yes, and that's precisely because Wine wants to preserve case, so that tools that look at the filesystem directly see the right thing. If you are going to store everything lowercase in the filesystem, then we could just as well do that inside Wine, without a FUSE module. We could do this for the drive_c directory but what about /? you cannot enforce that there. Maybe this part project should be divided into two different problems. 1. How to achieve that for drive_c, and 2. how to mount a proxy filesystem for / and apply some rules when conflicting files exist, as keep the newest conflicting file with the original name and rename other files like it was proposed on another thread on this list. We could do this in wine itself for drive_c, which is where most of the problem seems to lie (assuming no one links their program files to a different mount point). Most performance issues/problem are with patches or with installers searching for lots of files (within drive_c). While it would be ideal to do this for the entire FS, doing it for drive_c should resolve most issues.
Re: [GSoC][RFC] Case Insensitive Filesystem
Austin English wrote: On Wed, Mar 26, 2008 at 6:39 PM, Cesar Izurieta [EMAIL PROTECTED] wrote: Alexandre Julliard wrote: Yes, and that's precisely because Wine wants to preserve case, so that tools that look at the filesystem directly see the right thing. If you are going to store everything lowercase in the filesystem, then we could just as well do that inside Wine, without a FUSE module. We could do this for the drive_c directory but what about /? you cannot enforce that there. Maybe this part project should be divided into two different problems. 1. How to achieve that for drive_c, and 2. how to mount a proxy filesystem for / and apply some rules when conflicting files exist, as keep the newest conflicting file with the original name and rename other files like it was proposed on another thread on this list. We could do this in wine itself for drive_c, which is where most of the problem seems to lie (assuming no one links their program files to a different mount point). Most performance issues/problem are with patches or with installers searching for lots of files (within drive_c). While it would be ideal to do this for the entire FS, doing it for drive_c should resolve most issues. You're very right that doing this for just drive_c solves most issues (as we can always fall back on Wine's existing behavior if we're outside of C), however unless we have some Wine process always running there's no way we can guarantee preserving the case integrity of drive_c just inside Wine. Thanks, Scott Ritchie
[GSoC][RFC] Case Insensitive Filesystem
Hi,
During the last week i have been playing around with the idea
of writing a case insensitive fuse filesystem and i am know
at the point were i can demonstrate a proof of concept
implementation. And yes i have seen that there is already
someone else interested in doing this but i thought it was fun
anyways.
The filesystem converts every path to lower case before further
operations take place. On file creation the original filename is
stored in an extended attribute and later returned upon request.
This means you will have a directory on a regular POSIX filesystem
with all filenames in lower case, this directory can then be mounted
in case insensitive fashion.
In order to compile the code you will need the fuse and libattr[1]
development files. Furthermore you will have to make sure that the
underlying filesystem supports extended attributes (for example
for ext{2,3} you need a kernel with CONFIG_EXT{2,3}_FS_XATTR enabled).
You probably also want to mount the underlying filesystem with the
user_xattr option which allows non root users to create extended
attributes.
So fetch the code from the url below, extract the archive and type
make. This should generate a binary named ciopfs (which stands for
Case Insenstive On Purpose File System, alternative naming suggestions
are welcome ;).
http://www.brain-dump.org/projects/ciopfs/ciopfs-25.03.2008.tar.gz
Mounting the file system is straight forward, see `ciopfs -h` for
further options. But be carefull i haven't yet done any serious
testing, so only use it with unimportant data [insert standard
warning about possible data loss here].
ciopfs /path/to/my/directory /path/to/my/mountpoint
As i said before the current implementation is just a proof of
concept. That's why it currently only supports ascii filenames, or
whatever tolower(3) can cope with. This brings us to another
problem[2,3], in certain languages there isn't a direct mapping
between upper and lower case characters and case insensitivity in
filesystem is generally a bad thing.
Any comments are appreciated.
Regards,
Marc
[1] http://acl.bestbits.at/download.html#Attr
[2]
http://www.somethinkodd.com/oddthinking/2005/10/25/the-world-of-upper-lower-case-mappings/
[3] http://drewthaler.blogspot.com/2007/12/case-against-insensitivity.html
--
Marc Andre Tanner http://www.brain-dump.org/ GPG key: CF7D56C0
