[ANNOUNCE] ciopfs-0.2
Hi, Just in case someone is interested in a fuse based case insensitive overlay file system. I have just released ciopfs 0.2 which implements unicode support based on glib and has better error handling in out of memory situations. http://www.brain-dump.org/projects/ciopfs/ Have fun, Marc -- Marc Andre Tanner http://www.brain-dump.org/ GPG key: CF7D56C0
Re: [GSoC][RFC] Case Insensitive Filesystem
Hi, Just wanted to let you know that i have finally found the time to set up a little page for ciopfs: http://www.brain-dump.org/projects/ciopfs/ There are certainly a few things which could be improved but it in my tests it worked quite well in it's current form. As it seems that the idea of a fuse mounted .wine directory isn't that popular i will probably leave the code as it is and move on to other things. Anyway it was fun to do some work with fuse and in case someone is interested in it you now know where you can find it. Regards, Marc -- Marc Andre Tanner http://www.brain-dump.org/ GPG key: CF7D56C0
Re: Some considerations about the Wine FUSE project for GSoC
Kai Blin wrote: On Monday 31 March 2008 16:08:00 Kai Blin wrote: On Monday 31 March 2008 15:30:17 Marc Andre Tanner wrote: I prototyped something and thought this way I would at least get some feedback (after all Talk is cheap. Show me the code. is common practice in the open source world). But this wasn't really the case, so i am still wondering what should be done differently? Whoops, I didn't see that url. Sorry, but things were a bit busy recently. I'll give that a whirl later today. Thanks for testing btw. I'm seeing some funny behaviour in a directory containing the files foo.txt, Foo.txt and bAr.tXt After mounting, I can only see a foo.txt in the directory, That's intended, filenames which aren't all lower case are ignored. This makes sure that there won't be conflicts. Ideally you should start with an empty directory mount the fs and then untar your actual content in mountpoint. containing the Foo.txt contents. This would be a bug, are you really sure you are getting the contents of Foo.txt and not foo.txt? I ask because i can't reproduce it here. If so could you please compile in debug support make debug and then run ciopfs with option -d. This should print some debug information to stderr. Thanks, Marc -- Marc Andre Tanner http://www.brain-dump.org/ GPG key: CF7D56C0
Re: Some considerations about the Wine FUSE project for GSoC
Kai Blin wrote: Hi folks, after another chat about Wine's goals in respect to FUSE file systems on IRC, I just wanted to point the students who applied for this project already (I've Bcced them) at the goals Wine has for this GSoC project. For Wine, the goal is to create prototype FUSE file systems so we can figure out if this approach is useful to Wine at all. There is no immediate goal to include the file systems into Wine in the near future, we first want to know if they're what we need. First, I haven't yet applied for the project mainly because i will probably not be able to work full time on a GSoC project due to real life issues (part time job, family related things etc). On the other hand I doubt that it will require ~3 months to implement, I have done my prototype over more or less a weekend. I miss the technical discussion on how the problem should be solved. I think the idea of keeping the original filename in an extended attribute and storing everything in lower case is a pretty good one. Andrew Tridgell also listed it on one of his slides from WineConf05 [1]. I prototyped something and thought this way I would at least get some feedback (after all Talk is cheap. Show me the code. is common practice in the open source world). But this wasn't really the case, so i am still wondering what should be done differently? Regards, Marc [1]http://us5.samba.org/samba/ftp/slides/tridge_wineconf05.pdf (page 18) -- Marc Andre Tanner http://www.brain-dump.org/ GPG key: CF7D56C0
Re: Some considerations about the Wine FUSE project for GSoC
On Mon, Mar 31, 2008 at 04:08:00PM +0200, Kai Blin wrote: On Monday 31 March 2008 15:30:17 Marc Andre Tanner wrote: I prototyped something and thought this way I would at least get some feedback (after all Talk is cheap. Show me the code. is common practice in the open source world). But this wasn't really the case, so i am still wondering what should be done differently? Whoops, I didn't see that url. Sorry, but things were a bit busy recently. I'll give that a whirl later today. I just pushed some primarily unicode based case mapping code implemented with help of http://www.icu-project.org/ to the git repository. I am not yet sure whether the transformation works as i think it should, but at least it translates some german umlauts correctly. On Debian system the required package is called libicu-dev. Fetch the latest code from: git://repo.or.cz/ciopfs.git Regards, Marc -- Marc Andre Tanner http://www.brain-dump.org/ GPG key: CF7D56C0
Re: [GSoC][RFC] Case Insensitive Filesystem
On Wed, Mar 26, 2008 at 04:17:57PM -0700, Scott Ritchie wrote:
Alexandre Julliard wrote:
Marc Andre Tanner [EMAIL PROTECTED] writes:
Yes that's true the case information is lost but is it really needed?
If you create the files with the original mixed case you will have to
scan the whole directory in order to match it to a given filename.
And if i am not mistaken this is exactly what wine does.
Yes, and that's precisely because Wine wants to preserve case, so that
tools that look at the filesystem directly see the right thing. If you
are going to store everything lowercase in the filesystem, then we could
just as well do that inside Wine, without a FUSE module.
It seems like we have a few goals:
* Prevent having to read the whole directory when looking for a
case-insensitive file
So what are the different method's to achieve this? Apart from
translating every thing to lower case which i did.
* Allow non-Wine programs to work with the cased filenames
* Prevent the creation of conflicting files
With my proposal it basically works as long as your accessing the files
via the mount point of the fuse filesystem, however when you start messing
around with the underlying directory directly things may go wrong.
The last one is important, as we need to have a 1:1 correspondence
between cased and uncased files in the .wine folder to prevent breakages
when someone, say, installs a patch with different casing from a zip file.
I don't see any reason why it would be bad to have the FUSE module
mounted at login rather than when Wine starts. This would make your
Wine folder no different to other apps than, say, a vfat disk mounted
there. I should be able to extract a zip with file-roller containing
foo.txt and have it overwrite Foo.txt even if Wine isn't running.
Again this will work if you do it via the mountpoint:
mkdir -p ~/ciopfs/{data,mountpoint} cd ~/ciopfs
ciopfs -d data mountpoint
echo Testdata mountpoint/Foo.txt
at this point your directories should look like this:
|-- data
| `-- foo.txt
`-- mountpoint
`-- Foo.txt
and 'getfattr -dR data' should show the original file name of foo.txt:
# file: data/foo.txt
user.filename=Foo.txt
now you extract your archive in mountpoint and should get:
|-- data
| `-- foo.txt
`-- mountpoint
`-- foo.txt
What i am trying to say is that as long as you interact with your data
via mountpoint everything is fine.
When you start creating files directly in data which are not all
lowercase they will currently simply be ignored.
So the question is whether it is acceptable to say: don't touch data
directly do everything via mountpoint?
Regards,
Marc
--
Marc Andre Tanner http://www.brain-dump.org/ GPG key: CF7D56C0
Re: [GSoC][RFC] Case Insensitive Filesystem
On Wed, Mar 26, 2008 at 12:49:16PM +0100, Francois Gouget wrote: On Tue, 25 Mar 2008, Marc Andre Tanner wrote: [...] The filesystem converts every path to lower case before further operations take place. On file creation the original filename is stored in an extended attribute and later returned upon request. Shouldn't it be the other way around? I guess the way you've done it is simpler because you can simply rely on the standard kernel code to ensure filename uniqueness, but it also means that anyone accessing the underlying files directly will lose the case information. Yes that's true the case information is lost but is it really needed? If you create the files with the original mixed case you will have to scan the whole directory in order to match it to a given filename. And if i am not mistaken this is exactly what wine does. Note also that in the current form having files with upper case letters in the underlying directory will cause problems (i will probably just ignore them in readdir for now). Another quite crazy idea is to transform the whole directory to lowercase on mount and then on unmount back to the original mixed case representation. That would preserve the case information, but the mount/unmount will be quite expensive and probably cause some problems if someone is messing around with de underlying directory directly. By the way there is now a git repository available for those who want to play with it. git clone git://repo.or.cz/ciopfs.git Regards, Marc -- Marc Andre Tanner http://www.brain-dump.org/ GPG key: CF7D56C0
Re: [GSoC][RFC] Case Insensitive Filesystem
On Wed, Mar 26, 2008 at 06:10:16PM +0100, Alexandre Julliard wrote: Marc Andre Tanner [EMAIL PROTECTED] writes: Yes that's true the case information is lost but is it really needed? If you create the files with the original mixed case you will have to scan the whole directory in order to match it to a given filename. And if i am not mistaken this is exactly what wine does. Yes, and that's precisely because Wine wants to preserve case, so that tools that look at the filesystem directly see the right thing. If you are going to store everything lowercase in the filesystem, then we could just as well do that inside Wine, without a FUSE module. I am not quite sure whether i understand what you mean. Let's say we mount the case insensitive filesystem at ~/.wine/drive_c this will then preserve the case of the filenames the actual data which is all lowercase is in ~/.wine/.drive_c. So as long as the applications are looking for the files in ~/.wine/drive_c everything is fine. Now when you unmount the filesystem or simply look into ~/.wine/.drive_c the case information is 'lost' (it's still there in the extended attributes but no one cares about it). If this is all wrong, then what is the FUSE module supposed to do? Thanks, Marc -- Marc Andre Tanner http://www.brain-dump.org/ GPG key: CF7D56C0
[GSoC][RFC] Case Insensitive Filesystem
Hi,
During the last week i have been playing around with the idea
of writing a case insensitive fuse filesystem and i am know
at the point were i can demonstrate a proof of concept
implementation. And yes i have seen that there is already
someone else interested in doing this but i thought it was fun
anyways.
The filesystem converts every path to lower case before further
operations take place. On file creation the original filename is
stored in an extended attribute and later returned upon request.
This means you will have a directory on a regular POSIX filesystem
with all filenames in lower case, this directory can then be mounted
in case insensitive fashion.
In order to compile the code you will need the fuse and libattr[1]
development files. Furthermore you will have to make sure that the
underlying filesystem supports extended attributes (for example
for ext{2,3} you need a kernel with CONFIG_EXT{2,3}_FS_XATTR enabled).
You probably also want to mount the underlying filesystem with the
user_xattr option which allows non root users to create extended
attributes.
So fetch the code from the url below, extract the archive and type
make. This should generate a binary named ciopfs (which stands for
Case Insenstive On Purpose File System, alternative naming suggestions
are welcome ;).
http://www.brain-dump.org/projects/ciopfs/ciopfs-25.03.2008.tar.gz
Mounting the file system is straight forward, see `ciopfs -h` for
further options. But be carefull i haven't yet done any serious
testing, so only use it with unimportant data [insert standard
warning about possible data loss here].
ciopfs /path/to/my/directory /path/to/my/mountpoint
As i said before the current implementation is just a proof of
concept. That's why it currently only supports ascii filenames, or
whatever tolower(3) can cope with. This brings us to another
problem[2,3], in certain languages there isn't a direct mapping
between upper and lower case characters and case insensitivity in
filesystem is generally a bad thing.
Any comments are appreciated.
Regards,
Marc
[1] http://acl.bestbits.at/download.html#Attr
[2]
http://www.somethinkodd.com/oddthinking/2005/10/25/the-world-of-upper-lower-case-mappings/
[3] http://drewthaler.blogspot.com/2007/12/case-against-insensitivity.html
--
Marc Andre Tanner http://www.brain-dump.org/ GPG key: CF7D56C0
