Bug#500540: Considerations.
Hi Raúl, I'm sorry I didn't respond to your comment on this bug as I am not subscribed to it. On Saturday 08 November 2008 19:01:23 Raúl Sánchez Siles wrote: Some time ago, no sooner than Etch was released, CONFIG_FAT_DEFAULT_CODEPAGE was 437(cp437) and CONFIG_FAT_DEFAULT_IOCHARSET was iso8859-1 (latin1). But after http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=417324 it was changed to current 437 and utf8, respectively. Things being like this, using utf8 as long file name encoding, results in case sensitive file names. I find this indeed a drawback, but using uft8 has the advantage of support for non-ascii chars. [Or at least supporting it correctly] As far as I can tell your analysis is correct. But (and this is a big but) you seem to have ignored the utf8 flag. The effect of using a non-utf8 iocharset and the utf8 flag is that long filenames will be encoded in utf8 and the iocharset will only be used for case comparison. IMHO this is the real solution that you are looking for, but there is (or was?) a problem with making it the default. (See Message#48 for details). Cheers, -henrik -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#505682: apache2.2-common: Please add Provides: apache2-common
Package: apache2.2-common Version: 2.2.3-4+etch6 Severity: normal The package has a Replaces: apache2-common line but no Provides: apache2-common line. Thus it makes it impossible to install packages that depend on apache2-common. Try to installing phpix to see the result. Eventhough apache2.2-common is installed, apt-get and aptitude both want to install apache because for phpix it is an alternative to apache2-common. I tried to build a fake package with equivs that would provide apache2-common but the resulting package conflicts with apache2.2-common. So I was forced to install a fake package that provides apache instead. This is a very crude hack and will certainly break my neck as soon as I install packages that realy depend on apache :-( -- System Information: Debian Release: 4.0 APT prefers stable APT policy: (500, 'stable') Architecture: i386 (i686) Shell: /bin/sh linked to /bin/bash Kernel: Linux 2.6.24 Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8) Versions of packages apache2.2-common depends on: ii apache2-utils 2.2.3-4+etch6 utility programs for webservers ii libmagic1 4.17-5etch3 File type determination library us ii lsb-base 3.1-23.2etch1 Linux Standard Base 3.1 init scrip ii mime-support 3.39-1MIME files 'mime.types' 'mailcap ii net-tools 1.60-17 The NET-3 networking toolkit ii procps 1:3.2.7-3 /proc file system utilities apache2.2-common recommends no packages. -- no debconf information -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#500540: kdebase: automounting vfat (partialy) case sensitive due to utf8 is plain wrong and dangerous
On Fri, Oct 10, 2008 at 11:31:27PM +0300, Teemu Likonen wrote: Heinrich Langos [EMAIL PROTECTED] writes: Now lets try again with more sane vfat options: # mount | grep vfat /dev/sda1 on /mnt type vfat (rw,nosuid,nodev,noatime,uhelper=hal,flush,uid=1000,shortname=lower,check=relaxed,codepage=850,iocharset=iso8859-1) shortname=lower should be used but don't take my word for it. shortname=mixed works nicely with utf8 flag, and command touch test Test teSt touches the same file three times. The main question was if it preserves the case if you do touch TeSt test TEST In my setting (shortname=lower) the resulting file was test. In your case it should be TeSt but opening test should also work. The problem with case sensitive mounting and short names is that the application doesn't know which shortname option was used. So if my program writes /media/ipod/foo.mp3 and you used shortname=win95 than the name on the filesystem will be FOO.MP3 When you mount that filesystem with iocharset=utf8 then my program WILL break as it is not be able to open the file it wrote. gnupod for example uses short filenames explicitly to avoid other problems and DOES definetly break in such a situation. I've read of other music management software that does the same thing. Long filenames are a different beast altogether... And who came up with the idea to mount vfat with utf8 anyway? It was never designed to take short utf8 names. Those are strictly 8.3 and if you try to stick utf8 characters in there, you get all kinds of length checking problems. Long names on vfat are stored in unicode anyway. So whats the big gain here? For the sake of squeezing utf8 into places where it never was ment to be, we get messed up filesystems? I admit that some of the ideas may have come from me. I have described one aspect of this issue in the kernel bug #417324: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=417324 I'm pretty confused with all these iocharset, codepage and utf8 flags but I'm certain that in Debian Etch (and its default locale settings [UTF-8] and kernel settings) filenames get converted totally wrong. Long filenames in FAT filesystem are in the form we call UTF-16 today. In default Etch system FAT's UTF-16 filenames get converted to ISO-8859-1 if the filesystem is not mounted with utf8 flag. The other direction is so that Etch's UTF-8 filenames are assumed to be in ISO-8859-1 and, since it's a single-byte encoding, every byte (even in a UTF-8 multibyte character) gets converted separately to UTF-16. This produces complete garbage of course. KDE is nice enough to use utf8 flag but someone reported that Gnome does not (or at least did not) mount with this flag. Thus it produces filenames which are unreadable in other systems (including MS Windows). I guess the change in kernel settings made you see this issue after upgrading from Etch to Lenny. The option CONFIG_FAT_DEFAULT_IOCHARSET was changed from iso-8859-1 to utf8. Ok, so in order to fix a gnome bug you broke everything else? :-) As far as I have seen in archives and related bug reports the blame for this problem gets shifted around from KDE to pmount to the kernel itself and all the way back. Everybody happily points fingers at the others. This seems to be pretty complicated. We have to make VFAT/UTF-16 -- Debian/UTF-8 conversion work somehow, and in Etch it does not work (except when KDE does the mounting). In Lenny the conversion currently works by default with just mount without any options; this is because the change in the kernel settings. But then there's the issue you reported... :-( In my experience shortname=mixed works nicely without character case problems. Please read this: http://www.nslu2-linux.org/wiki/HowTo/MountFATFileSystems If you need a short explaination of ther vfat mount options codepage, iocharset and utf8. Let me Quote: When the utf8 flag is specified along with iocharset the iocharset value only controls the character case handling - it has no effect on the encoding of the UNICODE characters as this will always use UTF-8. So, according to it the right thing would be to select the iocharset according to your language AND specifying the utf8 flag. So in my case mount ... iocharset=iso8859-1,utf8 If gnome doesn't do it right, please go ahead and fix gnome but leave the kernel default iocharset as it was. Unfortunately the right thing can't be done by default: Let me quote once more: The utf8 flag cannot be specified by default - it must be given as an explicit argument to mount I am not sure if that is still the case with the current kernel. Maybe somebody grew tired of this limitation and fixed it. Could you check that? I'm drowning in work... could you also add a bts reference to 417324 to show the dependency and reopen that bug? -henrik (Using Debian since buzz.) Wow, I'm from the Woody/Sarge generation
Bug#500540: kdebase: automounting vfat (partialy) case sensitive due to utf8 is plain wrong and dangerous
Package: kdebase Version: 4:3.5.9.dfsg.1-5 Severity: grave Justification: causes non-serious data loss I just switched from etch to lenny and now gnupod warns me about the case sensitive filesystem on my iPod: $ gnupod_check.pl gnupod_check.pl Version 0.99.8 (C) Adrian Ulrich /usr/local/bin/gnupod_check.pl: Warning: /media/IPOD is mounted case sensitive, that's bad: FAT32-iPods should be mounted case in-sensitive! (try 'mount ... -o check=relaxed') Pass 1: Checking Files in the GNUtunesDB.xml... Pass 2: Checking Files on the iPod... ..finished ... The filesystem gets automounted by KDE's mediamanager and never caused a problem when I used Etch. The kernel also complains: [490945.600269] usb 4-3: new high speed USB device using ehci_hcd and address 23 [490945.733907] usb 4-3: configuration #1 chosen from 2 choices [490945.761374] scsi18 : SCSI emulation for USB Mass Storage devices [490945.762123] usb 4-3: New USB device found, idVendor=05ac, idProduct=1262 [490945.762138] usb 4-3: New USB device strings: Mfr=1, Product=2, SerialNumber=3 [490945.762147] usb 4-3: Product: iPod [490945.762153] usb 4-3: Manufacturer: Apple Inc. [490945.762159] usb 4-3: SerialNumber: 000A27001B256A33 [490945.762731] usb-storage: device found at 23 [490945.762742] usb-storage: waiting for device to settle before scanning [490950.760450] usb-storage: device scan complete [490950.776262] scsi 18:0:0:0: Direct-Access AppleiPod 1.62 PQ: 0 ANSI: 0 [490950.824634] sd 18:0:0:0: [sda] 1941441 4096-byte hardware sectors (7952 MB) [490950.825383] sd 18:0:0:0: [sda] Write Protect is off [490950.825397] sd 18:0:0:0: [sda] Mode Sense: 68 00 00 08 [490950.825404] sd 18:0:0:0: [sda] Assuming drive cache: write through [490950.827876] sd 18:0:0:0: [sda] 1941441 4096-byte hardware sectors (7952 MB) [490950.828416] sd 18:0:0:0: [sda] Write Protect is off [490950.828429] sd 18:0:0:0: [sda] Mode Sense: 68 00 00 08 [490950.828436] sd 18:0:0:0: [sda] Assuming drive cache: write through [490950.828443] sda: sda1 [490950.830215] sd 18:0:0:0: [sda] Attached SCSI removable disk [490951.932053] FAT: utf8 is not a recommended IO charset for FAT filesystems, filesystem will be case sensitive! And it is damn right to complain! So don't give me any of the should be ignored comments that you might have googled here: http://www.linuxfromscratch.org/lfs/view/development/chapter08/fstab.html Now lets see the effects of case sensitive vfat, shall we?: # mount | grep vfat /dev/sda1 on /media/IPOD type vfat (rw,nosuid,nodev,noatime,uhelper=hal,flush,uid=1000,utf8,shortname=lower) # cd /media/IPOD /media/IPOD# touch TEST TESt TEsT touch: cannot touch `TESt': File exists touch: cannot touch `TEsT': File exists /media/IPOD# touch TESTDIRECTORY TEStDIRECTORY TEsTDIRECTORY /media/IPOD# ls -lsa TE* 0 -rwxr-xr-x 1 hlangos root 0 2008-09-28 15:33 TEsTDIRECTORY 0 -rwxr-xr-x 1 hlangos root 0 2008-09-28 15:33 TEStDIRECTORY 0 -rwxr-xr-x 1 hlangos root 0 2008-09-28 15:33 TESTDIRECTORY /media/IPOD# ls -lsa te* ls: cannot access test: No such file or directory /media/IPOD# ls -lsa ls: cannot access test: No such file or directory total 32 4 drwxr-xr-x 8 hlangos root 4096 2008-09-28 15:33 . 4 drwxr-xr-x 4 rootroot 4096 2008-09-28 15:31 .. 4 drwxr-xr-x 2 hlangos root 4096 2000-02-09 07:06 Calendars 4 drwxr-xr-x 2 hlangos root 4096 2008-04-09 15:23 Contacts 4 drwxr-xr-x 11 hlangos root 4096 2008-02-13 00:01 iPod_Control 0 -rwxr-xr-x 1 hlangos root0 2000-02-09 07:06 .metadata_never_index 4 drwxr-xr-x 2 hlangos root 4096 2008-04-09 15:25 Notes 4 drwxr-xr-x 3 hlangos root 4096 2008-01-15 23:08 Photos 4 drwxr-xr-x 2 hlangos root 4096 2000-02-09 07:06 Recordings ? -? ? ? ? ?? test 0 -rwxr-xr-x 1 hlangos root0 2008-09-28 15:33 TEsTDIRECTORY 0 -rwxr-xr-x 1 hlangos root0 2008-09-28 15:33 TEStDIRECTORY 0 -rwxr-xr-x 1 hlangos root0 2008-09-28 15:33 TESTDIRECTORY /media/IPOD# That is FUBAR ! 1. touch TEST TESt TEsT should not complain about existing files. 2. touch TESTDIRECTORY TEStDIRECTORY TEsTDIRECTORY should not create more than one file. 3. And WTF happend to test anaway? BTW: If you try to delete some of those testdirectory files you may run into more of those nasty questionmarks. I didn't dare to continue working on a filesystem in such a state but to me this looks like a corrupted file system. Hence severity grave. Now lets try again with more sane vfat options: # mount | grep vfat /dev/sda1 on /mnt type vfat (rw,nosuid,nodev,noatime,uhelper=hal,flush,uid=1000,shortname=lower,check=relaxed,codepage=850,iocharset=iso8859-1) # cd /mnt /mnt# touch TEST TESt TEsT /mnt# touch TESTDIRECTORY TEStDIRECTORY TEsTDIRECTORY /mnt# ls -lsa TE* 0 -rwxr-xr-x 1 hlangos root 0 2008-09-28 15:41 TESTDIRECTORY /mnt# ls -lsa total 32 4 drwxr-xr-x 8 hlangos root 4096 2008-09-28 15:41 . 4
