[Bug 255255] Re: blkid library detects JFS partition as FAT32
Fix released in 1.41.2-1ubuntu1. ** Changed in: e2fsprogs (Ubuntu) Status: New => Fix Released -- blkid library detects JFS partition as FAT32 https://bugs.launchpad.net/bugs/255255 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 255255] Re: blkid library detects JFS partition as FAT32
Note: a fix for this bug was released in e2fsprogs 1.41.1. -- blkid library detects JFS partition as FAT32 https://bugs.launchpad.net/bugs/255255 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 255255] Re: blkid library detects JFS partition as FAT32
On Mon, Aug 25, 2008 at 11:22:43PM -, boga wrote: > So I guess your latest idea is correct: if a boot sector looks like > FAT12/16 and contains "JFS " or "HPFS " at 0x36 then it is not > FAT. However are you sure about memcmp(ms->ms_magic, "JFS ", 8) and > memcmp(ms->ms_magic, "HPFS ", 8) ? They seem not to have enough > spaces. What I checked in has enoguh spaces. I think that got zapped by the my e-mail MUA auto-fill function, and I didn't catch it. - Ted -- blkid library detects JFS partition as FAT32 https://bugs.launchpad.net/bugs/255255 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 255255] Re: blkid library detects JFS partition as FAT32
Theodore, DFSee seems to have originated as an OS/2 partition management utility so there's no wonder it simulates OS/2 UHPFS.DLL/UJFS.DLL behaviour. I've made some tests and the problem seems to involve only those file systems (e.g. when creating an ext2/ext3/reiser partition DFSee simply fills the boot sector with F6). The reason for using FAT12/16 style pseudo-superblock seems to be quite simple: there was no FAT32 at the time HPFS was developed. So I guess your latest idea is correct: if a boot sector looks like FAT12/16 and contains "JFS " or "HPFS" at 0x36 then it is not FAT. However are you sure about memcmp(ms->ms_magic, "JFS ", 8) and memcmp(ms->ms_magic, "HPFS ", 8) ? They seem not to have enough spaces. -- blkid library detects JFS partition as FAT32 https://bugs.launchpad.net/bugs/255255 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 255255] Re: blkid library detects JFS partition as FAT32
On Sun, Aug 24, 2008 at 11:28:42AM +0100, Jan van Wijk wrote:
> >Hmm, looking at what you sent me, it looks like DFSee was being
> >"clever". It looks like it deliberately put something which looks
> >like almost exactly like a FAT12/FAT16 header, passing all of the
> >normal checks one might make for seeing if the various fields make
> >sense.
>
> Those fields contents (similar to FAT) are actually standard for
> any JFS filesystem formatted by the OS/2 FORMAT command
> (or rather UJFS.DLL on an OS/2 or eComStation system :-)
Thanks, that's useful information. The filesystem in question was
almost certainly created by a Linux user who didn't use OS/2 to format
his partition --- and /sbin/mkfs.jfs zeros out the 32k of the
filesystem to avoid confusion by filesystem identification programs
like blkid, vol_id, disktype, et. al.
And while I do care about making blkid be able to support ancient
filesystems for obsolete operating systems (and the
file-system-sampler from disktype.cvs.sourceforge.net has an excellent
set of filesystem exemplars for testing, including an Atari ST floppy
and a BeOS BFS filesystem), the primary goal is to be able to detect
filesystems that are likely to be used by Linux users --- so I've only
seen this situation with this pseudo FAT12/16 header with DFSee.
Maybe there are other partition management programs that cause this,
but I've not seen it in my experience.
> >The reason why I don't just check ms_magic is that there have been
> >some (badly behaved) FAT filesystem formatters which don't set
> >ms_magic, or set it to some garbage value other than "FAT12 " or
> >"FAT16 ".
>
> OK, understand, but you may see the same 'FAT like fields' for OS/2
> formatted JFS filesystems with a ms_sysid of 'IBM 4.50' ...
>
> Actually, bootsectors for an HPFS filesystems look very much like
> these as well, so you may see a similar behaviour there too.
Hmm, so maybe what I should do is put in an explicit check for "JFS "
and "HPFS " in the ms_magic field (at offset 0x36) and fail it as a
FAT filesystem if I see those magic values there. That should cover
the situation with OS/2 formatted filesystems.
Out of curiosity, does DFSee try to do something similar with any
other filesystems? Or is this something you only did for JFS and HPFS
to replicate what OS/2 was doing?
> I assume your code (that analyses the bootsector) does take into
> account that the 'ms_magic' field on FAT32 is at a different offset ?
> It is at 0x36 for FAT16 (and others) but at 0x52 for FAT32
Oh yes, e2fsprogs definitely checks for "FAT32 " and "MSWIN" at offset
0x52.
One of the things which confused me is why DFSee (and OS/2,
apparently, according to your reports) is using a FAT12/16 style
pseudo-superblock, and not a FAT32 style superblock. I also wonder
what programs were actually looking at the FAT12/16-style pseudo
superblock information for JFS and HPFS filesystems, since clearly
Linux doesn't need it, and as far as I know the MBR boot sector
doesn't require it to be present.
In any case, thanks, Jan, for your comments, they have been very
helpful!
- Ted
P.S. Here's the new version of the patch which I've applied to blkid.
commit 78d89cda680498957e4ad78602751d1f905cee08
Author: Theodore Ts'o <[EMAIL PROTECTED]>
Date: Sat Aug 23 22:11:01 2008 -0400
libblkid: Fix false detection of DFSee created filesystems.
OS/2 and DFSee creates a pseudo FAT-12/16 header in the first 512
bytes of a filesystem which looks enough like a FAT-12/16 to fool
blkid. Part of this is because we don't require ms_magic or vs_magic
to be the strings "FAT12 ", "FAT16 ", or "FAT32 ", since some FAT
filesystem formatters don't set ms_magic or vs_magic. To address
this, we explicitly test for "JFS " and "HPFS" in ms_magic,
and if they are found, we assume the filesystem is definitely not
a FAT filesystem.
Addresses-Launchpad-Bug: #255255
Signed-off-by: "Theodore Ts'o" <[EMAIL PROTECTED]>
diff --git a/lib/blkid/probe.c b/lib/blkid/probe.c
index e2d8a25..225116a 100644
--- a/lib/blkid/probe.c
+++ b/lib/blkid/probe.c
@@ -566,25 +566,38 @@ static int probe_fat_nomagic(struct blkid_probe *probe,
struct blkid_magic *id __BLKID_ATTR((unused)),
unsigned char *buf)
{
- struct vfat_super_block *vs;
+ struct msdos_super_block *ms;
- vs = (struct vfat_super_block *)buf;
+ ms = (struct msdos_super_block *)buf;
/* heads check */
- if (vs->vs_heads == 0)
+ if (ms->ms_heads == 0)
return 1;
/* cluster size check*/
- if (vs->vs_cluster_size == 0 ||
- (vs->vs_cluster_size & (vs->vs_cluster_size-1)))
+ if (ms->ms_cluster_size == 0 ||
+ (ms->ms_cluster_size & (ms->ms_cluster_size-1)))
return 1;
/* media check */
- if (vs->vs_media < 0xf8 &
Re: [Bug 255255] Re: blkid library detects JFS partition as FAT32
On Wed, Aug 06, 2008 at 03:34:30PM -, boga wrote:
> May be you can improve JFS test strength by adding check for the correct
> relations between the superblock fields: s_bsize, s_l2bsize,
> s_l2bfactor, s_pbsize, s_l2pbsize?
This is also a good idea, even if I don't move the probe order of the
tests around. (The patch to explicitly check for DFSee should solve
your particular issue.)
- Ted
commit ddd46b9b85c158fb51c0ba45ec11b7c032fb457f
Author: Theodore Ts'o <[EMAIL PROTECTED]>
Date: Sat Aug 23 22:27:22 2008 -0400
libblkid: Strength the JFS probe routine
Check to make sure a JFS filesystem is really correct by checking the
relationship between the following fields in the JFS superblock:
s_bsize, s_l2bsize, s_pbsize, s_l2pbsize, and s_l2bfactor. Thanks to
Lesh Bogdanow for this suggestion.
Signed-off-by: "Theodore Ts'o" <[EMAIL PROTECTED]>
diff --git a/lib/blkid/probe.c b/lib/blkid/probe.c
index 31f3c74..91db9a1 100644
--- a/lib/blkid/probe.c
+++ b/lib/blkid/probe.c
@@ -781,6 +781,16 @@ static int probe_jfs(struct blkid_probe *probe,
js = (struct jfs_super_block *)buf;
+ if (blkid_le32(js->js_bsize) != (1 << blkid_le16(js->js_l2bsize)))
+ return 1;
+
+ if (blkid_le32(js->js_pbsize) != (1 << blkid_le16(js->js_l2pbsize)))
+ return 1;
+
+ if ((blkid_le16(js->js_l2bsize) - blkid_le16(js->js_l2pbsize)) !=
+ blkid_le16(js->js_l2bfactor))
+ return 1;
+
if (strlen((char *) js->js_label))
label = (char *) js->js_label;
blkid_set_tag(probe->dev, "LABEL", label, sizeof(js->js_label));
diff --git a/lib/blkid/probe.h b/lib/blkid/probe.h
index ac3a083..993156d 100644
--- a/lib/blkid/probe.h
+++ b/lib/blkid/probe.h
@@ -181,10 +181,13 @@ struct jfs_super_block {
unsigned char js_magic[4];
__u32 js_version;
__u64 js_size;
- __u32 js_bsize;
- __u32 js_dummy1;
- __u32 js_pbsize;
- __u32 js_dummy2[27];
+ __u32 js_bsize; /* 4: aggregate block size in bytes */
+ __u16 js_l2bsize; /* 2: log2 of s_bsize */
+ __u16 js_l2bfactor; /* 2: log2(s_bsize/hardware block size)
*/
+ __u32 js_pbsize; /* 4: hardware/LVM block size in bytes
*/
+ __u16 js_l2pbsize;/* 2: log2 of s_pbsize */
+ __u16 js_pad; /* 2: padding necessary for alignment */
+ __u32 js_dummy2[26];
unsigned char js_uuid[16];
unsigned char js_label[16];
unsigned char js_loguuid[16];
--
blkid library detects JFS partition as FAT32
https://bugs.launchpad.net/bugs/255255
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 255255] Re: blkid library detects JFS partition as FAT32
FYI, this is what I've checked into the e2fsprogs git repository. - Ted commit 8305cdf401b05e3748b87d0b63c4fa5bad8f8eb6 Author: Theodore Ts'o <[EMAIL PROTECTED]> Date: Sat Aug 23 22:11:01 2008 -0400 libblkid: Fix false detection of DFSee created filesystems. DFSee appears to create a pseudo FAT-12/16 header in the first 512 bytes of a filesystem which looks enough like a FAT-12/16 to fool blkid. Part of this is because we don't require ms_magic or vs_magic to be the strings "FAT12 ", "FAT16 ", or "FAT32 ", since some FAT filesystem formatters don't set ms_magic or vs_magic. To address this, we add an explicit test for DFSee in the ms_sysid field, and if ms_sysid has a DFSee signature, then we require the ms_magic/vs_magic fields to be filled in appropriately. Signed-off-by: "Theodore Ts'o" <[EMAIL PROTECTED]> diff --git a/lib/blkid/probe.c b/lib/blkid/probe.c index e2d8a25..31f3c74 100644 --- a/lib/blkid/probe.c +++ b/lib/blkid/probe.c @@ -587,6 +587,20 @@ static int probe_fat_nomagic(struct blkid_probe *probe, if (!vs->vs_fats) return 1; + /* + * The DFSee product apparently creates a FAT12-like boot + * block for non-FAT filesystems. It seems to do this at + * least for JFS (see Launchpad Bug #255255) and possibly + * others. We already check for the FAT12/16/32 magic strings + * in the probe table, so if we got here, it must be because + * we have something created by DFSee but which isn't actually + * a FAT filesystem. I'm assuming here that DFSee actually + * does fill in ms_magic and vs_magic correctly, but that + * seems like a safe bet. + */ + if (memcmp(vs->vs_sysid, "DFSee", 5) == 0) + return 1; + return probe_fat(probe, id, buf); } -- blkid library detects JFS partition as FAT32 https://bugs.launchpad.net/bugs/255255 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 255255] Re: blkid library detects JFS partition as FAT32
(Sorry for the delay in getting back to you; I've been travelling.)
Hmm, looking at what you sent me, it looks like DFSee was being
"clever". It looks like it deliberately put something which looks
like almost exactly like a FAT12/FAT16 header, passing all of the
normal checks one might make for seeing if the various fields make
sense.
(gdb) p *((struct msdos_super_block *) buf)
$14 = {ms_ignored = "�\037", ms_sysid = "DFSee8.x",
ms_sector_size = "\000\002", ms_cluster_size = 8 '\b', ms_reserved = 1,
ms_fats = 2 '\002', ms_dir_entries = "\000\002", ms_sectors = "\000",
ms_media = 248 '�', ms_fat_length = 134, ms_secs_track = 63, ms_heads = 255,
ms_hidden = 63, ms_total_sect = 16868184, ms_unknown = "\200\001)",
ms_serno = "�e\223U", ms_label = "\000\000\000\000\000\000\000\000\000\000",
ms_magic = "JFS ",
ms_dummy2 = " [EMAIL PROTECTED]
9O\000d\000\b\000\b\000px\002\000S\000\000\000�w\002\034\000\000\000\000�w\002\034\200x\002\000�}\002\034\000\000\000\000^\003\000\000�\205\000\000�x\002\000\b\000\000\000\002\000\000\000\000\000\000'|x\002\000(y\002\000�\036�\037\b\000\000\000\b\000\000\000d\000\000\000
[EMAIL PROTECTED] y\002\000([EMAIL PROTECTED]
\000\000\000�y$\000\033\000\004\000\000\000\000\000\0009\006\000�y\002\000�y\210x\002\034�y\027",
ms_pmagic = "\000\026"}
The reason why I don't just check ms_magic is that there have been
some (badly behaved) FAT filesystem formatters which don't set
ms_magic, or set it to some garbage value other than "FAT12 " or
"FAT16 ".
What I suspect I will end up doing is checking to see if ms_sysid
begins with "DFSee" (since I assume "8.x" is the version number), and
if it is set, I will insist on ms_magic being "FAT12 " or "FAT16 ".
Can you do me a favor? Can you create for me a series of filesystems
that are FAT12, FAT16, FAT32, and perhaps a couple of other
filesystems that DFSee supports, using your version of DFSee, and send
me the first 32k of said filesystems? I want to see how what sort
"magic" it is putting into those filesystems, so I can arm blkid
appropriately to detect them.
I've also cc'ed the author of DFSee in hopes that he might be able to
shed a bit more light on what's going on here.
- Ted
--
blkid library detects JFS partition as FAT32
https://bugs.launchpad.net/bugs/255255
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 255255] Re: blkid library detects JFS partition as FAT32
May be you can improve JFS test strength by adding check for the correct relations between the superblock fields: s_bsize, s_l2bsize, s_l2bfactor, s_pbsize, s_l2pbsize? -- blkid library detects JFS partition as FAT32 https://bugs.launchpad.net/bugs/255255 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 255255] Re: blkid library detects JFS partition as FAT32
I've attached it to the e-mail listed in your profile. -- blkid library detects JFS partition as FAT32 https://bugs.launchpad.net/bugs/255255 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 255255] Re: blkid library detects JFS partition as FAT32
Can you send me the first 64k of the device? i.e.: sudo dd if=/dev/sdb5 of=/tmp/jfs.img bs=64k count=1 And then upload /tmp/jfs.img I have no doubt that moving the JFS-related entry before the VFAT entry will solve the problem, but the probe_fat_nomagic() does a lot of tests before concluding that the filesystem is VFAT. This leads me to believe that /dev/sdb5 was previously formatted as VFAT, and the JFS mkfs didn't do enough to eradicate the traces of the old VFAT partition. The problem is that there is a much weaker test for JFS, so this could lead to false positives in the other direction. (i.e., a JFS filesystem which is then formatted to be a VFAT partition.) So I'd like to get a copy of the beginning of the filesystem to see if there's a better fix. My guess is the other problem is that mkfs.jfs needs to do a better job wiping out the first few kilobytes of the image to prevent this kind of false positive. -- blkid library detects JFS partition as FAT32 https://bugs.launchpad.net/bugs/255255 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 255255] Re: blkid library detects JFS partition as FAT32
Studying lib/blkid/probe.c: the problem occurs because the JFS partition
matches the type_array[] entry:
{ "vfat", 0, 0x1fe, 2, "\125\252", probe_fat_nomagic },
The problem can be fixed by moving JFS-related entry before this VFAT entry.
** Attachment added: "Fix for JFS detected as FAT32"
http://launchpadlibrarian.net/16580737/e2fsprogs-1.40.8-jfs.diff
--
blkid library detects JFS partition as FAT32
https://bugs.launchpad.net/bugs/255255
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
