Re: [PATCH] fat: Relax checks for sector size and media type
Pali Rohár writes: >> If there is real user to use that, I'm ok though (of course, need >> serious tests). However, FAT would be for exchange data with other >> devices, and there is "cluster per sector", and spec recommends sector >> size == device sector size. So I suspect this format is not useful. > > I looked into OpenBSD, FreeBSD and NetBSD source code and there is no > explicit upper limit for sector size. Just that sector size must be > power of two. > > I have not did tests yet, but you are right that some testing should be > done. > > As FAT operates with clusters and cluster size is defined by sector > size, then sectors per cluster and sector size defines cluster size. And > cluster size itself implies maximal size of FAT filesystem. > > So increasing sector size could be useful to create larger FAT32 > filesystems as current limit hit by sector size = 512 bytes. > > What do you think, which operating systems should be tested? Again, I suspect those custom extension (can't read by some uefi or windows) is not useful though. Testing on kernel that has PAGE_SIZE >= 8192, and setting FAT sector_size >= 8192. After that, it would be safe to remove 4096 limitation. Thanks. -- OGAWA Hirofumi
Re: [PATCH] fat: Relax checks for sector size and media type
Pali Rohár writes: >> If there is real user to use that, I'm ok though (of course, need >> serious tests). However, FAT would be for exchange data with other >> devices, and there is "cluster per sector", and spec recommends sector >> size == device sector size. So I suspect this format is not useful. > > I looked into OpenBSD, FreeBSD and NetBSD source code and there is no > explicit upper limit for sector size. Just that sector size must be > power of two. > > I have not did tests yet, but you are right that some testing should be > done. > > As FAT operates with clusters and cluster size is defined by sector > size, then sectors per cluster and sector size defines cluster size. And > cluster size itself implies maximal size of FAT filesystem. > > So increasing sector size could be useful to create larger FAT32 > filesystems as current limit hit by sector size = 512 bytes. > > What do you think, which operating systems should be tested? Again, I suspect those custom extension (can't read by some uefi or windows) is not useful though. Testing on kernel that has PAGE_SIZE >= 8192, and setting FAT sector_size >= 8192. After that, it would be safe to remove 4096 limitation. Thanks. -- OGAWA Hirofumi
Re: [PATCH] fat: Relax checks for sector size and media type
On Monday 03 September 2018 17:19:15 OGAWA Hirofumi wrote: > Pali Rohár writes: > > >> That source seems to check power_of_2(size) and 128 <= size <= > >> 4096. Rather why do you want to support larger than 4096? Or I'm missing > >> something? > > > > I looked into (Linux) mkfs.fat and it supports formatting disk also with > > sector size > 4096. Therefore I thought it may be good idea for ability > > to mount and use it (on Linux). > > > > I could check what other operating system would do with FAT sector size > > larger then 4096. > > If there is real user to use that, I'm ok though (of course, need > serious tests). However, FAT would be for exchange data with other > devices, and there is "cluster per sector", and spec recommends sector > size == device sector size. So I suspect this format is not useful. I looked into OpenBSD, FreeBSD and NetBSD source code and there is no explicit upper limit for sector size. Just that sector size must be power of two. I have not did tests yet, but you are right that some testing should be done. As FAT operates with clusters and cluster size is defined by sector size, then sectors per cluster and sector size defines cluster size. And cluster size itself implies maximal size of FAT filesystem. So increasing sector size could be useful to create larger FAT32 filesystems as current limit hit by sector size = 512 bytes. What do you think, which operating systems should be tested? -- Pali Rohár pali.ro...@gmail.com
Re: [PATCH] fat: Relax checks for sector size and media type
On Monday 03 September 2018 17:19:15 OGAWA Hirofumi wrote: > Pali Rohár writes: > > >> That source seems to check power_of_2(size) and 128 <= size <= > >> 4096. Rather why do you want to support larger than 4096? Or I'm missing > >> something? > > > > I looked into (Linux) mkfs.fat and it supports formatting disk also with > > sector size > 4096. Therefore I thought it may be good idea for ability > > to mount and use it (on Linux). > > > > I could check what other operating system would do with FAT sector size > > larger then 4096. > > If there is real user to use that, I'm ok though (of course, need > serious tests). However, FAT would be for exchange data with other > devices, and there is "cluster per sector", and spec recommends sector > size == device sector size. So I suspect this format is not useful. I looked into OpenBSD, FreeBSD and NetBSD source code and there is no explicit upper limit for sector size. Just that sector size must be power of two. I have not did tests yet, but you are right that some testing should be done. As FAT operates with clusters and cluster size is defined by sector size, then sectors per cluster and sector size defines cluster size. And cluster size itself implies maximal size of FAT filesystem. So increasing sector size could be useful to create larger FAT32 filesystems as current limit hit by sector size = 512 bytes. What do you think, which operating systems should be tested? -- Pali Rohár pali.ro...@gmail.com
Re: [PATCH] fat: Relax checks for sector size and media type
Pali Rohár writes: >> That source seems to check power_of_2(size) and 128 <= size <= >> 4096. Rather why do you want to support larger than 4096? Or I'm missing >> something? > > I looked into (Linux) mkfs.fat and it supports formatting disk also with > sector size > 4096. Therefore I thought it may be good idea for ability > to mount and use it (on Linux). > > I could check what other operating system would do with FAT sector size > larger then 4096. If there is real user to use that, I'm ok though (of course, need serious tests). However, FAT would be for exchange data with other devices, and there is "cluster per sector", and spec recommends sector size == device sector size. So I suspect this format is not useful. Thanks. -- OGAWA Hirofumi
Re: [PATCH] fat: Relax checks for sector size and media type
Pali Rohár writes: >> That source seems to check power_of_2(size) and 128 <= size <= >> 4096. Rather why do you want to support larger than 4096? Or I'm missing >> something? > > I looked into (Linux) mkfs.fat and it supports formatting disk also with > sector size > 4096. Therefore I thought it may be good idea for ability > to mount and use it (on Linux). > > I could check what other operating system would do with FAT sector size > larger then 4096. If there is real user to use that, I'm ok though (of course, need serious tests). However, FAT would be for exchange data with other devices, and there is "cluster per sector", and spec recommends sector size == device sector size. So I suspect this format is not useful. Thanks. -- OGAWA Hirofumi
Re: [PATCH] fat: Relax checks for sector size and media type
On Monday 03 September 2018 17:01:03 OGAWA Hirofumi wrote: > Pali Rohár writes: > > >> Just relaxing validation doesn't work. The block layer doesn't support > >> smaller than 512, and lager than PAGE_SIZE. (And in specification, fat > >> doesn't support lager than 4096.) > > > > Hi! I just sent this patch for discussion, with links to (now open > > source) Windows implementation. I guess that Windows driver > > implementation is more "authoritative" then Microsoft's own > > specification. It is known that Windows implementation does not match > > Microsoft specification. > > > > I know at least 3 FAT specifications (MS EFI FAT, MS/SD card FAT, > > ECMA-107) and you are right that Microsoft's one does not allow sector > > sizes larger then 4096. > > > > If there is limitation by block layer, then: > > > > 1) Why we do not check for PAGE_SIZE? > > That source seems to check power_of_2(size) and 128 <= size <= > 4096. Rather why do you want to support larger than 4096? Or I'm missing > something? I looked into (Linux) mkfs.fat and it supports formatting disk also with sector size > 4096. Therefore I thought it may be good idea for ability to mount and use it (on Linux). I could check what other operating system would do with FAT sector size larger then 4096. > > 2) Is check in fat driver really needed (if block layer checks it)? > > Yes, isolating block layer error and fat format error to be better error > report. Ok. -- Pali Rohár pali.ro...@gmail.com
Re: [PATCH] fat: Relax checks for sector size and media type
On Monday 03 September 2018 17:01:03 OGAWA Hirofumi wrote: > Pali Rohár writes: > > >> Just relaxing validation doesn't work. The block layer doesn't support > >> smaller than 512, and lager than PAGE_SIZE. (And in specification, fat > >> doesn't support lager than 4096.) > > > > Hi! I just sent this patch for discussion, with links to (now open > > source) Windows implementation. I guess that Windows driver > > implementation is more "authoritative" then Microsoft's own > > specification. It is known that Windows implementation does not match > > Microsoft specification. > > > > I know at least 3 FAT specifications (MS EFI FAT, MS/SD card FAT, > > ECMA-107) and you are right that Microsoft's one does not allow sector > > sizes larger then 4096. > > > > If there is limitation by block layer, then: > > > > 1) Why we do not check for PAGE_SIZE? > > That source seems to check power_of_2(size) and 128 <= size <= > 4096. Rather why do you want to support larger than 4096? Or I'm missing > something? I looked into (Linux) mkfs.fat and it supports formatting disk also with sector size > 4096. Therefore I thought it may be good idea for ability to mount and use it (on Linux). I could check what other operating system would do with FAT sector size larger then 4096. > > 2) Is check in fat driver really needed (if block layer checks it)? > > Yes, isolating block layer error and fat format error to be better error > report. Ok. -- Pali Rohár pali.ro...@gmail.com
Re: [PATCH] fat: Relax checks for sector size and media type
Pali Rohár writes: >> Just relaxing validation doesn't work. The block layer doesn't support >> smaller than 512, and lager than PAGE_SIZE. (And in specification, fat >> doesn't support lager than 4096.) > > Hi! I just sent this patch for discussion, with links to (now open > source) Windows implementation. I guess that Windows driver > implementation is more "authoritative" then Microsoft's own > specification. It is known that Windows implementation does not match > Microsoft specification. > > I know at least 3 FAT specifications (MS EFI FAT, MS/SD card FAT, > ECMA-107) and you are right that Microsoft's one does not allow sector > sizes larger then 4096. > > If there is limitation by block layer, then: > > 1) Why we do not check for PAGE_SIZE? That source seems to check power_of_2(size) and 128 <= size <= 4096. Rather why do you want to support larger than 4096? Or I'm missing something? > 2) Is check in fat driver really needed (if block layer checks it)? Yes, isolating block layer error and fat format error to be better error report. -- OGAWA Hirofumi
Re: [PATCH] fat: Relax checks for sector size and media type
Pali Rohár writes: >> Just relaxing validation doesn't work. The block layer doesn't support >> smaller than 512, and lager than PAGE_SIZE. (And in specification, fat >> doesn't support lager than 4096.) > > Hi! I just sent this patch for discussion, with links to (now open > source) Windows implementation. I guess that Windows driver > implementation is more "authoritative" then Microsoft's own > specification. It is known that Windows implementation does not match > Microsoft specification. > > I know at least 3 FAT specifications (MS EFI FAT, MS/SD card FAT, > ECMA-107) and you are right that Microsoft's one does not allow sector > sizes larger then 4096. > > If there is limitation by block layer, then: > > 1) Why we do not check for PAGE_SIZE? That source seems to check power_of_2(size) and 128 <= size <= 4096. Rather why do you want to support larger than 4096? Or I'm missing something? > 2) Is check in fat driver really needed (if block layer checks it)? Yes, isolating block layer error and fat format error to be better error report. -- OGAWA Hirofumi
Re: [PATCH] fat: Relax checks for sector size and media type
On Monday 03 September 2018 16:17:26 OGAWA Hirofumi wrote: > Pali Rohár writes: > > > Windows fastfat.sys driver accepts also media types 0x00 and 0x01 and > > sector sizes 128 and 256 bytes. Linux mkfs.fat can format disk also to > > larger FAT sector sizes then 4096 bytes, therefore relax also upper limit > > restriction. > > > - if (!is_power_of_2(bpb->fat_sector_size) > > - || (bpb->fat_sector_size < 512) > > - || (bpb->fat_sector_size > 4096)) { > > + if (!is_power_of_2(bpb->fat_sector_size)) { > > Just relaxing validation doesn't work. The block layer doesn't support > smaller than 512, and lager than PAGE_SIZE. (And in specification, fat > doesn't support lager than 4096.) Hi! I just sent this patch for discussion, with links to (now open source) Windows implementation. I guess that Windows driver implementation is more "authoritative" then Microsoft's own specification. It is known that Windows implementation does not match Microsoft specification. I know at least 3 FAT specifications (MS EFI FAT, MS/SD card FAT, ECMA-107) and you are right that Microsoft's one does not allow sector sizes larger then 4096. If there is limitation by block layer, then: 1) Why we do not check for PAGE_SIZE? 2) Is check in fat driver really needed (if block layer checks it)? > > static inline int fat_valid_media(u8 media) > > { > > - return 0xf8 <= media || media == 0xf0; > > + return 0xf8 <= media || media == 0xf0 || media == 0x00 || media == 0x01; > > } > > #endif /* !_LINUX_MSDOS_FS_H */ > > This is ok though, this would be for ancient floppy media. Ok. -- Pali Rohár pali.ro...@gmail.com
Re: [PATCH] fat: Relax checks for sector size and media type
On Monday 03 September 2018 16:17:26 OGAWA Hirofumi wrote: > Pali Rohár writes: > > > Windows fastfat.sys driver accepts also media types 0x00 and 0x01 and > > sector sizes 128 and 256 bytes. Linux mkfs.fat can format disk also to > > larger FAT sector sizes then 4096 bytes, therefore relax also upper limit > > restriction. > > > - if (!is_power_of_2(bpb->fat_sector_size) > > - || (bpb->fat_sector_size < 512) > > - || (bpb->fat_sector_size > 4096)) { > > + if (!is_power_of_2(bpb->fat_sector_size)) { > > Just relaxing validation doesn't work. The block layer doesn't support > smaller than 512, and lager than PAGE_SIZE. (And in specification, fat > doesn't support lager than 4096.) Hi! I just sent this patch for discussion, with links to (now open source) Windows implementation. I guess that Windows driver implementation is more "authoritative" then Microsoft's own specification. It is known that Windows implementation does not match Microsoft specification. I know at least 3 FAT specifications (MS EFI FAT, MS/SD card FAT, ECMA-107) and you are right that Microsoft's one does not allow sector sizes larger then 4096. If there is limitation by block layer, then: 1) Why we do not check for PAGE_SIZE? 2) Is check in fat driver really needed (if block layer checks it)? > > static inline int fat_valid_media(u8 media) > > { > > - return 0xf8 <= media || media == 0xf0; > > + return 0xf8 <= media || media == 0xf0 || media == 0x00 || media == 0x01; > > } > > #endif /* !_LINUX_MSDOS_FS_H */ > > This is ok though, this would be for ancient floppy media. Ok. -- Pali Rohár pali.ro...@gmail.com
Re: [PATCH] fat: Relax checks for sector size and media type
Pali Rohár writes: > Windows fastfat.sys driver accepts also media types 0x00 and 0x01 and > sector sizes 128 and 256 bytes. Linux mkfs.fat can format disk also to > larger FAT sector sizes then 4096 bytes, therefore relax also upper limit > restriction. > - if (!is_power_of_2(bpb->fat_sector_size) > - || (bpb->fat_sector_size < 512) > - || (bpb->fat_sector_size > 4096)) { > + if (!is_power_of_2(bpb->fat_sector_size)) { Just relaxing validation doesn't work. The block layer doesn't support smaller than 512, and lager than PAGE_SIZE. (And in specification, fat doesn't support lager than 4096.) > static inline int fat_valid_media(u8 media) > { > - return 0xf8 <= media || media == 0xf0; > + return 0xf8 <= media || media == 0xf0 || media == 0x00 || media == 0x01; > } > #endif /* !_LINUX_MSDOS_FS_H */ This is ok though, this would be for ancient floppy media. Thanks. -- OGAWA Hirofumi
Re: [PATCH] fat: Relax checks for sector size and media type
Pali Rohár writes: > Windows fastfat.sys driver accepts also media types 0x00 and 0x01 and > sector sizes 128 and 256 bytes. Linux mkfs.fat can format disk also to > larger FAT sector sizes then 4096 bytes, therefore relax also upper limit > restriction. > - if (!is_power_of_2(bpb->fat_sector_size) > - || (bpb->fat_sector_size < 512) > - || (bpb->fat_sector_size > 4096)) { > + if (!is_power_of_2(bpb->fat_sector_size)) { Just relaxing validation doesn't work. The block layer doesn't support smaller than 512, and lager than PAGE_SIZE. (And in specification, fat doesn't support lager than 4096.) > static inline int fat_valid_media(u8 media) > { > - return 0xf8 <= media || media == 0xf0; > + return 0xf8 <= media || media == 0xf0 || media == 0x00 || media == 0x01; > } > #endif /* !_LINUX_MSDOS_FS_H */ This is ok though, this would be for ancient floppy media. Thanks. -- OGAWA Hirofumi