Re: ZFS installs on HD with 4k physical blocks without any warning as on 512 block size device
On 23/08/2011 03:23, Peter Jeremy wrote: On 2011-Aug-22 12:45:08 +0200, Ivan Vorasivo...@freebsd.org wrote: It would be suboptimal but only for the slight waste of space that would have otherwise been reclaimed if the block or fragment size remained 512 or 2K. This waste of space is insignificant for the vast majority of users and there are no performance penalties, so it seems that switching to 4K sectors by default for all file systems would actually be a good idea. This is heavily dependent on the size distribution. I can't quickly check for ZFS but I've done some quick checks on UFS. The following are sizes in MB for my copies of the listed trees with different UFS frag size. These include directories but not indirect blocks: 1b 512b 1024b 2048b 4096b 4430 4511 4631 4875 5457 /usr/ncvs 4910 5027 5181 5499 6133 Old FreeBSD SVN repo 299 370 485733 1252 /usr/ports cheched out from CVS 467 485 509557656 /usr/src 8-stable checkout from CVS Note that the ports tree grew by 50% going from 1K to 2K frags and will grow by another 70% going to 4KB frags. Similar issues will be seen when you have lots of small file. I agree but there are at least two things going for making the increase anyway: 1) 2 TB drives cost $80 2) Where the space is really important, the person in charge usually knows it and can choose a non-default size like 512b fragments. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: ZFS installs on HD with 4k physical blocks without any warning as on 512 block size device
On 23 August 2011 10:52, Ivan Voras ivo...@freebsd.org wrote: I agree but there are at least two things going for making the increase anyway: 1) 2 TB drives cost $80 2) Where the space is really important, the person in charge usually knows it and can choose a non-default size like 512b fragments. helpers like sysinstall should help with choosing the smaller blocks for smaller drives (especially SSD) Aled ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: ZFS installs on HD with 4k physical blocks without any warning as on 512 block size device
On 23/08/2011 11:59, Aled Morris wrote: On 23 August 2011 10:52, Ivan Vorasivo...@freebsd.org wrote: I agree but there are at least two things going for making the increase anyway: 1) 2 TB drives cost $80 2) Where the space is really important, the person in charge usually knows it and can choose a non-default size like 512b fragments. helpers like sysinstall should help with choosing the smaller blocks for smaller drives (especially SSD) Only via hints and help text. Too much magic in the installer leads to awkward choices :) (e.g. first you need to distinguish between a VM with a small drive, a SSD small drive, or a SAN small volume... it quickly turns into an AI-class problem). ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: ZFS installs on HD with 4k physical blocks without any warning as on 512 block size device
On 19/08/2011 14:21, Aled Morris wrote: On 19 August 2011 11:15, Tom Evanstevans...@googlemail.com wrote: On Thu, Aug 18, 2011 at 6:50 PM, Yuriy...@rawbw.com wrote: Some latest hard drives have logical sectors of 512 byte when they actually have 4k physical sectors. ... Shouldn't UFS and ZFS drivers be able to either read the right sector size from the underlying device or at least issue a warning? The device never reports the actual sector size, so unless FreeBSD keeps a database of 4k sector hard drives that report as 512 byte sector hard drives, there is nothing that can be done. At what point should we change the default in newfs/zfs to 4k? It is already changed for UFS in 9. I guess formatting the filesystem for 4k sectors on a 512b drive would still work but it would be suboptimal. What would the performance penalty be in reality? It would be suboptimal but only for the slight waste of space that would have otherwise been reclaimed if the block or fragment size remained 512 or 2K. This waste of space is insignificant for the vast majority of users and there are no performance penalties, so it seems that switching to 4K sectors by default for all file systems would actually be a good idea. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: ZFS installs on HD with 4k physical blocks without any warning as on 512 block size device
On 2011-Aug-22 12:45:08 +0200, Ivan Voras ivo...@freebsd.org wrote: It would be suboptimal but only for the slight waste of space that would have otherwise been reclaimed if the block or fragment size remained 512 or 2K. This waste of space is insignificant for the vast majority of users and there are no performance penalties, so it seems that switching to 4K sectors by default for all file systems would actually be a good idea. This is heavily dependent on the size distribution. I can't quickly check for ZFS but I've done some quick checks on UFS. The following are sizes in MB for my copies of the listed trees with different UFS frag size. These include directories but not indirect blocks: 1b 512b 1024b 2048b 4096b 4430 4511 4631 4875 5457 /usr/ncvs 4910 5027 5181 5499 6133 Old FreeBSD SVN repo 299 370 485733 1252 /usr/ports cheched out from CVS 467 485 509557656 /usr/src 8-stable checkout from CVS Note that the ports tree grew by 50% going from 1K to 2K frags and will grow by another 70% going to 4KB frags. Similar issues will be seen when you have lots of small file. -- Peter Jeremy pgp3V7msgMGk0.pgp Description: PGP signature
Re: ZFS installs on HD with 4k physical blocks without any warning as on 512 block size device
On Thu, Aug 18, 2011 at 6:50 PM, Yuri y...@rawbw.com wrote: Some latest hard drives have logical sectors of 512 byte when they actually have 4k physical sectors. Here is the document describing what to do in such case: http://ivoras.net/blog/tree/2011-01-01.freebsd-on-4k-sector-drives.html . For UFS: newfs -U -f 4096 /dev/da0 For ZFS: gnop create -S 4096 /dev/da0 zpool create data /dev/da0.nop I am sure most people just install such hard drive without doing this and potentially get suboptimal performance since they aren't aware about this. Shouldn't UFS and ZFS drivers be able to either read the right sector size from the underlying device or at least issue a warning? Yuri The device never reports the actual sector size, so unless FreeBSD keeps a database of 4k sector hard drives that report as 512 byte sector hard drives, there is nothing that can be done. Cheers Tom ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: ZFS installs on HD with 4k physical blocks without any warning as on 512 block size device
On Friday, August 19, 2011 12:15:30 PM Tom Evans wrote: On Thu, Aug 18, 2011 at 6:50 PM, Yuri y...@rawbw.com wrote: Some latest hard drives have logical sectors of 512 byte when they actually have 4k physical sectors. Here is the document describing what to do in such case: http://ivoras.net/blog/tree/2011-01-01.freebsd-on-4k-sector-drives.html . For UFS: newfs -U -f 4096 /dev/da0 For ZFS: gnop create -S 4096 /dev/da0 zpool create data /dev/da0.nop I am sure most people just install such hard drive without doing this and potentially get suboptimal performance since they aren't aware about this. Shouldn't UFS and ZFS drivers be able to either read the right sector size from the underlying device or at least issue a warning? Yuri The device never reports the actual sector size, so unless FreeBSD keeps a database of 4k sector hard drives that report as 512 byte sector hard drives, there is nothing that can be done. Cheers Tom In -current at least there is a quirk table for these drives - the stripe size is set to 4K. Other tools use this size to align stuff on. Also, the default fragment size of UFS was increased to 4K. - Pieter ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: ZFS installs on HD with 4k physical blocks without any warning as on 512 block size device
On 19 August 2011 11:15, Tom Evans tevans...@googlemail.com wrote: On Thu, Aug 18, 2011 at 6:50 PM, Yuri y...@rawbw.com wrote: Some latest hard drives have logical sectors of 512 byte when they actually have 4k physical sectors. ... Shouldn't UFS and ZFS drivers be able to either read the right sector size from the underlying device or at least issue a warning? The device never reports the actual sector size, so unless FreeBSD keeps a database of 4k sector hard drives that report as 512 byte sector hard drives, there is nothing that can be done. At what point should we change the default in newfs/zfs to 4k? I guess formatting the filesystem for 4k sectors on a 512b drive would still work but it would be suboptimal. What would the performance penalty be in reality? Aled ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
ZFS installs on HD with 4k physical blocks without any warning as on 512 block size device
Some latest hard drives have logical sectors of 512 byte when they actually have 4k physical sectors. Here is the document describing what to do in such case: http://ivoras.net/blog/tree/2011-01-01.freebsd-on-4k-sector-drives.html . For UFS: newfs -U -f 4096 /dev/da0 For ZFS: gnop create -S 4096 /dev/da0 zpool create data /dev/da0.nop I am sure most people just install such hard drive without doing this and potentially get suboptimal performance since they aren't aware about this. Shouldn't UFS and ZFS drivers be able to either read the right sector size from the underlying device or at least issue a warning? Yuri ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
