Re: [zfs-discuss] Losts of small files vs fewer big files
> "dt" == Don Turnbull writes: dt> Any idea why this is? maybe prefetch? WAG, though. dt> I work with Greenplum which is essentially a number of dt> Postgres database instances clustered together. haha, yeah I know who you are. Too bad the open source postgres can't do that. :/ AFFERO. pgpK4PMndIbty.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Losts of small files vs fewer big files
Thanks for the suggestion! We've fiddled with this in the past. Our app is 32k instead of 8k blocks and it is data warehousing so the I/O model is a lot more long sequential reads generally. Changing the blocksize has very little effect on us. I'll have to look at fsync; hadn't considered that. Compression is a killer; it costs us up to 50% of the performance sadly. CPU is not always a problem for us but it can be depending on the query workload and the servers involved. Bryan Allen wrote: Have you set the recordsize for the filesystem to the blocksize Postgres is using (8K)? Note this has to be done before any files are created. Other thoughts: Disable postgres's fsync, enable filesystem compression if disk I/O is your bottleneck as opposed to CPU. I do this with MySQL and it has proven useful. My rule of thumb there is 60% for InnoDB cache, 40% for ZFS ARC, but YMMV. http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Losts of small files vs fewer big files
Have you set the recordsize for the filesystem to the blocksize Postgres is using (8K)? Note this has to be done before any files are created. Other thoughts: Disable postgres's fsync, enable filesystem compression if disk I/O is your bottleneck as opposed to CPU. I do this with MySQL and it has proven useful. My rule of thumb there is 60% for InnoDB cache, 40% for ZFS ARC, but YMMV. http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide -- bda cyberpunk is dead. long live cyberpunk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
