Hello, I'd like to plan a storage solution for a system currently in production.
The system's storage is based on code which writes many files to the file system, with overall storage needs currently around 40TB and expected to reach hundreds of TBs. The average file size of the system is ~100K, which translates to ~500 million files today, and billions of files in the future. This storage is accessed over NFS by a rack of 40 Linux blades, and is mostly read-only (99% of the activity is reads). While I realize calling this sub-optimal system design is probably an understatement, the design of the system is beyond my control and isn't likely to change in the near future. The system's current storage is based on 4 VxFS filesystems, created on SVM meta-devices each ~10TB in size. A 2-node Sun Cluster serves the filesystems, 2 filesystems per node. Each of the filesystems undergoes growfs as more storage is made available. We're looking for an alternative solution, in an attempt to improve performance and ability to recover from disasters (fsck on 2^42 files isn't practical, and I'm getting pretty worried due to this fact - even the smallest filesystem inconsistency will leave me lots of useless bits). Question is - does anyone here have experience with large ZFS filesystems with many small-files? Is it practical to base such a solution on a few (8) zpools, each with single large filesystem in it? Many thanks in advance for any advice, - Yaniv This message posted from opensolaris.org
