Thanks Rudi, that helps as you have good luck with all of them. I see they have some boards that go up to 192gb (but not DDR3), but some do 144gb as well. I just need to find out if the POD supports extended ATX and I see others have just used regular ATX boards.
-- Jason On Sunday, May 8, 2011 at 12:12 PM, Rudi Ahlers wrote: > > > On Sun, May 8, 2011 at 9:06 PM, Jason <slackmoehrle.li...@gmail.com> wrote: > > Rudy, > > > > > > Do you have a recommendation of a motherboard? > > Well, choose one here: http://www.supermicro.com/products/motherboard/matrix/ > > I don't have specific recommendations but we've had great success with all > our SuperMicro servers, both with single & dual CPU configurations, ranging > from 4GB - 128GB RAM > > > I am still reading the rest of your post. Thanks! > > > > -Jason > > > > -- > > Jason > > > > On Sunday, May 8, 2011 at 11:26 AM, Rudi Ahlers wrote: > > > > > > > > > On Sun, May 8, 2011 at 8:03 PM, Jason <slackmoehrle.li...@gmail.com> > > > wrote: > > > > Hi All, > > > > > > > > I am about to embark on a project that deals with allowing information > > > > archival, over time and seeing change over time as well. I can explain > > > > it a lot better, but I would certainly talk your ear off. I really > > > > don't have a lot of money to throw at the initial concept, but I have > > > > some. This device will host all of the operations for the first few > > > > months until I can afford to build a duplicate device. I already had a > > > > few parts of the idea done and ready to get live. > > > > > > > > I am contemplating building a BackBlaze Style POD. The goal of the > > > > device is to start acting as a place to have the crawls store > > > > information, massage it, get it into db's and then notify the user the > > > > task is done so they can start looking at the results. > > > > > > > > For reference here are a few links: > > > > > > > > http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/ > > > > > > > > and > > > > > > > > http://cleanenergy.harvard.edu/index.php?ira=Jabba&tipoContenido=sidebar&sidebar=science > > > > > > > > There is room for 45 drives in the case (technically a few more). > > > > > > > > 45 x 1tb 7200rpm drives is really cheap, about $60 each. > > > > > > > > 45 x 1.5tb 7200rpm drives are about $70 each. > > > > > > > > 45 x 2tb 7200rpm drives are about $120 each > > > > > > > > 45 x 3tb 7200rpm drives are about $180-$230 each (or more, some are > > > > almost $400) > > > > > > > > I have question before I commit to building one and I was hoping to get > > > > advice. > > > > > > > > 1. Can anyone recommend a mobo/processor setup that can hold lots of > > > > RAM? Like 24gb or 64gb or more? > > > > > > Any brand server motherboard will do. I prefer supermicro, but you can > > > use Dell, HP, Intell, etc, etc. > > > > > > > > > > > 2. Hardware RAID or Software RAID for this? > > > > > > Hardware RAID will be expensive on 45 drives. IF you can, split the 45 > > > drives into a few smaller RAID arrays. To rebuild 1x large 45TB RAID > > > array, with either hardware or software would probably take a week, or > > > more, depending on which RAID type you use - i.e. RAID 5, or 6, or 10. I > > > prefer RAID 10 since it's best for speed and the rebuilds are the > > > quickest. But you loose half the space, i.e. 45TB drives will give you > > > about 22TB space. 45x 2TB HDD's would give you about 44TB space though. > > > > > > > > > > > 3. Would CentOS be a good choice? I have never used CentOS on a device > > > > so massive. Just ordinary servers, so to speak. I assume that it could > > > > handle so many drives, a large, expanding file system. > > > > > > Yes it would be fine. > > > > > > > > > > 4. Someone recommended ZFS but I dont recall that being available on > > > > CentOS, but it is on FreeBSD which I have little experience with. > > > > > > I would also prefer to use ZFS for this type of setup. use one 128GB SL > > > type SSD drive as a cache drive to speed up things and 2x log drives to > > > help with drive recovery. With ZFS you would be able to use one large > > > RAID array if you have the log drives since it was recover from driver > > > failure much better than other file systems. Although you can install ZFS > > > as user-land tools, which will be slower than running it via the kernel. > > > But, it would be better to use Solaris or FreeBSD for this - look @ > > > Nexenta / FreeNAS / OpenIndia for this. > > > > > > > > > > > 5. How would someone realistically back something like this up? > > > > > > To another one as large :) > > > > > > OR, more realistically, if you already have some backup servers, and the > > > full 45TB isn't full of data yet, then simply backup what you have. By > > > the sounds of it your project is still new so your data won't be that > > > much. I would simply build a gluster / CLVM cluster of smaller cheaper > > > servers - which basically allows you to add say 4TB / 8TB (depending on > > > what chassis you use and how many drives it can take) at a time to the > > > backup cluster, which will be cheaper than buying another one identical > > > to this right now. > > > > > > > > > > > Ultimately I know over time I need to distribute my architecture out > > > > and have a number of web-servers, balancing, etc but to get started I > > > > think this device with good backups might fit the bill. > > > > > > If this device will be used for web + mail + SQL, then you may probably > > > look at using 4 quad core CPU's + 128GB RAM. With this many drives (or > > > rather, this much data) you'll probably run out of RAM / CPU / Network > > > resources before you run out of HDD space. > > > > > > > > > > > > With a device this big (in terms of storage) I would rather have 2 > > > separate "processing" servers which just mounts LUN's from this POD > > > (exported as NFS / iSCSI / FCoE / etc) and then have a few faster SAS / > > > SSD drives for SQL / log processing. > > > > > > > > > > > I can be way more detailed if it helps, I just didn't want to clutter > > > > with information that might not be relevant. > > > > -- > > > > Jason > > > > > > > > _______________________________________________ > > > > CentOS mailing list > > > > CentOS@centos.org > > > > http://lists.centos.org/mailman/listinfo/centos > > > > > > > > > -- > > > Kind Regards > > > Rudi Ahlers > > > SoftDux > > > > > > Website: http://www.SoftDux.com > > > Technical Blog: http://Blog.SoftDux.com > > > Office: 087 805 9573 > > > Cell: 082 554 7532 > > > _______________________________________________ > > > CentOS mailing list > > > CentOS@centos.org > > > http://lists.centos.org/mailman/listinfo/centos > > > > _______________________________________________ > > CentOS mailing list > > CentOS@centos.org > > http://lists.centos.org/mailman/listinfo/centos > > > -- > Kind Regards > Rudi Ahlers > SoftDux > > Website: http://www.SoftDux.com > Technical Blog: http://Blog.SoftDux.com > Office: 087 805 9573 > Cell: 082 554 7532 > _______________________________________________ > CentOS mailing list > CentOS@centos.org > http://lists.centos.org/mailman/listinfo/centos > _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
