Re: [ilugd] NAS idea
--- Mithun Bhattacharya [EMAIL PROTECTED] wrote: If my understanding is correct a simple NAS is: 1. A storage space available over the network 2. Fault tolerant 3. Expandable 4. Transparent 5. Compatible across OSes The way I see it is I need the following to get it going 1. Gigabit network 2. RAID 1/5 3. LVM 4. RAID and LVM can be configured to handle hot swap 5. Share disk space using NFS/SAMBA Those who have been tracking this thread might want to read this (rather enjoyable) discussion as well: http://ask.slashdot.org/askslashdot/06/01/16/1840209.shtml Prashant Verma ___ ilugd mailinglist -- ilugd@lists.linux-delhi.org http://frodo.hserus.net/mailman/listinfo/ilugd Archives at: http://news.gmane.org/gmane.user-groups.linux.delhi http://www.mail-archive.com/ilugd@lists.linux-delhi.org/
Re: [ilugd] NAS idea
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Mithun Bhattacharya Sent: Saturday, January 14, 2006 1:00 PM To: The Linux-Delhi mailing list Subject: Re: [ilugd] NAS idea --- Manish Verma [EMAIL PROTECTED] wrote: i would like to add few more points while designing a NAS solution 1.writing of data on to HDD should be fast enough, at time when you are mounting central storage on multiple server the concurrent NFS operation makes it very difficult and the write on the box slows down e.g netapp uses wafl filesystem (or for that matter all NAS storage uses the same way of writing through cache) which does the writing of data from cache. Your solution should be based on using the cache to the maximum for both read and write. I have seen xfs,ext3 going down in the load condition. I guess corporate customers would like SCSI disks, even home users shouldnt go for anything less than SATA disks. ** Its not the question of corporate of SCSI, people are using SATA disk in production as well. Its based on your requirement. netapp R200, Intransa IP SAN, they all use either SATA or PATA disks. If you are writing data from cache performence is not a problem only thing which you need to take care while using SATA / PATA is of redundancy, Dual Parity does that. I do recall NFS having lad issues that definitely needs to be looked into. Guess we will have to develop a proper set of test cases based on your feedback :). * If you are ready with you NAS header solution i can offer you a DE in my IDC. I would be more than happy to be your beta customer. 2. You will have to check the boot time of your NAS box and it should be under control there are NAS boxes ( i would not put the name here, but you can easily find it out) which takes 15-20 minutes to boot. The box should be able to boot in 3-4 minutes even if it a abnormal reboot. I have used pretty simple installations of LVM over RAID5 on Fedora Core 3 and it did come up in a few min. I do agree complex LVM and RAID installations need to be tested. 3.you should benchmark your NFS box for NFS operation not I/O operation on disk. You may get good I/O on disk but may not be able to get good NFS operation. 4.The redundancy of storage array should not be limited to RAID5 because now the storage capacity of individual disks are going high (500GB SATA, 300GB SCSI and FCAL are in the market) you should be able to handle dual disk failure in the same RAID group lets say you have 14 disk RAID group and your one disk failed and rebuilding is going on (which will take some time as the capacity is more) and during that time if your second disk also fails then your whole of the RAID volume will go offline, you should have dual parity kind of solution. Well isnt the concept of spare disks in RAID meant for these purposes ? *** spare disks are used accross the different RAID group they are teh global spares. If you have multiple RAID grup in a single box you can fail one disk in each raid group and spare will take over but if you are failing more than one disk raid group will crash. 5.NAS solution should be modular i.e i should be able to add storage and processing both e.g if i am handling n NFS operation using one NAS header today with NTB storage and tomorrow if i want to increase my NFS operation i should be able to add more processing (NAS header to the same storage) and lets say i don't want to increase the NFS operation but i want to add more storage i should be able to do that as well and that too on the fly because if my box is in production i cant shut it down. Adding disks would be limited by the enclosure holding them. A highly modular system I am afraid wont be there in a first cut we would probably have to set up a research lab or something where CPU and hard disks both are modular and are being used efficiently. Also if I add a CPU does it contribute to a existing box or creates a new box. There are various degrees of modularity which could be achieved but definitely not everything in the first version. * Its not CPU and disk modularity...it should be DEs(disk enclosures) and Controller (NAS header) modularity i was talking about. May be there are other things also can be considered and included but that is all i could put down off hand. When ever you are ready with your NAS solution i could be a good customer Haha as soon as I get someone interested in coughing up resources you can be the beta tester. Unless ofcourse you have some spare cash and the knowhow to make cabinets suitable for housing hard disks with proper cooling. I wonder how temperature could be measured in the cabinet over a period of time - anyone has any thoughts on the same ? *** The SCSI DEs are available if you want to go for branded one there are vault (Dell) or MSA30/500/1000 from HP . All
Re: [ilugd] NAS idea
--- Manish Verma [EMAIL PROTECTED] wrote: i would like to add few more points while designing a NAS solution 1.writing of data on to HDD should be fast enough, at time when you are mounting central storage on multiple server the concurrent NFS operation makes it very difficult and the write on the box slows down Any particular reason why NFS should be part of a NAS solution? Regards, --Naresh -- Naresh __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ ilugd mailinglist -- ilugd@lists.linux-delhi.org http://frodo.hserus.net/mailman/listinfo/ilugd Archives at: http://news.gmane.org/gmane.user-groups.linux.delhi http://www.mail-archive.com/ilugd@lists.linux-delhi.org/
Re: [ilugd] NAS idea
how would place the storage in to your network then? we should then use SAN or IP SAN or for that matter DAS would be a good idea. Regds Manish -Original Message- From: Naresh Narang [mailto:[EMAIL PROTECTED] Sent: Saturday, January 14, 2006 2:33 PM To: [EMAIL PROTECTED]; The Linux-Delhi mailing list Subject: Re: [ilugd] NAS idea --- Manish Verma [EMAIL PROTECTED] wrote: i would like to add few more points while designing a NAS solution 1.writing of data on to HDD should be fast enough, at time when you are mounting central storage on multiple server the concurrent NFS operation makes it very difficult and the write on the box slows down Any particular reason why NFS should be part of a NAS solution? Regards, --Naresh -- Naresh __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ ilugd mailinglist -- ilugd@lists.linux-delhi.org http://frodo.hserus.net/mailman/listinfo/ilugd Archives at: http://news.gmane.org/gmane.user-groups.linux.delhi http://www.mail-archive.com/ilugd@lists.linux-delhi.org/
Re: [ilugd] NAS idea
--- Mithun Bhattacharya [EMAIL PROTECTED] wrote: I have this brewing in my head for quite sometime and was wondering whether anyone was ready to try it out. I currently cant due to lack of resources but if it works you could probably earn some glory/money whatever. Basically I am looking at a Linux based NAS solution. Unless I am mistaken most NAS systems are costing 5-6 times the cost of the hardware involved. If my understanding is correct a simple NAS is: 1. A storage space available over the network 2. Fault tolerant 3. Expandable 4. Transparent 5. Compatible across OSes The way I see it is I need the following to get it going 1. Gigabit network 2. RAID 1/5 3. LVM 4. RAID and LVM can be configured to handle hot swap 5. Share disk space using NFS/SAMBA Of course it runs Linux or I wouldn't be talking here. There are two things currently lacking off the shelf. 1. A cabinet to hold/expand hard disks at least 4 hard disks to begin with 2. A web based application to manage the RAID and LVM setup. I am hoping I am making sense here and would like to solicit feedback as to the viability and interest in trying out the setup. The closest match with above characteristics is this product from adaptec - http://www.snapappliance.com/ Check it out. Of course you can build your own. Regards, --Naresh -- Naresh __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ ilugd mailinglist -- ilugd@lists.linux-delhi.org http://frodo.hserus.net/mailman/listinfo/ilugd Archives at: http://news.gmane.org/gmane.user-groups.linux.delhi http://www.mail-archive.com/ilugd@lists.linux-delhi.org/
Re: [ilugd] NAS idea
Hi, The main problem with NFS is that it doesn't have common way to add users manage disk quota. I have tried for hosting automation but it has not worked. And according to Psoft have to use NetApp ( http://www.netapp.com ) for this. Regards, Manoj -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Mithun Bhattacharya Sent: Friday, January 13, 2006 6:30 PM To: Linux Delhi Cc: Linux India Help Subject: [ilugd] NAS idea I have this brewing in my head for quite sometime and was wondering whether anyone was ready to try it out. I currently cant due to lack of resources but if it works you could probably earn some glory/money whatever. Basically I am looking at a Linux based NAS solution. Unless I am mistaken most NAS systems are costing 5-6 times the cost of the hardware involved. If my understanding is correct a simple NAS is: 1. A storage space available over the network 2. Fault tolerant 3. Expandable 4. Transparent 5. Compatible across OSes The way I see it is I need the following to get it going 1. Gigabit network 2. RAID 1/5 3. LVM 4. RAID and LVM can be configured to handle hot swap 5. Share disk space using NFS/SAMBA Of course it runs Linux or I wouldn't be talking here. There are two things currently lacking off the shelf. 1. A cabinet to hold/expand hard disks at least 4 hard disks to begin with 2. A web based application to manage the RAID and LVM setup. I am hoping I am making sense here and would like to solicit feedback as to the viability and interest in trying out the setup. Mithun __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ ilugd mailinglist -- ilugd@lists.linux-delhi.org http://frodo.hserus.net/mailman/listinfo/ilugd Archives at: http://news.gmane.org/gmane.user-groups.linux.delhi http://www.mail-archive.com/ilugd@lists.linux-delhi.org/ ___ ilugd mailinglist -- ilugd@lists.linux-delhi.org http://frodo.hserus.net/mailman/listinfo/ilugd Archives at: http://news.gmane.org/gmane.user-groups.linux.delhi http://www.mail-archive.com/ilugd@lists.linux-delhi.org/
Re: [ilugd] NAS idea
--- Manoj Kumar Mishra [EMAIL PROTECTED] wrote: Hi, The main problem with NFS is that it doesn't have common way to add users manage disk quota. I have tried for hosting automation but it has not worked. And according to Psoft have to use NetApp ( http://www.netapp.com ) for this. err maybe we should take some simple steps before we start targetting the business users ? Most SOHO users probably can survive without qouta's - assuming there is absolutely no way to maintain it across platforms. To think about it quotas need to be implemented at the Linux end - so would there be an issue if a web based interface is provided to manage quotas or am I looking at the problem differently ? Mithun __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ ilugd mailinglist -- ilugd@lists.linux-delhi.org http://frodo.hserus.net/mailman/listinfo/ilugd Archives at: http://news.gmane.org/gmane.user-groups.linux.delhi http://www.mail-archive.com/ilugd@lists.linux-delhi.org/
Re: [ilugd] NAS idea
You can use NFS3 as NFS2 has locking issues while multiple clients accessing the same mnts, other wise no issue as NAS And can be consider of http://www.openfiler.com/ Regards, Manoj -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Mithun Bhattacharya Sent: Saturday, January 14, 2006 12:20 AM To: The Linux-Delhi mailing list Cc: Linux India Help Subject: Re: [ilugd] NAS idea --- Manoj Kumar Mishra [EMAIL PROTECTED] wrote: Hi, The main problem with NFS is that it doesn't have common way to add users manage disk quota. I have tried for hosting automation but it has not worked. And according to Psoft have to use NetApp ( http://www.netapp.com ) for this. err maybe we should take some simple steps before we start targetting the business users ? Most SOHO users probably can survive without qouta's - assuming there is absolutely no way to maintain it across platforms. To think about it quotas need to be implemented at the Linux end - so would there be an issue if a web based interface is provided to manage quotas or am I looking at the problem differently ? Mithun __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ ilugd mailinglist -- ilugd@lists.linux-delhi.org http://frodo.hserus.net/mailman/listinfo/ilugd Archives at: http://news.gmane.org/gmane.user-groups.linux.delhi http://www.mail-archive.com/ilugd@lists.linux-delhi.org/ ___ ilugd mailinglist -- ilugd@lists.linux-delhi.org http://frodo.hserus.net/mailman/listinfo/ilugd Archives at: http://news.gmane.org/gmane.user-groups.linux.delhi http://www.mail-archive.com/ilugd@lists.linux-delhi.org/
Re: [ilugd] NAS idea
Mithun Bhattacharya [EMAIL PROTECTED] wrote: I have this brewing in my head for quite sometime and was wondering whether anyone was ready to try it out. I currently cant due to lack of resources but if it works you could probably earn some glory/money whatever. You can always build 'much' better product than most of those big offerings. But what most people, who do not have experience with business, ignore business systems. With software you can be flexible with so many models like open source model or net company model or whatever you are comfortable with. Even if you are not successful, you earn a lot of name from the software you make especially with open models. When it comes to 'hardware selling' you need to have skills for business systems and it require different type of skills. You can always make better burger 'but' can you beat 'Mc donalds business system'? If you can, its not just NAS devices, there are many many ideas everywhere and any of these can make you next richie. By the way, there is nothing like 'lack of resources' and there is nothing like you can not beat world top b-school bean counters in business systems -- if you have that desire. Regards ramana -- http://www.livejournal.com/users/evramana/ mail: ramana at intraperson dot com http://www.intraperson.com - Yahoo! Photos Got holiday prints? See all the ways to get quality prints in your hands ASAP. ___ ilugd mailinglist -- ilugd@lists.linux-delhi.org http://frodo.hserus.net/mailman/listinfo/ilugd Archives at: http://news.gmane.org/gmane.user-groups.linux.delhi http://www.mail-archive.com/ilugd@lists.linux-delhi.org/
Re: [ilugd] NAS idea
--- ramana [EMAIL PROTECTED] wrote: By the way, there is nothing like 'lack of resources' and there is nothing like you can not beat world top b-school bean counters in business systems -- if you have that desire. I am hoping you have been encouraging in in your email :). As for targetting business users - I am not against it but I would prefer to first see what hurdles I encounter in the proof of concept. I agree running a business is a different ball game but then I can also see that there is ample scope for starting one considering the competition in India is almost non existant because of the price factor. Mithun __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ ilugd mailinglist -- ilugd@lists.linux-delhi.org http://frodo.hserus.net/mailman/listinfo/ilugd Archives at: http://news.gmane.org/gmane.user-groups.linux.delhi http://www.mail-archive.com/ilugd@lists.linux-delhi.org/
Re: [ilugd] NAS idea
Mithun Bhattacharya [EMAIL PROTECTED] wrote: --- ramana wrote: By the way, there is nothing like 'lack of resources' and there is nothing like you can not beat world top b-school bean counters in business systems -- if you have that desire. I am hoping you have been encouraging in in your email :). As for targetting business users - I am not against it but I would prefer to first see what hurdles I encounter in the proof of concept. I agree running a business is a different ball game but then I can also see that there is ample scope for starting one considering the competition in India is almost non existant because of the price factor. Yep. I am encouraging you and I speak positively. Your last point is the best one. India is still not touched in many areas and I see many times whenever I step outside. People who has the courage can take advantage of this. Regards ramana -- http://www.livejournal.com/users/evramana/ mail: ramana at intraperson dot com http://www.intraperson.com - Yahoo! Photos Ring in the New Year with Photo Calendars. Add photos, events, holidays, whatever. ___ ilugd mailinglist -- ilugd@lists.linux-delhi.org http://frodo.hserus.net/mailman/listinfo/ilugd Archives at: http://news.gmane.org/gmane.user-groups.linux.delhi http://www.mail-archive.com/ilugd@lists.linux-delhi.org/
Re: [ilugd] NAS idea
--- Manish Verma [EMAIL PROTECTED] wrote: i would like to add few more points while designing a NAS solution 1.writing of data on to HDD should be fast enough, at time when you are mounting central storage on multiple server the concurrent NFS operation makes it very difficult and the write on the box slows down e.g netapp uses wafl filesystem (or for that matter all NAS storage uses the same way of writing through cache) which does the writing of data from cache. Your solution should be based on using the cache to the maximum for both read and write. I have seen xfs,ext3 going down in the load condition. I guess corporate customers would like SCSI disks, even home users shouldnt go for anything less than SATA disks. I do recall NFS having lad issues that definitely needs to be looked into. Guess we will have to develop a proper set of test cases based on your feedback :). 2. You will have to check the boot time of your NAS box and it should be under control there are NAS boxes ( i would not put the name here, but you can easily find it out) which takes 15-20 minutes to boot. The box should be able to boot in 3-4 minutes even if it a abnormal reboot. I have used pretty simple installations of LVM over RAID5 on Fedora Core 3 and it did come up in a few min. I do agree complex LVM and RAID installations need to be tested. 3.you should benchmark your NFS box for NFS operation not I/O operation on disk. You may get good I/O on disk but may not be able to get good NFS operation. 4.The redundancy of storage array should not be limited to RAID5 because now the storage capacity of individual disks are going high (500GB SATA, 300GB SCSI and FCAL are in the market) you should be able to handle dual disk failure in the same RAID group lets say you have 14 disk RAID group and your one disk failed and rebuilding is going on (which will take some time as the capacity is more) and during that time if your second disk also fails then your whole of the RAID volume will go offline, you should have dual parity kind of solution. Well isnt the concept of spare disks in RAID meant for these purposes ? 5.NAS solution should be modular i.e i should be able to add storage and processing both e.g if i am handling n NFS operation using one NAS header today with NTB storage and tomorrow if i want to increase my NFS operation i should be able to add more processing (NAS header to the same storage) and lets say i don't want to increase the NFS operation but i want to add more storage i should be able to do that as well and that too on the fly because if my box is in production i cant shut it down. Adding disks would be limited by the enclosure holding them. A highly modular system I am afraid wont be there in a first cut we would probably have to set up a research lab or something where CPU and hard disks both are modular and are being used efficiently. Also if I add a CPU does it contribute to a existing box or creates a new box. There are various degrees of modularity which could be achieved but definitely not everything in the first version. May be there are other things also can be considered and included but that is all i could put down off hand. When ever you are ready with your NAS solution i could be a good customer Haha as soon as I get someone interested in coughing up resources you can be the beta tester. Unless ofcourse you have some spare cash and the knowhow to make cabinets suitable for housing hard disks with proper cooling. I wonder how temperature could be measured in the cabinet over a period of time - anyone has any thoughts on the same ? for you :). also read on onstor while designing the solution. Mithun __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ ilugd mailinglist -- ilugd@lists.linux-delhi.org http://frodo.hserus.net/mailman/listinfo/ilugd Archives at: http://news.gmane.org/gmane.user-groups.linux.delhi http://www.mail-archive.com/ilugd@lists.linux-delhi.org/
