hi Masters: i want to develop a log structure filesystem based on hdfs。this filesystem used to host virtualizaion machine image file 。 on hdfs i can implement snapshot and data redundancy;as log structure fs ,which support random access。 i also hope to use map reduce way to do segment cleaning partly。 i am not sure if it is reasonable 。i really hope to build a env that support online app and offline app at same time。i am trying do it on hdfs。can you give me some advice? thanks kanghua 发自我的 iPhone
在 2011-9-15,14:54,Norman Maurer <norman.mau...@googlemail.com> 写道: > You should keep in mind that HDFS is not POSIX conform so you will > have a hard time to use it as "real fs". I know there is a fuse driver > for it but I would not use it for heavy usage. Also HDFS is not really > a good fit for random access at all. > > If you really need a POSIX fs I would recomment you to have a look at > DRBD or glusterfs.. > > Bye, > Norman > > > 2011/9/15 Per Steffensen <st...@designware.dk>: >> David Rosenstrauch skrev: >>> >>> On 09/14/2011 02:02 PM, Per Steffensen wrote: >>>> >>>> Hi >>>> >>>> If my goal is to have multiple physical disks seem as one big disk with >>>> redundancy built in, why would I use a HDFS cluster among machines with >>>> one disk each, instead of using software RAID like md(adm) directly on >>>> top of the disks? I am looking for pros and cons on the two solutions. >>>> http://en.wikipedia.org/wiki/RAID#Software-based_RAID >>>> http://en.wikipedia.org/wiki/Mdadm >>>> >>>> Regards, Per Steffensen >>> >>> HDFS was never intended to be a general-purpose file system. It is a >>> system optimized for a) running map/reduce, and b) holding large files. It >>> should not be considered as a replacement for RAID. >>> >>> DR >> >> Thanks for you reply, David. Despite that HDFS wasnt intended to be used for >> this, I guess it could be. So if we forget for a moment that it was not >> designed/optimized to be used as a general purpose file system (GPFS), what >> are the pros and cons for using it as a GPFS with built in redundancy vs >> using software RAID. Is HDFS too slow for some kind of file operations, or >> what will the problems (and benefits) be? Hope for some input - I need >> arguments for and against to be used in a discussion with a customer. >> Thanks! >>> >>> >> >> >