Thankyou. Switching to ext4 On Tue, Mar 29, 2011 at 8:23 AM, Eric <eric.x...@gmail.com> wrote:
> Rita, another issue I've seen is that when you have lots of XFS filesystems > that are heavily used, the Linux kernel will at some point crash. So the XFS > driver seems to have problems that only appear with large volumes of data. I > will switch to EXT4 soon because of this. > > > 2011/3/29 Rita <rmorgan...@gmail.com> > >> Thanks with ext4 i created 2 16TB volumes and they are seen. I think it >> maybe a issue with XFS. >> >> >> >> On Mon, Mar 28, 2011 at 3:50 PM, Todd Lipcon <t...@cloudera.com> wrote: >> >>> On Fri, Mar 25, 2011 at 9:06 PM, Rita <rmorgan...@gmail.com> wrote: >>> >>>> Using 0.21 >>>> >>>> When I have a filesystem (XFS) with 1TB it detects the >>>> datanode detects it immediately. When I create 3 identical file systems all >>>> 3TB are visible immediately. >>>> >>>> If I create a 6TB filesystem (XFS) and I add it to dfs.data.dir and I >>>> restart the datanode, "hdfs dfsadmin -report" does not see the new 6TB >>>> filesystem. >>>> >>>> In all of these occasions the datanode does create a 'finalized' file >>>> structure in respective directories? >>>> >>>> >>>> My questions are: >>>> Is there a limitation in the size of dfs.data.dir? What is the largest >>>> filesystem that can be part of it? >>>> >>> >>> I've heard that there is a 4T limit, but I've never tried to replicate. >>> Given that single disks aren't this large, it indicates you might be running >>> RAID or SPAN rather than recommended JBOD. >>> >>> >>>> Could this be a block scanner issue? Is it possible to make my block >>>> scanning more aggressive? >>>> >>> >>> Unrelated to block scanning most likely. >>> >>> -Todd >>> -- >>> Todd Lipcon >>> Software Engineer, Cloudera >>> >> >> >> >> -- >> --- Get your facts first, then you can distort them as you please.-- >> > > -- --- Get your facts first, then you can distort them as you please.--
