Hi Jason,
There's nothing that fuse does that should cause this I don't think. If this is
a cat operation, fuse will do this in chunks of 32 K which you can change by
mounting with -ordbuffer_size=#bytes.
Do you have the stack trace or the NN log while this app is running? Were you
doing the equivalent of this app with dfs -cat or something before with no
problems? 1.5 M files sounds like it might be close to the limit of a 1.5GB NN?
-- pete
On 11/17/08 9:29 AM, Jason Venner [EMAIL PROTECTED] wrote:
We recently setup a fuse mount using the 18.2 fuse code, against our
18.1 hdfs, which has been running stably for some time.
We have about 20 datanodes and 50TB or so in our hdfs. The namenode is
running an i686 kernel and has been running with -Xmx1500m.
We have 1,492,093 files in our hdfs at present.
Since we put up the fuse mount which reads 2 large files at a time,
about 6 a second or so, we have had the namenode OOM multiple times.
We have upped the memory foot print to the max allowed for a 32bit jvm.
What memory size should we be running?
Is the fuse mount doing something that is increasing the memory use of
the namenode?
Happy week all.
