[Lustre-discuss] No space left on device for just one file

2010-01-08 Thread Michael Robbert
I have a user that reported a problem creating a file on our Lustre filesystem. When I investigated I found that the problem appears to be unique to just one filename in one directory. I have tried numerous ways of creating the file including echo, touch, and "lfs setstripe" all return "No space

Re: [Lustre-discuss] No space left on device for just one file

2010-01-11 Thread Mag Gam
Can you paste us the file name? I want to see if we can touch something like this. On Fri, Jan 8, 2010 at 1:36 PM, Michael Robbert wrote: > I have a user that reported a problem creating a file on our Lustre > filesystem. When I investigated I found that the problem appears to be unique > to ju

Re: [Lustre-discuss] No space left on device for just one file

2010-01-11 Thread Michael Robbert
The filename is not very unique. I can create a file with the same name in another directory or on another Lustre filesystem. It is just this exact path on this filesystem. The full path is: /lustre/scratch/smoqbel/Cenval/CLM/Met.Forcing/18X11/NLDAS.APCP.007100.pfb.00164 The mount point for this

Re: [Lustre-discuss] No space left on device for just one file

2010-01-11 Thread Bernd Schubert
Hello Robert, could you please send a mail into our ticket system? Kit or I would then start to investigate tomorrow. Thanks, Bernd On Monday 11 January 2010, Michael Robbert wrote: > The filename is not very unique. I can create a file with the same name in > another directory or on another L

Re: [Lustre-discuss] No space left on device for just one file

2010-01-11 Thread Andreas Dilger
On 2010-01-11, at 15:59, Michael Robbert wrote: > The filename is not very unique. I can create a file with the same > name in another directory or on another Lustre filesystem. It is > just this exact path on this filesystem. The full path is: > /lustre/scratch/smoqbel/Cenval/CLM/Met.Forcing/1

Re: [Lustre-discuss] No space left on device for just one file

2010-01-12 Thread Michael Robbert
Andreas, Here are the results of my debugging. This problem does show up on multiple (presumably all) clients. I followed your instructions, changing lustre to lnet in step 2, and got debug output on both machines, but the -28 text only showed up on the client. [r...@ra 18X11]# grep -- "-28" /t

Re: [Lustre-discuss] No space left on device for just one file

2010-01-12 Thread Bernd Schubert
Hello Mike, you really should fill a ticket to us (DDN). I think your problem is from these MDS messages: LDISKFS-fs warning (device dm-1): ldiskfs_dx_add_entry: Directory index full! LDISKFS-fs warning (device dm-1): ldiskfs_dx_add_entry: Directory index full! And /dev/dm-1 is also the scrat

Re: [Lustre-discuss] No space left on device for just one file

2010-01-12 Thread Bernd Schubert
Hmm, seems there is no solution available except of to turn off dir-indexing. I never looked into the dir-index code so far, but wouldn't it make sense to skip the index just for this directory? https://bugzilla.lustre.org/show_bug.cgi?id=10129 http://kerneltrap.org/mailarchive/linux-kernel/200

Re: [Lustre-discuss] No space left on device for just one file

2010-01-12 Thread Andreas Dilger
On 2010-01-12, at 15:30, Bernd Schubert wrote: > you really should fill a ticket to us (DDN). I think your problem is > from > these MDS messages: > > LDISKFS-fs warning (device dm-1): ldiskfs_dx_add_entry: Directory > index full! > LDISKFS-fs warning (device dm-1): ldiskfs_dx_add_entry: Direct

Re: [Lustre-discuss] No space left on device for just one file

2010-01-13 Thread Michael Robbert
Andreas, I never saw this message either. It is showing up in the output of dmesg, but is not written to any log files that I can find. I miscounted the number of files in my original message. The actual number is a little more than 11 million files. I am in the process of working with the user