[Lustre-discuss] LBUG with 1.6.5~rc2 and lfs quotacheck

2008-05-13 Thread Patrick Winnertz 
Hey 

I've some troubles with quota on 1.6.5. (Yes I know this rc software :)).
Since I don't find any bugreport about this issue (and this is not yet 
released) I'm unsure what do do. Here the problem description:

After reformatting the fs and mounting it. I've tried to execute this 
command:
debian:/mnt# lfs quotacheck -ug /mnt/lustre_client/

Right after hitting enter I got this: 
Message from [EMAIL PROTECTED] at Tue May 13 13:05:08 2008 ...
debian kernel: LustreError: 2921:0:
(fsfilt-ldiskfs.c:2066:fsfilt_ldiskfs_quotainfo()) LBUG

Furthermore a tmp. file was created. I've attached it to this mail. 
Anybody know what's going wrong here? 

Greetings
Winnie

-- 
Patrick Winnertz
Tel.: +49 (0) 2161 / 4643 - 0

credativ GmbH, HRB Mönchengladbach 12080
Hohenzollernstr. 133, 41061 Mönchengladbach
Geschäftsführung: Dr. Michael Meskes, Jörg Folz


lustre-log.1210676708.2921
Description: Binary data
___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

Re: [Lustre-discuss] LBUG with 1.6.5~rc2 and lfs quotacheck

2008-05-13 Thread Andrew Perepechko 
Hello

Patrick, have you got the kernel dump (dmesg output) from 
client node by chance?

Thanks
Andrew.

On Tuesday 13 May 2008 13:18:06 Patrick Winnertz wrote:
 Hey 
 
 I've some troubles with quota on 1.6.5. (Yes I know this rc software :)).
 Since I don't find any bugreport about this issue (and this is not yet 
 released) I'm unsure what do do. Here the problem description:
 
 After reformatting the fs and mounting it. I've tried to execute this 
 command:
 debian:/mnt# lfs quotacheck -ug /mnt/lustre_client/
 
 Right after hitting enter I got this: 
 Message from [EMAIL PROTECTED] at Tue May 13 13:05:08 2008 ...
 debian kernel: LustreError: 2921:0:
 (fsfilt-ldiskfs.c:2066:fsfilt_ldiskfs_quotainfo()) LBUG
 
 Furthermore a tmp. file was created. I've attached it to this mail. 
 Anybody know what's going wrong here? 
 
 Greetings
 Winnie
 

___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

Re: [Lustre-discuss] LBUG with 1.6.5~rc2 and lfs quotacheck

2008-05-13 Thread Andrew Perepechko 
Patrick,

it seems that you are running different versions of mds/lquota modules
on one node and ost/fsfilt_ldiskfs modules on another. Is this true?

Andrew.

On Tuesday 13 May 2008 13:18:06 Patrick Winnertz wrote:
 Hey 
 
 I've some troubles with quota on 1.6.5. (Yes I know this rc software :)).
 Since I don't find any bugreport about this issue (and this is not yet 
 released) I'm unsure what do do. Here the problem description:
 
 After reformatting the fs and mounting it. I've tried to execute this 
 command:
 debian:/mnt# lfs quotacheck -ug /mnt/lustre_client/
 
 Right after hitting enter I got this: 
 Message from [EMAIL PROTECTED] at Tue May 13 13:05:08 2008 ...
 debian kernel: LustreError: 2921:0:
 (fsfilt-ldiskfs.c:2066:fsfilt_ldiskfs_quotainfo()) LBUG
 
 Furthermore a tmp. file was created. I've attached it to this mail. 
 Anybody know what's going wrong here? 
 
 Greetings
 Winnie
 

___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

Re: [Lustre-discuss] LBUG with 1.6.5~rc2 and lfs quotacheck

2008-05-13 Thread Andrew Perepechko 
Patrick, could you try a rebuild of the source, please?

According to the binaries (to assembly) you use, it seems,
lquota and fsfilt_ldiskfs modules do not match each other, 
but generally they have to.

Thank you very much!
Andrew.

On Tuesday 13 May 2008 15:53:05 Patrick Winnertz wrote:
 
 On Tuesday 13 May 2008 13:09:56 you wrote:
  Patrick,
 
  it seems that you are running different versions of mds/lquota modules
  on one node and ost/fsfilt_ldiskfs modules on another. Is this true?
 Mh.. I don't think so. I've attached a modinfo of the modules to the 
 bugreport.  (mgs and ost have exactly the same packages installed).
 
 Greetings
 Winnie
 

___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

[Lustre-discuss] supporting fcntl, byte-range file locking

2008-05-13 Thread Wei-keng Liao 

What versions of Lustre support fcntl for byte-range file locking? 
Attached is a test program extracted from ROMIO to test fcntl() for 
locking. It ran fine on some Lustre versions.

Wei-keng#include stdio.h
#include stdlib.h

#include unistd.h
#include fcntl.h
#include errno.h

#include mpi.h

static int rank;

void ADIOI_Set_lock(int fd, int cmd, int type, long long offset, int whence, 
long long len)
{
int err;
struct flock lock;
char *cmd_str, *type_str;

if (len == 0) return;

lock.l_type   = type;
lock.l_whence = whence;
lock.l_start  = offset;
lock.l_len= len;

cmd_str = (cmd == F_GETLK )? F_GETLK :
 ((cmd == F_SETLK )? F_SETLK :
 ((cmd == F_SETLKW)? F_SETLKW : UNEXPECTED));
type_str = (type == F_RDLCK)? F_RDLCK :
  ((type == F_WRLCK)? F_WRLCK :
  ((type == F_UNLCK)? F_UNLOCK : UNEXPECTED));
errno = 0;
do {
err = fcntl(fd, cmd, lock);
} while (err  (errno == EINTR));

if (err  (errno != EBADF))
printf(%d: File locking failed in (fd %X,cmd %s/%X,type %s/%X,whence 
%X) with return value %X and errno %X.\n,
   rank, fd, cmd_str, cmd, type_str, type, whence, err, errno);
else
printf(%d: File locking succeed in (fd %X,cmd %s/%X,type %s/%X,whence 
%X)\n,
   rank, fd, cmd_str, cmd, type_str, type, whence);

return;
}

int main(int argc, char **argv)
{
int fd, buf[100];

MPI_Init(argc, argv);
MPI_Comm_rank(MPI_COMM_WORLD, rank);

if (argc != 2) {
printf(Usage: %s filename\n, argv[0]);
return 1;
}

if ((fd = open(argv[1], O_CREAT|O_RDWR, 0600)) == -1) {
printf(Error: opening file %s\n, argv[1]);
return 1;
}

MPI_Barrier(MPI_COMM_WORLD);
ADIOI_Set_lock(fd, F_SETLKW, F_WRLCK, 0, SEEK_SET, 10);
write(fd, buf, 100);
ADIOI_Set_lock(fd, F_SETLK,  F_UNLCK, 0, SEEK_SET, 10);

close(fd);

MPI_Finalize();
return 0;
}

___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

[Lustre-discuss] Size of MDT, used space

2008-05-13 Thread Thomas Roth 
Hi all,

I'm still in trouble with numbers: the available, used and necessary 
space on my MDT:
According to lfs df, I have now filled my file system with 115.3 TB.
All of these files are sized 5 MB. That should be roughly 24 million files.
For the MDT, lfs df reports 28.2 GB used.

Now I believed that creating a file on Lustre means using one inode on 
the MDT. Since all of my Lustre partitions were formatted with the 
default options (all of this is running Lustre v. 1.6.4.3, btw), an 
inode should eat up 4kB on the MDT partition. Of course, 24 million 
files times 4 kB gives you 91 GB rather than 28GB.
Obviously, there is something I missed completely. Perhaps somebody 
could illuminate me here?

This issue could also be phrased as How large should my MDT be to 
accommodate n TB storage space? The manual's answer boils down to = 
number of files * 4 kB   (*2 per recommendation). That's how I 
calculated above - maybe my test system is broken? I can't check on the 
content of these files, it's just 5MB test files created with the 
'stress' utility.

  Thanks and regards,
Thomas

___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

Re: [Lustre-discuss] Size of MDT, used space

2008-05-13 Thread Cliff White 
Thomas Roth wrote:
 Hi all,
 
 I'm still in trouble with numbers: the available, used and necessary 
 space on my MDT:
 According to lfs df, I have now filled my file system with 115.3 TB.
 All of these files are sized 5 MB. That should be roughly 24 million files.
 For the MDT, lfs df reports 28.2 GB used.
 
 Now I believed that creating a file on Lustre means using one inode on 
 the MDT. Since all of my Lustre partitions were formatted with the 
 default options (all of this is running Lustre v. 1.6.4.3, btw), an 
 inode should eat up 4kB on the MDT partition. Of course, 24 million 
 files times 4 kB gives you 91 GB rather than 28GB.
 Obviously, there is something I missed completely. Perhaps somebody 
 could illuminate me here?
 
 This issue could also be phrased as How large should my MDT be to 
 accommodate n TB storage space? The manual's answer boils down to = 
 number of files * 4 kB   (*2 per recommendation). That's how I 
 calculated above - maybe my test system is broken? I can't check on the 
 content of these files, it's just 5MB test files created with the 
 'stress' utility.
 
   Thanks and regards,
 Thomas

The size of the MDS inode depends on the number of stripes. 4.5k is the 
maximum, 512k the minimum. Actually size varies with number of stripes 
in the file. So, we advise using 4k as an estimate, as that will cover 
the vast majority of cases, but actual use in almost all situations will 
be smaller than 4k.
cliffw

 
 ___
 Lustre-discuss mailing list
 Lustre-discuss@lists.lustre.org
 http://lists.lustre.org/mailman/listinfo/lustre-discuss

___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

Re: [Lustre-discuss] supporting fcntl, byte-range file locking

2008-05-13 Thread Oleg Drokin 
Hello!

On May 13, 2008, at 12:37 PM, Wei-keng Liao wrote:
 What versions of Lustre support fcntl for byte-range file locking?
 Attached is a test program extracted from ROMIO to test fcntl() for
 locking. It ran fine on some Lustre versions.

Certainly many versions, going far back to 1.2 and prior.
This functionality is disabled by default though.
To enable cluster-consistent fcntl locking you need to mount your lustre
fs on a client with -o flock option.
If you do not need cluster-consistency, you can mount with -o  
localflock,
it would only be consistent on local node of course, but much faster
(only useful for some stupid applications that just test the presence
without actually using the functionality, of course, or when no
cross-node flock interactions are expected otherwise).
This disabled by default case got into effect in 1.4.8, prior to that
if you do not use -o flock option, then only locally-consistent locking
is provided.

Bye,
 Oleg
___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

Re: [Lustre-discuss] supporting fcntl, byte-range file locking

2008-05-13 Thread Tom.Wang 
Wei-keng Liao wrote:
 What versions of Lustre support fcntl for byte-range file locking? 
 Attached is a test program extracted from ROMIO to test fcntl() for 
 locking. It ran fine on some Lustre versions.

 Wei-keng
   
   
 Lustre has supported flock since 1.4.3.

 But you need mount with -o flock or localflock(simulate local flock on 
client, since 1.4.10) to enable the flock on client.
 


Thanks
WangDi
 

 #include stdio.h
 #include stdlib.h

 #include unistd.h
 #include fcntl.h
 #include errno.h

 #include mpi.h

 static int rank;

 void ADIOI_Set_lock(int fd, int cmd, int type, long long offset, int whence, 
 long long len)
 {
 int err;
 struct flock lock;
 char *cmd_str, *type_str;

 if (len == 0) return;

 lock.l_type   = type;
 lock.l_whence = whence;
 lock.l_start  = offset;
 lock.l_len= len;

 cmd_str = (cmd == F_GETLK )? F_GETLK :
  ((cmd == F_SETLK )? F_SETLK :
  ((cmd == F_SETLKW)? F_SETLKW : UNEXPECTED));
 type_str = (type == F_RDLCK)? F_RDLCK :
   ((type == F_WRLCK)? F_WRLCK :
   ((type == F_UNLCK)? F_UNLOCK : UNEXPECTED));
 errno = 0;
 do {
 err = fcntl(fd, cmd, lock);
 } while (err  (errno == EINTR));

 if (err  (errno != EBADF))
 printf(%d: File locking failed in (fd %X,cmd %s/%X,type %s/%X,whence 
 %X) with return value %X and errno %X.\n,
rank, fd, cmd_str, cmd, type_str, type, whence, err, errno);
 else
 printf(%d: File locking succeed in (fd %X,cmd %s/%X,type 
 %s/%X,whence %X)\n,
rank, fd, cmd_str, cmd, type_str, type, whence);

 return;
 }

 int main(int argc, char **argv)
 {
 int fd, buf[100];

 MPI_Init(argc, argv);
 MPI_Comm_rank(MPI_COMM_WORLD, rank);

 if (argc != 2) {
 printf(Usage: %s filename\n, argv[0]);
 return 1;
 }

 if ((fd = open(argv[1], O_CREAT|O_RDWR, 0600)) == -1) {
 printf(Error: opening file %s\n, argv[1]);
 return 1;
 }

 MPI_Barrier(MPI_COMM_WORLD);
 ADIOI_Set_lock(fd, F_SETLKW, F_WRLCK, 0, SEEK_SET, 10);
 write(fd, buf, 100);
 ADIOI_Set_lock(fd, F_SETLK,  F_UNLCK, 0, SEEK_SET, 10);

 close(fd);

 MPI_Finalize();
 return 0;
 }

   
 

 ___
 Lustre-discuss mailing list
 Lustre-discuss@lists.lustre.org
 http://lists.lustre.org/mailman/listinfo/lustre-discuss
   


-- 
Regards,
Tom Wangdi
--
Sun Lustre Group
System Software Engineer 
http://www.sun.com

___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss