Re: [lustre-discuss] LFS Quota
Dear All, Can anyone look into it. Thanks, ANS On Mon, Jan 7, 2019 at 6:38 PM ANS wrote: > Dear All, > > I am trying to set quota on lustre but unfortunately i have issued the > below commands:- > > tunefs.lustre --param ost.quota_type=ug /dev/mapper/mds1 > checking for existing Lustre data: found > Reading CONFIGS/mountdata > >Read previous values: > Target: data-MDT > Index: 0 > Lustre FS: data > Mount type: ldiskfs > Flags: 0x1001 > (MDT no_primnode ) > Persistent mount opts: user_xattr,errors=remount-ro > Parameters: mgsnode=192.168.2.9@o2ib:192.168.2.10@o2ib > failover.node=192.168.2.9@o2ib:192.168.2.10@o2ib > >Permanent disk data: > Target: data-MDT > Index: 0 > Lustre FS: data > Mount type: ldiskfs > Flags: 0x1041 > (MDT update no_primnode ) > Persistent mount opts: user_xattr,errors=remount-ro > Parameters: mgsnode=192.168.2.9@o2ib:192.168.2.10@o2ib > failover.node=192.168.2.9@o2ib:192.168.2.10@o2ib ost.quota_type=ug > > After this i have issued lctl conf_param home.quota.mdt=ugp and now i am > able to see quota enabled in mdt but will there be any affect to the above > version as i have issue that command to 2 mdts where i am having 4 mdts in > total. Do i need to issue any further commands to revert those. > > After enabling quota and file space crossed soft limit but i am unable to > get warning till it reached near hard limit. > > Can anyone help me out. > > Thanks, > ANS > -- Thanks, ANS. ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
[lustre-discuss] Disable identity_upcall and ACL
Hello all, When disabling identity_upcall on a MDT, you get this message in system logs: lustre-MDT: disable "identity_upcall" with ACL enabled maybe cause unexpected "EACCESS" I’m trying to understand what could be a scenario that shows this problem? What is the implication, or rather, how identity_upcall works? Thanks for your help Aurélien ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] Disable identity_upcall and ACL
Hi Aurélien! Am 09.01.19 um 11:48 schrieb Degremont, Aurelien: > When disabling identity_upcall on a MDT, you get this message in system > logs: > > lustre-MDT: disable "identity_upcall" with ACL enabled maybe cause > unexpected "EACCESS" > > I’m trying to understand what could be a scenario that shows this problem? > What is the implication, or rather, how identity_upcall works? Without an identity_upcall, all Lustre users effectively lose their secondary group memberships. These are not passed in the RPCs, but evaluated on the MDS instead. The default l_getidentity receives a numeric uid, queries NSS to obtain the corresponding account's list of gids, and passes the list back to the kernel. As a test scenario, just try to access a file or directory from an account that only has access permissions via one of its secondardy groups. (The log message is a bit misleading--you don't actually need to use ACLs, ordinary group permissions are sufficient.) Kind regards, Daniel -- Daniel Kobras Principal Architect Puzzle ITC Deutschland +49 7071 14316 0 www.puzzle-itc.de -- Puzzle ITC Deutschland GmbH Sitz der Gesellschaft: Jurastr. 27/1, 72072 Tübingen Eingetragen am Amtsgericht Stuttgart HRB 765802 Geschäftsführer: Lukas Kallies, Daniel Kobras, Mark Pröhl ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] LFS Quota
Hi ANS, About the soft limits and not receiving any warning or notification when the soft quota is reached, this would be the expected behavior. The soft quota is used together with the grace period to give some “extra” time to the user to remove inodes/blocks, as per the Lustre Operations Manual: Soft limit -- The grace timer is started once the soft limit is exceeded. At this point, the user/group/project can still allocate block/inode. When the grace time expires and if the user is still above the soft limit, the soft limit becomes a hard limit and the user/group/project can't allocate any new block/inode any more. The user/group/project should then delete files to be under the soft limit. The soft limit MUST be smaller than the hard limit. If the soft limit is not needed, it should be set to zero (0). I’m not aware of any warnings triggered by Lustre when the soft quota is reached, though that would be interesting to have. I know some people using external tools to monitor Lustre quotas and trigger warnings or similar for users exceeding their soft quota. Regards, Diego From: lustre-discuss on behalf of ANS Date: Wednesday, 9 January 2019 at 10:00 To: "lustre-discuss@lists.lustre.org" Subject: Re: [lustre-discuss] LFS Quota Dear All, Can anyone look into it. Thanks, ANS On Mon, Jan 7, 2019 at 6:38 PM ANS mailto:ans3...@gmail.com>> wrote: Dear All, I am trying to set quota on lustre but unfortunately i have issued the below commands:- tunefs.lustre --param ost.quota_type=ug /dev/mapper/mds1 checking for existing Lustre data: found Reading CONFIGS/mountdata Read previous values: Target: data-MDT Index: 0 Lustre FS: data Mount type: ldiskfs Flags: 0x1001 (MDT no_primnode ) Persistent mount opts: user_xattr,errors=remount-ro Parameters: mgsnode=192.168.2.9@o2ib:192.168.2.10@o2ib failover.node=192.168.2.9@o2ib:192.168.2.10@o2ib Permanent disk data: Target: data-MDT Index: 0 Lustre FS: data Mount type: ldiskfs Flags: 0x1041 (MDT update no_primnode ) Persistent mount opts: user_xattr,errors=remount-ro Parameters: mgsnode=192.168.2.9@o2ib:192.168.2.10@o2ib failover.node=192.168.2.9@o2ib:192.168.2.10@o2ib ost.quota_type=ug After this i have issued lctl conf_param home.quota.mdt=ugp and now i am able to see quota enabled in mdt but will there be any affect to the above version as i have issue that command to 2 mdts where i am having 4 mdts in total. Do i need to issue any further commands to revert those. After enabling quota and file space crossed soft limit but i am unable to get warning till it reached near hard limit. Can anyone help me out. Thanks, ANS -- Thanks, ANS. ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] Disable identity_upcall and ACL
Hi Daniel! The secondary group thing was ok to me. I got this idea even if there is some weird results during my tests. Looks like you can overwrite MDT checks if user and group is properly defined on client node. Cache effects? ACL is really the thing I was interested in. Who is validating the ACLs? MDT, client or both? Do you think ACL could be properly applied if user/groups are only defined on client side and identity_upcall is disabled on MDT side? Thanks Aurélien Le 09/01/2019 12:22, « lustre-discuss au nom de Daniel Kobras » a écrit : Hi Aurélien! Am 09.01.19 um 11:48 schrieb Degremont, Aurelien: > When disabling identity_upcall on a MDT, you get this message in system > logs: > > lustre-MDT: disable "identity_upcall" with ACL enabled maybe cause > unexpected "EACCESS" > > I’m trying to understand what could be a scenario that shows this problem? > What is the implication, or rather, how identity_upcall works? Without an identity_upcall, all Lustre users effectively lose their secondary group memberships. These are not passed in the RPCs, but evaluated on the MDS instead. The default l_getidentity receives a numeric uid, queries NSS to obtain the corresponding account's list of gids, and passes the list back to the kernel. As a test scenario, just try to access a file or directory from an account that only has access permissions via one of its secondardy groups. (The log message is a bit misleading--you don't actually need to use ACLs, ordinary group permissions are sufficient.) Kind regards, Daniel -- Daniel Kobras Principal Architect Puzzle ITC Deutschland +49 7071 14316 0 www.puzzle-itc.de -- Puzzle ITC Deutschland GmbH Sitz der Gesellschaft: Jurastr. 27/1, 72072 Tübingen Eingetragen am Amtsgericht Stuttgart HRB 765802 Geschäftsführer: Lukas Kallies, Daniel Kobras, Mark Pröhl ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] Disable identity_upcall and ACL
After most tests, when identity_upcall is set to NONE, it looks like only ACL using secondary groups are not satisfied. If secondary groups are properly defined on clients, accesses are granted as expected. Aurélien Le 09/01/2019 14:31, « lustre-discuss au nom de Degremont, Aurelien » a écrit : Hi Daniel! The secondary group thing was ok to me. I got this idea even if there is some weird results during my tests. Looks like you can overwrite MDT checks if user and group is properly defined on client node. Cache effects? ACL is really the thing I was interested in. Who is validating the ACLs? MDT, client or both? Do you think ACL could be properly applied if user/groups are only defined on client side and identity_upcall is disabled on MDT side? Thanks Aurélien Le 09/01/2019 12:22, « lustre-discuss au nom de Daniel Kobras » a écrit : Hi Aurélien! Am 09.01.19 um 11:48 schrieb Degremont, Aurelien: > When disabling identity_upcall on a MDT, you get this message in system > logs: > > lustre-MDT: disable "identity_upcall" with ACL enabled maybe cause > unexpected "EACCESS" > > I’m trying to understand what could be a scenario that shows this problem? > What is the implication, or rather, how identity_upcall works? Without an identity_upcall, all Lustre users effectively lose their secondary group memberships. These are not passed in the RPCs, but evaluated on the MDS instead. The default l_getidentity receives a numeric uid, queries NSS to obtain the corresponding account's list of gids, and passes the list back to the kernel. As a test scenario, just try to access a file or directory from an account that only has access permissions via one of its secondardy groups. (The log message is a bit misleading--you don't actually need to use ACLs, ordinary group permissions are sufficient.) Kind regards, Daniel -- Daniel Kobras Principal Architect Puzzle ITC Deutschland +49 7071 14316 0 www.puzzle-itc.de -- Puzzle ITC Deutschland GmbH Sitz der Gesellschaft: Jurastr. 27/1, 72072 Tübingen Eingetragen am Amtsgericht Stuttgart HRB 765802 Geschäftsführer: Lukas Kallies, Daniel Kobras, Mark Pröhl ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] Disable identity_upcall and ACL
Hi Aurélien! Am 09.01.19 um 14:30 schrieb Degremont, Aurelien: > The secondary group thing was ok to me. I got this idea even if there is some > weird results during my tests. Looks like you can overwrite MDT checks if > user and group is properly defined on client node. Cache effects? In a talk I gave a decade ago, I described a problem with authorization due to inconsistencies between client and MDT, depending on whether metadata was in the client cache or not (see p. 23 of http://wiki.lustre.org/images/b/ba/Tuesday_lustre_automotive.pdf -- you really managed to challenge my memory ;-) I faintly remember Andreas commenting that the MDT was always supposed to be authoritative, even for cached content, and the experienced behaviour was a bug. Indeed, other than those prehistoric versions, I'm not aware of any inconsistencies in authorization due to cache effects. > ACL is really the thing I was interested in. Who is validating the ACLs? MDT, > client or both? Do you think ACL could be properly applied if user/groups are > only defined on client side and identity_upcall is disabled on MDT side? Posix ACLs use numeric uids and gids, just like ordinary permission bits. Evaluation is supposed to happens on the MDT for both. If you can do without secondary groups, there's no need for user and group databases on the MDT--numeric id will work fine. (Unless you use Kerberos, which will typically require user names for proper id mapping.) Kind regards, Daniel -- Daniel Kobras Principal Architect Puzzle ITC Deutschland +49 7071 14316 0 www.puzzle-itc.de -- Puzzle ITC Deutschland GmbH Sitz der Gesellschaft: Jurastr. 27/1, 72072 Tübingen Eingetragen am Amtsgericht Stuttgart HRB 765802 Geschäftsführer: Lukas Kallies, Daniel Kobras, Mark Pröhl ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] Kernel Module Build
I configured like so:
attaufer@head-2:~/lustre-release> ./configure --disable-server
--with-linux=/nfshome/attaufer/kernel/usr/src/kernels/4.14.54-mos-tc3-20181113
The make failed as follows:
attaufer@head-2:~/lustre-release> make rpms
make dist-gzip am__post_remove_distdir='@:'
make[1]: Entering directory '/nfshome/attaufer/lustre-release'
grep -v config.h.in config.h.in > undef.h
if test -d "lustre-2.10.6"; then find "lustre-2.10.6" -type d ! -perm -200
-exec chmod u+w {} ';' && rm -rf "lustre-2.10.6" || { sleep 5 && rm -rf
"lustre-2.10.6"; }; else :; fi
test -d "lustre-2.10.6" || mkdir "lustre-2.10.6"
(cd ldiskfs && make top_distdir=../lustre-2.10.6
distdir=../lustre-2.10.6/ldiskfs \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[2]: Entering directory '/nfshome/attaufer/lustre-release/ldiskfs'
make[2]: Leaving directory '/nfshome/attaufer/lustre-release/ldiskfs'
(cd lustre-iokit && make top_distdir=../lustre-2.10.6
distdir=../lustre-2.10.6/lustre-iokit \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[2]: Entering directory '/nfshome/attaufer/lustre-release/lustre-iokit'
(cd obdfilter-survey && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/lustre-iokit/obdfilter-survey \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[3]: Entering directory
'/nfshome/attaufer/lustre-release/lustre-iokit/obdfilter-survey'
make[3]: Leaving directory
'/nfshome/attaufer/lustre-release/lustre-iokit/obdfilter-survey'
(cd sgpdd-survey && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/lustre-iokit/sgpdd-survey \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[3]: Entering directory
'/nfshome/attaufer/lustre-release/lustre-iokit/sgpdd-survey'
make[3]: Leaving directory
'/nfshome/attaufer/lustre-release/lustre-iokit/sgpdd-survey'
(cd ost-survey && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/lustre-iokit/ost-survey \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[3]: Entering directory
'/nfshome/attaufer/lustre-release/lustre-iokit/ost-survey'
make[3]: Leaving directory
'/nfshome/attaufer/lustre-release/lustre-iokit/ost-survey'
(cd ior-survey && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/lustre-iokit/ior-survey \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[3]: Entering directory
'/nfshome/attaufer/lustre-release/lustre-iokit/ior-survey'
make[3]: Leaving directory
'/nfshome/attaufer/lustre-release/lustre-iokit/ior-survey'
(cd mds-survey && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/lustre-iokit/mds-survey \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[3]: Entering directory
'/nfshome/attaufer/lustre-release/lustre-iokit/mds-survey'
make[3]: Leaving directory
'/nfshome/attaufer/lustre-release/lustre-iokit/mds-survey'
(cd stats-collect && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/lustre-iokit/stats-collect \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[3]: Entering directory
'/nfshome/attaufer/lustre-release/lustre-iokit/stats-collect'
make[3]: Leaving directory
'/nfshome/attaufer/lustre-release/lustre-iokit/stats-collect'
make[2]: Leaving directory '/nfshome/attaufer/lustre-release/lustre-iokit'
(cd libcfs && make top_distdir=../lustre-2.10.6
distdir=../lustre-2.10.6/libcfs \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[2]: Entering directory '/nfshome/attaufer/lustre-release/libcfs'
(cd libcfs && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/libcfs/libcfs \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[3]: Entering directory '/nfshome/attaufer/lustre-release/libcfs/libcfs'
(cd linux && make top_distdir=../../../lustre-2.10.6
distdir=../../../lustre-2.10.6/libcfs/libcfs/linux \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[4]: Entering directory
'/nfshome/attaufer/lustre-release/libcfs/libcfs/linux'
make[4]: Leaving directory
'/nfshome/attaufer/lustre-release/libcfs/libcfs/linux'
(cd util && make top_distdir=../../../lustre-2.10.6
distdir=../../../lustre-2.10.6/libcfs/libcfs/util \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[4]: Entering directory
'/nfshome/attaufer/lustre-release/libcfs/libcfs/util'
make[4]: Leaving directory '/nfshome/attaufer/lustre-release/libcfs/libcfs/util'
make[3]: Leaving directory '/nfshome/attaufer/lustre-release/libcfs/libcfs'
(cd include && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/libcfs/include \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[3]: Entering directory '/nfshome/attaufer/lustre-rele
Re: [lustre-discuss] Kernel Module Build
2.10.6 does not support Linux 4.14.
There is several patches that needs to be cherry picked to have it working.
2.12 is fine though.
Aurélien
Le 09/01/2019 18:43, « lustre-discuss au nom de Tauferner, Andrew T »
a écrit :
I configured like so:
attaufer@head-2:~/lustre-release> ./configure --disable-server
--with-linux=/nfshome/attaufer/kernel/usr/src/kernels/4.14.54-mos-tc3-20181113
The make failed as follows:
attaufer@head-2:~/lustre-release> make rpms
make dist-gzip am__post_remove_distdir='@:'
make[1]: Entering directory '/nfshome/attaufer/lustre-release'
grep -v config.h.in config.h.in > undef.h
if test -d "lustre-2.10.6"; then find "lustre-2.10.6" -type d ! -perm -200
-exec chmod u+w {} ';' && rm -rf "lustre-2.10.6" || { sleep 5 && rm -rf
"lustre-2.10.6"; }; else :; fi
test -d "lustre-2.10.6" || mkdir "lustre-2.10.6"
(cd ldiskfs && make top_distdir=../lustre-2.10.6
distdir=../lustre-2.10.6/ldiskfs \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[2]: Entering directory '/nfshome/attaufer/lustre-release/ldiskfs'
make[2]: Leaving directory '/nfshome/attaufer/lustre-release/ldiskfs'
(cd lustre-iokit && make top_distdir=../lustre-2.10.6
distdir=../lustre-2.10.6/lustre-iokit \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[2]: Entering directory '/nfshome/attaufer/lustre-release/lustre-iokit'
(cd obdfilter-survey && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/lustre-iokit/obdfilter-survey \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[3]: Entering directory
'/nfshome/attaufer/lustre-release/lustre-iokit/obdfilter-survey'
make[3]: Leaving directory
'/nfshome/attaufer/lustre-release/lustre-iokit/obdfilter-survey'
(cd sgpdd-survey && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/lustre-iokit/sgpdd-survey \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[3]: Entering directory
'/nfshome/attaufer/lustre-release/lustre-iokit/sgpdd-survey'
make[3]: Leaving directory
'/nfshome/attaufer/lustre-release/lustre-iokit/sgpdd-survey'
(cd ost-survey && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/lustre-iokit/ost-survey \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[3]: Entering directory
'/nfshome/attaufer/lustre-release/lustre-iokit/ost-survey'
make[3]: Leaving directory
'/nfshome/attaufer/lustre-release/lustre-iokit/ost-survey'
(cd ior-survey && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/lustre-iokit/ior-survey \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[3]: Entering directory
'/nfshome/attaufer/lustre-release/lustre-iokit/ior-survey'
make[3]: Leaving directory
'/nfshome/attaufer/lustre-release/lustre-iokit/ior-survey'
(cd mds-survey && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/lustre-iokit/mds-survey \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[3]: Entering directory
'/nfshome/attaufer/lustre-release/lustre-iokit/mds-survey'
make[3]: Leaving directory
'/nfshome/attaufer/lustre-release/lustre-iokit/mds-survey'
(cd stats-collect && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/lustre-iokit/stats-collect \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[3]: Entering directory
'/nfshome/attaufer/lustre-release/lustre-iokit/stats-collect'
make[3]: Leaving directory
'/nfshome/attaufer/lustre-release/lustre-iokit/stats-collect'
make[2]: Leaving directory '/nfshome/attaufer/lustre-release/lustre-iokit'
(cd libcfs && make top_distdir=../lustre-2.10.6
distdir=../lustre-2.10.6/libcfs \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[2]: Entering directory '/nfshome/attaufer/lustre-release/libcfs'
(cd libcfs && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/libcfs/libcfs \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[3]: Entering directory '/nfshome/attaufer/lustre-release/libcfs/libcfs'
(cd linux && make top_distdir=../../../lustre-2.10.6
distdir=../../../lustre-2.10.6/libcfs/libcfs/linux \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[4]: Entering directory
'/nfshome/attaufer/lustre-release/libcfs/libcfs/linux'
make[4]: Leaving directory
'/nfshome/attaufer/lustre-release/libcfs/libcfs/linux'
(cd util && make top_distdir=../../../lustre-2.10.6
distdir=../../../lustre-2.10.6/libcfs/libcfs/util \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[4
Re: [lustre-discuss] Kernel Module Build
Interesting that I can successfully build a 4.14 kernel and run the Lustre
client using the modules that came from that kernel. The Lustre code in the
kernel tree appears to be really old (2.7?).
Andrew Tauferner
1-952-562-4944 (office)
1-507-696-4609 (mobile)
-Original Message-
From: Degremont, Aurelien [mailto:degre...@amazon.com]
Sent: Wednesday, January 9, 2019 11:49 AM
To: Tauferner, Andrew T ; Andreas Dilger
Cc: lustre-discuss@lists.lustre.org
Subject: Re: [lustre-discuss] Kernel Module Build
2.10.6 does not support Linux 4.14.
There is several patches that needs to be cherry picked to have it working.
2.12 is fine though.
Aurélien
Le 09/01/2019 18:43, « lustre-discuss au nom de Tauferner, Andrew T »
a écrit :
I configured like so:
attaufer@head-2:~/lustre-release> ./configure --disable-server
--with-linux=/nfshome/attaufer/kernel/usr/src/kernels/4.14.54-mos-tc3-20181113
The make failed as follows:
attaufer@head-2:~/lustre-release> make rpms
make dist-gzip am__post_remove_distdir='@:'
make[1]: Entering directory '/nfshome/attaufer/lustre-release'
grep -v config.h.in config.h.in > undef.h
if test -d "lustre-2.10.6"; then find "lustre-2.10.6" -type d ! -perm -200
-exec chmod u+w {} ';' && rm -rf "lustre-2.10.6" || { sleep 5 && rm -rf
"lustre-2.10.6"; }; else :; fi
test -d "lustre-2.10.6" || mkdir "lustre-2.10.6"
(cd ldiskfs && make top_distdir=../lustre-2.10.6
distdir=../lustre-2.10.6/ldiskfs \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[2]: Entering directory '/nfshome/attaufer/lustre-release/ldiskfs'
make[2]: Leaving directory '/nfshome/attaufer/lustre-release/ldiskfs'
(cd lustre-iokit && make top_distdir=../lustre-2.10.6
distdir=../lustre-2.10.6/lustre-iokit \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[2]: Entering directory '/nfshome/attaufer/lustre-release/lustre-iokit'
(cd obdfilter-survey && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/lustre-iokit/obdfilter-survey \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[3]: Entering directory
'/nfshome/attaufer/lustre-release/lustre-iokit/obdfilter-survey'
make[3]: Leaving directory
'/nfshome/attaufer/lustre-release/lustre-iokit/obdfilter-survey'
(cd sgpdd-survey && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/lustre-iokit/sgpdd-survey \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[3]: Entering directory
'/nfshome/attaufer/lustre-release/lustre-iokit/sgpdd-survey'
make[3]: Leaving directory
'/nfshome/attaufer/lustre-release/lustre-iokit/sgpdd-survey'
(cd ost-survey && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/lustre-iokit/ost-survey \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[3]: Entering directory
'/nfshome/attaufer/lustre-release/lustre-iokit/ost-survey'
make[3]: Leaving directory
'/nfshome/attaufer/lustre-release/lustre-iokit/ost-survey'
(cd ior-survey && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/lustre-iokit/ior-survey \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[3]: Entering directory
'/nfshome/attaufer/lustre-release/lustre-iokit/ior-survey'
make[3]: Leaving directory
'/nfshome/attaufer/lustre-release/lustre-iokit/ior-survey'
(cd mds-survey && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/lustre-iokit/mds-survey \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[3]: Entering directory
'/nfshome/attaufer/lustre-release/lustre-iokit/mds-survey'
make[3]: Leaving directory
'/nfshome/attaufer/lustre-release/lustre-iokit/mds-survey'
(cd stats-collect && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/lustre-iokit/stats-collect \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[3]: Entering directory
'/nfshome/attaufer/lustre-release/lustre-iokit/stats-collect'
make[3]: Leaving directory
'/nfshome/attaufer/lustre-release/lustre-iokit/stats-collect'
make[2]: Leaving directory '/nfshome/attaufer/lustre-release/lustre-iokit'
(cd libcfs && make top_distdir=../lustre-2.10.6
distdir=../lustre-2.10.6/libcfs \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[2]: Entering directory '/nfshome/attaufer/lustre-release/libcfs'
(cd libcfs && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/libcfs/libcfs \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[3]: Entering directory '/nfshome/attaufer/lustre-release/libcfs/libcfs'
(cd linux && make top_distdir=../../../lustre-2.1
Re: [lustre-discuss] Kernel Module Build
Is Lustre 2.12 considered to be suitable for production or is that a
development version?
Andrew Tauferner
1-952-562-4944 (office)
1-507-696-4609 (mobile)
-Original Message-
From: Degremont, Aurelien [mailto:degre...@amazon.com]
Sent: Wednesday, January 9, 2019 11:49 AM
To: Tauferner, Andrew T ; Andreas Dilger
Cc: lustre-discuss@lists.lustre.org
Subject: Re: [lustre-discuss] Kernel Module Build
2.10.6 does not support Linux 4.14.
There is several patches that needs to be cherry picked to have it working.
2.12 is fine though.
Aurélien
Le 09/01/2019 18:43, « lustre-discuss au nom de Tauferner, Andrew T »
a écrit :
I configured like so:
attaufer@head-2:~/lustre-release> ./configure --disable-server
--with-linux=/nfshome/attaufer/kernel/usr/src/kernels/4.14.54-mos-tc3-20181113
The make failed as follows:
attaufer@head-2:~/lustre-release> make rpms
make dist-gzip am__post_remove_distdir='@:'
make[1]: Entering directory '/nfshome/attaufer/lustre-release'
grep -v config.h.in config.h.in > undef.h
if test -d "lustre-2.10.6"; then find "lustre-2.10.6" -type d ! -perm -200
-exec chmod u+w {} ';' && rm -rf "lustre-2.10.6" || { sleep 5 && rm -rf
"lustre-2.10.6"; }; else :; fi
test -d "lustre-2.10.6" || mkdir "lustre-2.10.6"
(cd ldiskfs && make top_distdir=../lustre-2.10.6
distdir=../lustre-2.10.6/ldiskfs \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[2]: Entering directory '/nfshome/attaufer/lustre-release/ldiskfs'
make[2]: Leaving directory '/nfshome/attaufer/lustre-release/ldiskfs'
(cd lustre-iokit && make top_distdir=../lustre-2.10.6
distdir=../lustre-2.10.6/lustre-iokit \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[2]: Entering directory '/nfshome/attaufer/lustre-release/lustre-iokit'
(cd obdfilter-survey && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/lustre-iokit/obdfilter-survey \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[3]: Entering directory
'/nfshome/attaufer/lustre-release/lustre-iokit/obdfilter-survey'
make[3]: Leaving directory
'/nfshome/attaufer/lustre-release/lustre-iokit/obdfilter-survey'
(cd sgpdd-survey && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/lustre-iokit/sgpdd-survey \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[3]: Entering directory
'/nfshome/attaufer/lustre-release/lustre-iokit/sgpdd-survey'
make[3]: Leaving directory
'/nfshome/attaufer/lustre-release/lustre-iokit/sgpdd-survey'
(cd ost-survey && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/lustre-iokit/ost-survey \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[3]: Entering directory
'/nfshome/attaufer/lustre-release/lustre-iokit/ost-survey'
make[3]: Leaving directory
'/nfshome/attaufer/lustre-release/lustre-iokit/ost-survey'
(cd ior-survey && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/lustre-iokit/ior-survey \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[3]: Entering directory
'/nfshome/attaufer/lustre-release/lustre-iokit/ior-survey'
make[3]: Leaving directory
'/nfshome/attaufer/lustre-release/lustre-iokit/ior-survey'
(cd mds-survey && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/lustre-iokit/mds-survey \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[3]: Entering directory
'/nfshome/attaufer/lustre-release/lustre-iokit/mds-survey'
make[3]: Leaving directory
'/nfshome/attaufer/lustre-release/lustre-iokit/mds-survey'
(cd stats-collect && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/lustre-iokit/stats-collect \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[3]: Entering directory
'/nfshome/attaufer/lustre-release/lustre-iokit/stats-collect'
make[3]: Leaving directory
'/nfshome/attaufer/lustre-release/lustre-iokit/stats-collect'
make[2]: Leaving directory '/nfshome/attaufer/lustre-release/lustre-iokit'
(cd libcfs && make top_distdir=../lustre-2.10.6
distdir=../lustre-2.10.6/libcfs \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[2]: Entering directory '/nfshome/attaufer/lustre-release/libcfs'
(cd libcfs && make top_distdir=../../lustre-2.10.6
distdir=../../lustre-2.10.6/libcfs/libcfs \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=:
distdir)
make[3]: Entering directory '/nfshome/attaufer/lustre-release/libcfs/libcfs'
(cd linux && make top_distdir=../../../lustre-2.10.6
distdir=../../../lustre-2.10.6/libcfs/libcfs/linux \
am__remove_distdir=: am__skip_length_chec
Re: [lustre-discuss] Kernel Module Build
Lustre 2.12.0 is the next LTS (Long Term Support) release. IMHO it would be much better than using the old in-kernel client. Note you can see which kernels the code will build against in lustre/ChangeLog. I see that 2.10.6 clients only work up to 4.4 kernels. Cheers, Andreas > On Jan 9, 2019, at 12:22, Tauferner, Andrew T > wrote: > > Is Lustre 2.12 considered to be suitable for production or is that a > development version? > > Andrew Tauferner > 1-952-562-4944 (office) > 1-507-696-4609 (mobile) > > > -Original Message- > From: Degremont, Aurelien [mailto:degre...@amazon.com] > Sent: Wednesday, January 9, 2019 11:49 AM > To: Tauferner, Andrew T ; Andreas Dilger > > Cc: lustre-discuss@lists.lustre.org > Subject: Re: [lustre-discuss] Kernel Module Build > > 2.10.6 does not support Linux 4.14. > There is several patches that needs to be cherry picked to have it working. > > 2.12 is fine though. > > > Aurélien > > Le 09/01/2019 18:43, « lustre-discuss au nom de Tauferner, Andrew T » > andrew.t.taufer...@intel.com> a écrit : > >I configured like so: > >attaufer@head-2:~/lustre-release> ./configure --disable-server > --with-linux=/nfshome/attaufer/kernel/usr/src/kernels/4.14.54-mos-tc3-20181113 > > ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] Disable identity_upcall and ACL
On Jan 9, 2019, at 07:52, Daniel Kobras wrote: > > Hi Aurélien! > > Am 09.01.19 um 14:30 schrieb Degremont, Aurelien: >> The secondary group thing was ok to me. I got this idea even if there is >> some weird results during my tests. Looks like you can overwrite MDT checks >> if user and group is properly defined on client node. Cache effects? > > In a talk I gave a decade ago, I described a problem with authorization > due to inconsistencies between client and MDT, depending on whether > metadata was in the client cache or not (see p. 23 of > http://wiki.lustre.org/images/b/ba/Tuesday_lustre_automotive.pdf -- you > really managed to challenge my memory ;-) I faintly remember Andreas > commenting that the MDT was always supposed to be authoritative, even > for cached content, and the experienced behaviour was a bug. Indeed, > other than those prehistoric versions, I'm not aware of any > inconsistencies in authorization due to cache effects. If admins completely trust the client nodes (e.g. they are on a secure network) or they completely _distrust_ them (e.g. subdirectory mounts with nodemaps/idmaps and Kerberos/SSK to identify them), or the data just isn't that secret, then allowing the client to handle the group lookups instead of the MDS is mostly OK. The main issue is for new, uncached lookups from the client. Since the RPC only includes the UID, GID, and maybe one supplementary GID, it is possible that the MDS (without the identity_upcall) may deny the lookup because the request does not contain any IDs that would allow file access. If the file is cached on the client, then the MDS doesn't get involved at all, and the VFS has full access to the IDs and can make the decision locally. This may lead to inconsistent behaviour as the file moves in and out of the client cache (e.g. a shared file that is accessed by multiple users, some of which have direct UID/GID access, others which only have access via supp GID). This access pattern is fairly uncommon, however. >> ACL is really the thing I was interested in. Who is validating the ACLs? >> MDT, client or both? Do you think ACL could be properly applied if >> user/groups are only defined on client side and identity_upcall is disabled >> on MDT side? > > Posix ACLs use numeric uids and gids, just like ordinary permission > bits. Evaluation is supposed to happens on the MDT for both. If you can > do without secondary groups, there's no need for user and group > databases on the MDT--numeric id will work fine. (Unless you use > Kerberos, which will typically require user names for proper id mapping. I believe the MDS is also verifying the file access via ACL, so if the only reason a user can access the file is because of a supp GID that is missing (due to no identity_upcall or inconsistent /etc/groups) then access would be denied. The other (maybe lesser?) risk is if access should be denied because of a supplementary group, the MDS will actually allow access to the inode to the client. I'm not sure if the client will revalidate the ACLs locally and deny this on first open or not. In summary, no identity_upcall would _mostly_ work, especially for simple usage modes. I guess the other question is why you are interested to get rid of it, or what issue you are seeing with it enabled? Cheers, Andreas --- Andreas Dilger Principal Lustre Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
