Re: [lustre-discuss] Nodemap, ssk and mutiple fileset from one client
The main reason is that clients are identified for the nodemap by their NID, and optionally verified by crypto (Kerberos or SSK). That makes it difficult to separate two different mounts from the same client. It would potentially be possible to have the primary client identification be done by the crypto key, which is passed at mount time, but I don't think anyone is planning to work on this feature. You would of course be welcome to submit a patch if this is important to you. Cheers, Andreas On Jan 26, 2020, at 14:54, Hans Henrik Happe wrote: Thanks, for the input. WRT to one LNET per fileset, is there some technical reason for this design? Cheers, Hans Henrik On 06.01.2020 09.41, Moreno Diego (ID SIS) wrote: I’m not sure about the SSK limitations but I know for sure that you can have multiple filesets belonging to the same filesystem on a client. As you already said, you’ll basically need to have one LNET per fileset (o2ib0, o2ib1, o2ib2), then mount each fileset with the option ‘-o network=’. I gave a talk on our setup during last LAD (https://bit.ly/35oaPl7), slide 24 contains a few details on this. It’s for a routed configuration but we also had it working without LNET routers. Diego From: lustre-discuss <mailto:lustre-discuss-boun...@lists.lustre.org> on behalf of Jeremy Filizetti <mailto:jeremy.filize...@gmail.com> Date: Tuesday, 31 December 2019 at 04:22 To: Hans Henrik Happe <mailto:ha...@nbi.dk> Cc: "lustre-discuss@lists.lustre.org"<mailto:lustre-discuss@lists.lustre.org> <mailto:lustre-discuss@lists.lustre.org> Subject: Re: [lustre-discuss] Nodemap, ssk and mutiple fileset from one client It doesn't look like this would be possible due to nodemap or SSK limitations. As you pointed out, nodemap must associate a NID with a single nodemap. SSK was intentionally tied to nodemap by design. It does a lookup on the nodemap of a NID to verify it matches what is found in the server key. I think even if you used multiple NIDs for a client like o2ib(ib0),o2ib1(ib0) you would still run into issues due to LNet, but I'm not certain on that. Jeremy On Mon, Dec 30, 2019 at 9:30 PM Hans Henrik Happe mailto:ha...@nbi.dk>> wrote: Hi, Is it possible to have one client mount multiple fileset's with different ssk keys. Basically, we would just like to hand out a key to clients that should be allowed to mount a specific fileset (subdir). First, it looks like the nodemap must contain the client NID for it to be able to mount. The key is not enough. Secondly, nodemaps are not allowed hold the same NIDs, so it seems impossible to have multiple ssk protected filesets mounted from one client, unless multiple NIDs are used? Example: For nodes A and B and filesets f0 (key0) and f1 (key1). A: Should be allowed to mount f0 (key0). B: Should be allowed to mount f0 (key0) and f1 (key1). Cheers, Hans Henrik ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] Nodemap, ssk and mutiple fileset from one client
Thanks, for the input. WRT to one LNET per fileset, is there some technical reason for this design? Cheers, Hans Henrik On 06.01.2020 09.41, Moreno Diego (ID SIS) wrote: > > I’m not sure about the SSK limitations but I know for sure that you > can have multiple filesets belonging to the same filesystem on a > client. As you already said, you’ll basically need to have one LNET > per fileset (o2ib0, o2ib1, o2ib2), then mount each fileset with the > option ‘-o network=’. > > > > I gave a talk on our setup during last LAD (https://bit.ly/35oaPl7), > slide 24 contains a few details on this. It’s for a routed > configuration but we also had it working without LNET routers. > > > > Diego > > > > > > *From: *lustre-discuss on > behalf of Jeremy Filizetti > *Date: *Tuesday, 31 December 2019 at 04:22 > *To: *Hans Henrik Happe > *Cc: *"lustre-discuss@lists.lustre.org" > *Subject: *Re: [lustre-discuss] Nodemap, ssk and mutiple fileset from > one client > > > > It doesn't look like this would be possible due to nodemap or SSK > limitations. As you pointed out, nodemap must associate a NID with a > single nodemap. SSK was intentionally tied to nodemap by design. It > does a lookup on the nodemap of a NID to verify it matches what is > found in the server key. I think even if you used multiple NIDs for a > client like o2ib(ib0),o2ib1(ib0) you would still run into issues due > to LNet, but I'm not certain on that. > > > > Jeremy > > > > On Mon, Dec 30, 2019 at 9:30 PM Hans Henrik Happe <mailto:ha...@nbi.dk>> wrote: > > Hi, > > Is it possible to have one client mount multiple fileset's with > different ssk keys. > > Basically, we would just like to hand out a key to clients that should > be allowed to mount a specific fileset (subdir). First, it looks like > the nodemap must contain the client NID for it to be able to > mount. The > key is not enough. Secondly, nodemaps are not allowed hold the same > NIDs, so it seems impossible to have multiple ssk protected filesets > mounted from one client, unless multiple NIDs are used? > > Example: For nodes A and B and filesets f0 (key0) and f1 (key1). > > A: Should be allowed to mount f0 (key0). > B: Should be allowed to mount f0 (key0) and f1 (key1). > > Cheers, > Hans Henrik > ___ > lustre-discuss mailing list > lustre-discuss@lists.lustre.org > <mailto:lustre-discuss@lists.lustre.org> > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org > ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] Nodemap, ssk and mutiple fileset from one client
I’m not sure about the SSK limitations but I know for sure that you can have multiple filesets belonging to the same filesystem on a client. As you already said, you’ll basically need to have one LNET per fileset (o2ib0, o2ib1, o2ib2), then mount each fileset with the option ‘-o network=’. I gave a talk on our setup during last LAD (https://bit.ly/35oaPl7), slide 24 contains a few details on this. It’s for a routed configuration but we also had it working without LNET routers. Diego From: lustre-discuss on behalf of Jeremy Filizetti Date: Tuesday, 31 December 2019 at 04:22 To: Hans Henrik Happe Cc: "lustre-discuss@lists.lustre.org" Subject: Re: [lustre-discuss] Nodemap, ssk and mutiple fileset from one client It doesn't look like this would be possible due to nodemap or SSK limitations. As you pointed out, nodemap must associate a NID with a single nodemap. SSK was intentionally tied to nodemap by design. It does a lookup on the nodemap of a NID to verify it matches what is found in the server key. I think even if you used multiple NIDs for a client like o2ib(ib0),o2ib1(ib0) you would still run into issues due to LNet, but I'm not certain on that. Jeremy On Mon, Dec 30, 2019 at 9:30 PM Hans Henrik Happe mailto:ha...@nbi.dk>> wrote: Hi, Is it possible to have one client mount multiple fileset's with different ssk keys. Basically, we would just like to hand out a key to clients that should be allowed to mount a specific fileset (subdir). First, it looks like the nodemap must contain the client NID for it to be able to mount. The key is not enough. Secondly, nodemaps are not allowed hold the same NIDs, so it seems impossible to have multiple ssk protected filesets mounted from one client, unless multiple NIDs are used? Example: For nodes A and B and filesets f0 (key0) and f1 (key1). A: Should be allowed to mount f0 (key0). B: Should be allowed to mount f0 (key0) and f1 (key1). Cheers, Hans Henrik ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] Nodemap, ssk and mutiple fileset from one client
It doesn't look like this would be possible due to nodemap or SSK limitations. As you pointed out, nodemap must associate a NID with a single nodemap. SSK was intentionally tied to nodemap by design. It does a lookup on the nodemap of a NID to verify it matches what is found in the server key. I think even if you used multiple NIDs for a client like o2ib(ib0),o2ib1(ib0) you would still run into issues due to LNet, but I'm not certain on that. Jeremy On Mon, Dec 30, 2019 at 9:30 PM Hans Henrik Happe wrote: > Hi, > > Is it possible to have one client mount multiple fileset's with > different ssk keys. > > Basically, we would just like to hand out a key to clients that should > be allowed to mount a specific fileset (subdir). First, it looks like > the nodemap must contain the client NID for it to be able to mount. The > key is not enough. Secondly, nodemaps are not allowed hold the same > NIDs, so it seems impossible to have multiple ssk protected filesets > mounted from one client, unless multiple NIDs are used? > > Example: For nodes A and B and filesets f0 (key0) and f1 (key1). > > A: Should be allowed to mount f0 (key0). > B: Should be allowed to mount f0 (key0) and f1 (key1). > > Cheers, > Hans Henrik > ___ > lustre-discuss mailing list > lustre-discuss@lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org > ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
[lustre-discuss] Nodemap, ssk and mutiple fileset from one client
Hi, Is it possible to have one client mount multiple fileset's with different ssk keys. Basically, we would just like to hand out a key to clients that should be allowed to mount a specific fileset (subdir). First, it looks like the nodemap must contain the client NID for it to be able to mount. The key is not enough. Secondly, nodemaps are not allowed hold the same NIDs, so it seems impossible to have multiple ssk protected filesets mounted from one client, unless multiple NIDs are used? Example: For nodes A and B and filesets f0 (key0) and f1 (key1). A: Should be allowed to mount f0 (key0). B: Should be allowed to mount f0 (key0) and f1 (key1). Cheers, Hans Henrik ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
