Hi Christof,
thanks! So the preferred way should be not to have Filesystem_Id. If I
have understood correctly, this is anyhow set up by default in CES to
666.666, so the suggested procedure during the CES setup should be to
manually modify gpfs.ganesha.exports.conf and remove this parameter from
all the exports, is that correct? Is there an easier way, or is there a
plan to remove the 666.666 default value?
In our case, we do rely on two separated CES clusters (one in prod and
one in stand by, so that we can perform upgrades with no downtime by
migrating the IPs from one to the other), so it might be safer to
explicilty set Filesystem_Id, to ensure consistency among the cluster -
would that make sense?
Thanks again!
Regards
leo
Paul Scherrer Institut
Dr. Leonardo Sala
Group Leader Data Analysis and Research Infrastructure
Deputy Department Head a.i Science IT Infrastructure and Services department
Science IT Infrastructure and Services department (AWI)
WHGA/036
Forschungstrasse 111
5232 Villigen PSI
Switzerland
Phone: +41 56 310 3369
leonardo.s...@psi.ch
www.psi.ch
On 6/28/23 15:22, Christof Schmitt wrote:
After another discussion it turns out that this parameter is not
required. While my previous comment is correct, that there is the need
to have unique handles across file systems, GPFS already provides that
information and Ganesha handles that correctly. So there is no need to
set the parameter in the Ganesha config.
Regards,
Christof
On Wed, 2023-06-28 at 14:33 +0200, Leonardo Sala wrote:
Hi Christof, thanks a lot! In our case we are exporting multiple
filesets from 2 filesystems, I guess we should fix unique Fileset_IDs
for each fileset? What would happen in case we just remove the
Fileset_id parameter, as suggested by the
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender
This message came from outside your organization.
Report Suspicious
<https://us-phishalarm-ewt.proofpoint.com/EWT/v1/PjiDSg!2k-hAt5_AJM6OsYbOMsPTZmPil8BIAHadeloM-6vntuXy4WCtpq_71hHj07-QEB3YfmuMTIe9Ezg-qdLzL_6DvzGmA12g21EJgQ$>
ZjQcmQRYFpfptBannerEnd
Hi Christof,
thanks a lot! In our case we are exporting multiple filesets from 2
filesystems, I guess we should fix unique Fileset_IDs for each
fileset? What would happen in case we just remove the Fileset_id
parameter, as suggested by the ganesha docs?
Regards
leo
Paul Scherrer Institut
Dr. Leonardo Sala
Group Leader Data Analysis and Research Infrastructure
Deputy Department Head a.i Science IT Infrastructure and Services department
Science IT Infrastructure and Services department (AWI)
WHGA/036
Forschungstrasse 111
5232 Villigen PSI
Switzerland
Phone: +41 56 310 3369
leonardo.s...@psi.ch
www.psi.ch
On 6/28/23 14:22, Christof Schmitt wrote:
The "FileSystem_Id" is a unique identifier for the file system. The
technical background is that Ganesha asks the file system for a file
handle, but that is only unique within the file system. If there are
NFS exports on different file systems, there needs to be a way to
make the file handles unique across multiple file systems. So if
there are NFS exports on different file systems, this parameter
should be set with a unique value for each file system. If there is
only one file system with NFS exports, then this should not be
necessary.
Regards,
Christof
On Wed, 2023-06-28 at 08:53 +0200, Leonardo Sala wrote:
Hi Ed, thanks! In our case we do have unique export ids, but the
same fsid, and this seems to create issues. Also, reading Ganesha
docs, I can see [*]: FileSystem_ID EXPORT Option There is an EXPORT
config option, FileSystem_ID. This really
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender
This message came from outside your organization.
Report Suspicious
<https://us-phishalarm-ewt.proofpoint.com/EWT/v1/PjiDSg!2k-ror9_yrma2gYRGMtvzfBw-f7-KrLp6ZZNwHChWJfOvNhVe0t6WfmouxHnuoc3gtMvXaxWV3AbFrgYAQT1JIEKoq1hoJ8cr1c$>
ZjQcmQRYFpfptBannerEnd
Hi Ed,
thanks! In our case we do have unique export ids, but the same
fsid, and this seems to create issues. Also, reading Ganesha docs,
I can see [*]:
FileSystem_ID EXPORT Option
There is an EXPORT config option, FileSystem_ID. This really should
not be used, all it does it designate an fsid to be used with the
attributes of all objects in the export. It will be folded to fit
into NFSv3. Because it applies to the entire export, it prevents
exporting multiple file systems since there will likely be issues
with collision of inode numbers on the client.
so before touching the defaults in GPFS CES configuration I would
like some guidance or experiences from this mlist :)
cheers
leo
[*]
https://github.com/nfs-ganesha/nfs-ganesha/wiki/File-Systems#FileSystem_ID_EXPORT_Option
Paul Scherrer Institut
Dr. Leonardo Sala
Group Leader Data Analysis and Research Infrastructure
Deputy Department Head a.i Science IT Infrastructure and Services department
Science IT Infrastructure and Services department (AWI)
WHGA/036
Forschungstrasse 111
5232 Villigen PSI
Switzerland
Phone: +41 56 310 3369
leonardo.s...@psi.ch
www.psi.ch
On 6/27/23 19:41, Wahl, Edward wrote:
I vaguely recall seeing this and testing it. My notes to myself
say: ‘As long as the export_id is unique, you are fine.’ See the
manuals, ganesha loves Camel Case so it’s more than likely
actually “Export_Id” or some such.
Ed Wahl
Ohio Supercomputer Center
*From:*gpfsug-discuss <gpfsug-discuss-boun...@gpfsug.org> *On
Behalf Of *Leonardo Sala
*Sent:* Tuesday, June 27, 2023 10:18 AM
*To:* gpfsug-disc...@spectrumscale.org
*Subject:* [gpfsug-discuss] CES, Ganesha, and Filesystem_id
Hallo, we are checking our current CES configuration, and we
noticed that by default GPFS puts always Filesystem_Id=666. 666
[*], no matter which Export_Id value the export has. To my
understanding (which is poor!), this means that all clients
Hallo,
we are checking our current CES configuration, and we noticed that
by default GPFS puts always Filesystem_Id=666.666 [*], no matter
which Export_Id value the export has. To my understanding (which
is poor!), this means that all clients will see all our exports
(~20) with the same device number, creating various possible
issues (e.g. file state handles). Questions:
* is there a reason for such default value? If we change it, are
there unpleasant effects we could see?
* what would be a reasonable value? Looking around I saw that
Filesystem_Id = Export_Id.Export_Id is quite common, with the
possible issue of using the forbidden 152.152 [**]
* what happens if we actually remove the Filesytem_Id parameter
from gpfs.ganesha.exports.conf?
* is there a way to modify Filesystem_Id in
gpfs.ganesha.exports.conf without editing the file, eg using mmnfs
commands (seems not, but I might be mistaken)?
Thanks a lot!
cheers
leo
[*]
https://www.ibm.com/docs/en/storage-scale/5.0.4?topic=exports-making-bulk-changes-nfs
[**] https://github.com/nfs-ganesha/nfs-ganesha/issues/615
--
Paul Scherrer Institut
Dr. Leonardo Sala
Group Leader Data Analysis and Research Infrastructure
Deputy Department Head a.i Science IT Infrastructure and Services department
Science IT Infrastructure and Services department (AWI)
WHGA/036
Forschungstrasse 111
5232 Villigen PSI
Switzerland
Phone: +41 56 310 3369
leonardo.s...@psi.ch
www.psi.ch
<http://www.psi.ch>
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at gpfsug.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at gpfsug.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org
<http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org>
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at gpfsug.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at gpfsug.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org
