[OpenAFS] Running OpenAFS on top of GPFS?
Dear All, The Central Computing Service at Edinburgh University is introducing a new University wide filesystem intended for research based data. We in Informatics have been asked about the possibility of us using of some of this new file space to either expand our existing cell or (more likely) set up a new cell for the whole University to use. Unfortunately, this new research file system is based on GPFS and so this would involve us running AFS on top of GPFS. Does anyone on this list have experience of running AFS on top of GPFS which they would be willing to share with us? Failing that, would anyone like to make an educated guess as to the problems we are likely to encounter if we try this? Any advice would be appreciated, Craig. --- Craig Strachan, Computing Officer, School of Informatics, University of Edinburgh -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] Running OpenAFS on top of GPFS?
Dear Craig, we are running some OSD-Servers on GPFS. The OSD-servers are in the production-run sense comparable to fileservers and since GPFS is POSIX-compatible (dont know to how many %), everything is fine. However, we run those servers as single-server GPFS. They are server and client alike. I don't see any major issues with running AFS on top of GPFS, but ... - GPFS is more likely to hang as any other network-FS compared to a local FS. - not sure how to deal with the /vicep - naming convention ( can you have two /vicepa partitions somehow?) I can check with the GPFS-admins here. HTH, Christof Am Fri, 30 Nov 2012 15:11:37 + schrieb Craig Strachan c...@inf.ed.ac.uk: Dear All, The Central Computing Service at Edinburgh University is introducing a new University wide filesystem intended for research based data. We in Informatics have been asked about the possibility of us using of some of this new file space to either expand our existing cell or (more likely) set up a new cell for the whole University to use. Unfortunately, this new research file system is based on GPFS and so this would involve us running AFS on top of GPFS. Does anyone on this list have experience of running AFS on top of GPFS which they would be willing to share with us? Failing that, would anyone like to make an educated guess as to the problems we are likely to encounter if we try this? Any advice would be appreciated, Craig. --- Craig Strachan, Computing Officer, School of Informatics, University of Edinburgh ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] Running OpenAFS on top of GPFS?
One of the big advantages of GPFS is that it is a fast cluster filesystem. With normal AFS you can not make use of this feature because the fileserver doesn't share his partitions with anyone else. So the fileserver would use GPFS just as his local fileystem instead of XFS or something else. GPFS is good for large files while many many files in AFS are small. In our cell with 800 TB and 200 million files 92 % of the files are smaller than 1 MB which is a reasonable block size for GPFS. However, since some years a special version of AFS called AFS/OSD exists which allows to store large files in object storage. This object storage are servers running a program called rxosd. The idea is to keep the small files in the fileserver's partition where the volume resides and have the large files in object storage. Now the point why GPFS is of special interest here: the GPFS used by the rxosd could be shared by all the compute nodes in a cluster and the modified AFS client would allow users on these compute nodes to read and write data form and to the AFS files located inside the GPFS rxosd partition directly with nearly the native GPFS speed (200-300 MB/s depending on the network being used). User's outside the cluster would see the files as normal AFS files and access them with the normal low transfer rate of AFS. I gave a talk about this some years ago: Embedded Filesystems (Direct Client Access to Vice Partitions) Talk at AFS Kerberos Best Practice Workshop 2009, Stanford, 2009, which you can download from http://www.rzg.mpg.de/~hwr/Stanford.pdf; If you wan't to know more about this, feel free to contact me. Hartmut Reuter Craig Strachan wrote: Dear All, The Central Computing Service at Edinburgh University is introducing a new University wide filesystem intended for research based data. We in Informatics have been asked about the possibility of us using of some of this new file space to either expand our existing cell or (more likely) set up a new cell for the whole University to use. Unfortunately, this new research file system is based on GPFS and so this would involve us running AFS on top of GPFS. Does anyone on this list have experience of running AFS on top of GPFS which they would be willing to share with us? Failing that, would anyone like to make an educated guess as to the problems we are likely to encounter if we try this? Any advice would be appreciated, Craig. --- Craig Strachan, Computing Officer, School of Informatics, University of Edinburgh -- - Hartmut Reuter e-mail reu...@rzg.mpg.de phone+49-89-3299-1328 fax +49-89-3299-1301 RZG (Rechenzentrum Garching)webhttp://www.rzg.mpg.de/~hwr Computing Center of the Max-Planck-Gesellschaft (MPG) and the Institut fuer Plasmaphysik (IPP) - ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
