[lustre-discuss] Can Linux FS-Cache/CacheFS run on Lustre 2.10.x

[Forrest.Wc.Ling]

Dear All:

The best practice of DGX-1 on storage for DL, requires 4x SSDs to be local 
cache backend to improve IO performance by using Linux cacheFS.
http://docs.nvidia.com/deeplearning/dgx/pdf/Best-Practices.pdf
6.1.1. Internal Storage
The first storage consideration is storage within the DGX-1 itself. For the 
best possible performance, a NFS read cache has been included in the DGX-1 
appliance using the Linux cacheFS capability. It uses four SSD’s in a RAID-0 
group. The drives are connected to a dedicated hardware RAID controller.

I looked Redhat Linux, the manual shows the FS-Cache is a persistent local 
cache that can be used by file systems to take data retrieved from over the 
network and cache it on local disk. This helps minimize network traffic for 
users accessing data from a file system mounted over the network (for example, 
NFS).

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/storage_administration_guide/ch-fscache

Can Linux FS-Cache/CacheFS run on Lustre 2.10.x   or will the Persistent 
Client-side Cache on Lustre 2.12 have the same function as FS-Cache on NFS to 
be released next year ?

Thanks,
Forrest Ling (凌巍才)
+86 18600622522
Dell HPC Product Technologist Greater China
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

[lustre-discuss] E5-2667 or E5-2697A for MDS

[Forrest.Wc.Ling]

Hello :

Our customer is going to configure Lustre FS, which will have a lot of small 
files to be accessed.

We are to configure 1TB Memory for MDS.

Regarding to CPU configuration , can we propose E5-2667 or E5-2697A for MDS for 
good performance ?

E5-2667 v4 : 3.2GHz, 8 Cores;
E5-2697A v4: 2.6GHz, 16 cores ;

Thanks,
Forrest Ling (凌巍才)
+86 18600622522
Dell HPC Product Technologist Greater China
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

[lustre-discuss] Can Lustre Be Mount to Kuberentes Pod

[Forrest.Wc.Ling]

Hi Lustre experts:

Can Lustre file system be mounted to Kuberentes Pods as a persistent volumes ?

Thanks,
Forrest Ling (凌巍才)
+86 18600622522
Dell HPC Product Technologist Greater China

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

[lustre-discuss] Best Practice of Copy within Lustre

[Forrest.Wc.Ling]

Dear all:

We have a customer deploy a 18 nodes cluster with a Lustre storage.

IOzone runs a good performance.

While my customer find the command “cp” is not as fast as IOzone. They want to 
copy

1.a big size file, or

2.   a directory

3.   or a large number of small size files
within the Lustre file system.

I asked Bright for if I can install MCP for the parallel copy, understanding 
that MCP requires a lot software packages on BCM. I will not install MCP on my 
BCM software.

Do you have any best practice of copy, within a Lustre FS ?

Will my customer have to write a Slurm Job Script to do the parallel copying ? 
or do you have any best practice on copying within a Lustre FS ?


Thanks,
Forrest Ling (凌巍才)
+86 18600622522
Dell HPC Product Technologist Greater China

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org