Hi Haomai,
can you please guide me to a running cluster with RDMA ?
regards
Gerhard W. Recher
net4sec UG (haftungsbeschränkt)
Leitenweg 6
86929 Penzing
+49 171 4802507
Am 28.09.2017 um 04:21 schrieb Haomai Wang:
> previously we have a infiniband cluster, recently we deploy a roce
> cluster. th
previously we have a infiniband cluster, recently we deploy a roce
cluster. they are both test purpose for users.
On Wed, Sep 27, 2017 at 11:38 PM, Gerhard W. Recher
wrote:
> Haomai,
>
> I looked at your presentation, so i guess you already have a running
> cluster with RDMA & mellanox
> (https:/
Yep ROcE
i followed up all recommendations in mellanox papers ...
*/etc/security/limits.conf*
* soft memlock unlimited
* hard memlock unlimited
root soft memlock unlimited
root hard memlock unlimited
also set properties on daemons (chapter 11) in
https://community.mellanox.com/docs/DOC-27
Haomai,
I looked at your presentation, so i guess you already have a running
cluster with RDMA & mellanox
(https://www.youtube.com/watch?v=Qb2SUWLdDCw)
Is nobody out there having a running cluster with RDMA ?
any help is appreciated !
Gerhard W. Recher
net4sec UG (haftungsbeschränkt)
Leitenweg
ah ok
but as i stated before : ceph.conf is a cluster wide file on proxmox!
so if i specify
[global]
//Set local GID for ROCEv2 interface used for CEPH
//The GID corresponding to IPv4 or IPv6 networks
//should be taken from show_gids command output
//This parameter should be uniquely set
https://community.mellanox.com/docs/DOC-2415
On Wed, Sep 27, 2017 at 10:01 PM, Gerhard W. Recher
wrote:
> How to set local gid option ?
>
> I have no glue :)
>
> Gerhard W. Recher
>
> net4sec UG (haftungsbeschränkt)
> Leitenweg 6
> 86929 Penzing
>
> +49 171 4802507
> Am 27.09.2017 um 15:59 schrie
How to set local gid option ?
I have no glue :)
Gerhard W. Recher
net4sec UG (haftungsbeschränkt)
Leitenweg 6
86929 Penzing
+49 171 4802507
Am 27.09.2017 um 15:59 schrieb Haomai Wang:
> do you set local gid option?
>
> On Wed, Sep 27, 2017 at 9:52 PM, Gerhard W. Recher
> wrote:
>> Yep ROcE ...
do you set local gid option?
On Wed, Sep 27, 2017 at 9:52 PM, Gerhard W. Recher
wrote:
> Yep ROcE
>
> i followed up all recommendations in mellanox papers ...
>
> */etc/security/limits.conf*
>
> * soft memlock unlimited
> * hard memlock unlimited
> root soft memlock unlimited
> root hard mem
Yep ROcE
i followed up all recommendations in mellanox papers ...
*/etc/security/limits.conf*
* soft memlock unlimited
* hard memlock unlimited
root soft memlock unlimited
root hard memlock unlimited
also set properties on daemons (chapter 11) in
https://community.mellanox.com/docs/DOC-27
Haomai,
ibstat
CA 'mlx4_0'
CA type: MT4103
Number of ports: 2
Firmware version: 2.40.7000
Hardware version: 0
Node GUID: 0x248a070300e26070
System image GUID: 0x248a070300e26070
Port 1:
State: Active
Physical
On Wed, Sep 27, 2017 at 8:33 PM, Gerhard W. Recher
wrote:
> Hi Folks!
>
> I'm totally stuck
>
> rdma is running on my nics, rping udaddy etc will give positive results.
>
> cluster consist of:
> proxmox-ve: 5.0-23 (running kernel: 4.10.17-3-pve)
> pve-manager: 5.0-32 (running version: 5.0-32/2560e
Hi Folks!
I'm totally stuck
rdma is running on my nics, rping udaddy etc will give positive results.
cluster consist of:
proxmox-ve: 5.0-23 (running kernel: 4.10.17-3-pve)
pve-manager: 5.0-32 (running version: 5.0-32/2560e073)
system(4 nodes): Supermicro 2028U-TN24R4T+
2 port Mellanox connect
12 matches
Mail list logo
