Hi Brett,
I am using MOFED “MLNX_OFED_LINUX-4.1-1.0.2.0” with kernel “3.10.0-514.el7.x86_64” o/p of modprobe –v ko2iblnd [root@mds1 ~]# modprobe -v ko2iblnd install /usr/sbin/ko2iblnd-probe require_privileged_port=0 use_privileged_port=0 timeout=150 retry_count=7 map_on_demand=32 peer_credits=63 concurrent_sends=63 ntx=32768 credits=32768 fmr_pool_size=8193 insmod /lib/modules/3.10.0-514.el7.x86_64/extra/lustre/net/ko2iblnd.ko require_privileged_port=0 use_privileged_port=0 timeout=150 retry_count=7 map_on_demand=32 peer_credits=63 concurrent_sends=63 ntx=32768 credits=32768 fmr_pool_size=8193 modprobe: ERROR: could not insert 'ko2iblnd': Invalid argument modprobe: ERROR: Error running install command for ko2iblnd modprobe: ERROR: could not insert 'ko2iblnd': Operation not permitted [root@mds1 ~]# Regards, Parag From: Brett Lee [mailto:brettlee.lus...@gmail.com] Sent: Saturday, November , 2017 8:08 PM To: Parag Khuraswar Cc: Mannthey, Keith; lustre-discuss@lists.lustre.org Subject: Re: [lustre-discuss] error while configuring lnet Hi Parag, You may need to confirm that the in-kernel IB and the IB in the kernel module "match" (are compatible). I think that loading the module (`sudo modprobe -v ko2iblnd`) may be sufficient to verify the match (it's been a while, others may correct me). Please indicate which kernel and which IB you are using. Brett -- Protect yourself against cybercrime PDS Software Solutions https://www.TrustPDS.com <https://www.trustpds.com/> On Fri, Nov 10, 2017 at 8:30 PM, Parag Khuraswar <para...@citilindia.com> wrote: Hi Keith, Below errors I am getting while adding lnet and mounting mdt. dmesg logs while adding lnet ========================================= [317831.432182] LNetError: 28362:0:(api-ni.c:1861:lnet_startup_lndnet()) Can't load LND o2ib, module ko2iblnd, rc=256 ========================================= dmesg logs while mounting mdt ========================================== [290476.172602] LNetError: 23040:0:(api-ni.c:1861:lnet_startup_lndnet()) Can't load LND o2ib, module ko2iblnd, rc=256 [317478.730515] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [317480.166277] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [317480.313296] LustreError: 28268:0:(ldlm_lib.c:483:client_obd_setup()) can't add initial connection [317480.313600] LustreError: 28268:0:(obd_config.c:608:class_setup()) setup MGC10.2.1.204@o2ib failed (-2) [317480.313603] LustreError: 28268:0:(obd_mount.c:202:lustre_start_simple()) MGC10.2.1.204@o2ib setup error -2 [317480.313632] LustreError: 28268:0:(obd_mount_server.c:1573:server_put_super()) no obd home-MDT0000 [317480.313635] LustreError: 28268:0:(obd_mount_server.c:132:server_deregister_mount()) home-MDT0000 not registered [317480.433934] Lustre: server umount home-MDT0000 complete [317480.433940] LustreError: 28268:0:(obd_mount.c:1504:lustre_fill_super()) Unable to mount (-2) ========================================== Regards, Parag From: Mannthey, Keith [mailto:keith.mannt...@intel.com] Sent: Saturday, November , 2017 2:06 AM To: Parag Khuraswar; lustre-discuss@lists.lustre.org Subject: RE: [lustre-discuss] error while configuring lnet If you have ib0 device check dmesg for more hints on what is going wrong. Thanks, Keith From: Parag Khuraswar [mailto:para...@citilindia.com] Sent: Friday, November 10, 2017 10:59 AM To: Mannthey, Keith <keith.mannt...@intel.com>; lustre-discuss@lists.lustre.org Subject: RE: [lustre-discuss] error while configuring lnet Hi, Basically I am trying to add lnet. Deleting is just try whether it is happing or not. Main is I want to add o2ib network. Which is giving error “invalid argument ” ================== [root@mds2 ~]# lnetctl net add --net o2ib --if ib0 add: - net: errno: -22 descr: "cannot add network: Invalid argument" ================ I am really not able to understand what argument is invalid in my command. I am able to ping ib0 network Regards, Parag From: Mannthey, Keith [mailto:keith.mannt...@intel.com] Sent: Friday, November , 2017 10:51 PM To: Parag Khuraswar; lustre-discuss@lists.lustre.org Subject: RE: [lustre-discuss] error while configuring lnet What are you trying to accomplish? >From below: 10.1.1.205@tcp is on 0@lo not eno1 and in general you should not need the “—if” option to delete a fabric. Try: # lnetctl net del --net tcp Can you do a normal ping over ib0? “dmesg” can sometime provide greater details about errors like this. Thanks, Keith From: lustre-discuss [mailto:lustre-discuss-boun...@lists.lustre.org] On Behalf Of Parag Khuraswar Sent: Friday, November 10, 2017 9:10 AM To: lustre-discuss@lists.lustre.org Subject: [lustre-discuss] error while configuring lnet Hi, I am trying to add lnet but getting below error. ====================== [root@mds2 ~]# lnetctl net show net: - net type: lo local NI(s): - nid: 0@lo status: up - net type: tcp local NI(s): - nid: 10.1.1.205@tcp status: up [root@mds2 ~]# lnetctl net add --net o2ib --if ib0 add: - net: errno: -22 descr: "cannot add network: Invalid argument" [root@mds2 ~]# lnetctl net del --net tcp --if eno1 del: - net: errno: -22 descr: "cannot del network: Invalid argument" [root@mds2 ~]# lctl list_nids 10.1.1.205@tcp [root@mds2 ~]# ==================================== Regards, Parag _______________________________________________ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
_______________________________________________ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
