Hello Hal and Richard. I found what is the problem.
Q: Can I create subinterface with 0 (0x8000)??? My Test result. partitions.conf Default=0x7fff, : ALL=full ; Net0=0x0000, ipoib : ALL=full ; Net1=0x0001, ipoib : ALL=full ; Net2=0x0002, ipoib : ALL=full ; 1) Server1 ib.8000 -> Server1 ib.8000 Ping NG 2) Server1 ib.8001 -> Server1 ib.8001 The differece is subinterfae number. Yesterday, I always tested subinterface 0x8000. And about previous post, The partitions.conf which I posted few hous ago, was semi colon missing. > * partitions.conf is the following > Default=0x7fff, ipoib : ALL=full > Net0=0x0001, ipoib : ALL=full Should be Default=0x7fff, ipoib : ALL=full ; Net0=0x0001, ipoib : ALL=full ; Thank you again. 2011/10/27 Hiroyuki Sato <hiroys...@gmail.com>: > Hello Hal > > Thank you for your information. > > I simplified my test environment > The environment is the folloing. > > I'm not sure what is wrong Test2 (Partition test) > And Could you please tell me how to check the problem?? > (tool, logfile, etc.. ) > > * Environment > > OS: Scientific Linux6.1 > OFED: 1.5.3. > > * Diagram > > +----------+ib0 ib0+----------+ > | Server1 |------------------| Server2 | > +----------+ +----------+ > > > > > 1) Test1 (Simple IPoIB no partition Ping test) > > Server1 > > a) /sbin/ifconfig ib0 inet 192.168.1.1/24 > b) /sbn/service opensmd start > 3) /sbin/service opensmd start > * no /etc/opensm/partitions.conf > > Server2 > /sbin/ifconfig ib0 inet 192.168.1.1/24 > > Test from server1(192.168.1.1) to server2(192.168.1.2) > ping 192.168.1.2 > The result was OK. > > 2) Test2 (Add partition by hand and Ping test) > > reboot Server 1 and Server 2 > > Server 1 > 1) echo 0x8001 > /sys/class/net/ib0/create_child > 2) /sbin/ifconfig ib0.8001 inet 192.168.0.1/24 > 3) /sbin/service opensmd start > > * partitions.conf is the following > Default=0x7fff, ipoib : ALL=full > Net0=0x0001, ipoib : ALL=full > > Server2 > 1) echo 0x8001 > /sys/class/net/ib0/create_child > 2) /sbin/ifconfig ib0.8001 inet 192.168.0.2/24 > > Test from server1(ib0.8001/192.168.0.1) to server2(ib0.8001/192.168.0.2) > > ping 192.168.0.2 > no response. > > 3) Test2 Log > > I enabled debug_log parameter in ib_ipoib modules. > > cd /sys/module/ib_ipoib/parameters > # cat debug_level > 1 > > Could you please tell me what is wrong?? > > ib0: enabling connected mode will cause multicast packet drops > ib0: mtu > 2044 will cause multicast packet drops. > ib0: mtu > 2044 will cause multicast packet drops. > ib1: enabling connected mode will cause multicast packet drops > ib1: mtu > 2044 will cause multicast packet drops. > ib1: mtu > 2044 will cause multicast packet drops. > ib0: Event 17 on device mthca0 port 1 > ib0: Not flushing - IPOIB_FLAG_INITIALIZED not set. > ib0: Event 11 on device mthca0 port 1 > ib0: Not flushing - IPOIB_FLAG_INITIALIZED not set. > ib0: Event 9 on device mthca0 port 1 > ib0: Not flushing - IPOIB_FLAG_INITIALIZED not set. > ib0.8001: max_srq_sge=27 > ib0.8001: max_cm_mtu = 0xfff0, num_frags=16 > ib0.8001: bringing up interface > ib0.8001: IPOIB_FLAG_OPER_UP not set > ib0.8001: IPOIB_FLAG_OPER_UP not set > ADDRCONF(NETDEV_UP): ib0.8001: link is not ready > ib0.8001: IPOIB_FLAG_OPER_UP not set > > regards. > > 2011/10/27 Hal Rosenstock <hal.rosenst...@gmail.com>: >> >> >> On Wed, Oct 26, 2011 at 10:03 AM, Hiroyuki Sato <hiroys...@gmail.com> wrote: >>> >>> Dear members. >>> >>> I have some question about Infiniband Partitions. >>> >>> I would like to build Linux box with IPoIB Router. >>> >>> * Questions >>> >>> My Question is the following. >>> >>> a) Can I create the following IPoIB network ?? >>> >>> b) If so, how to setup /etc/opensm/partitions.conf?? >>> >>> * Problem. >>> >>> Ping IB Client to Router : I got the followig error. >>> >>> Network is unreachable. >>> >>> * Environment >>> >>> OFED: OFED-1.5.3.2 >>> OS: Scientific Linux 6.1 >>> >>> * Linux box (Router) >>> >>> 1 ethernet interface (eth0) >>> 1 Infiniband interfae (ib0) >>> Run subnet manager. >>> >>> * Ethernet setup >>> >>> a) add VLAN ID 120 eth0.120 (192.168.120.0/24) >>> b) add VLAN ID 130 eth0.130 (192.168.130.0/24) >>> >>> * Infiniband Setup >>> a) add subinterface 0 ib0.8000 (192.168.0.0/24) >>> b) add subinterface 1 ib0.8001 (192.168.1.0/24) >>> c) Run opensm >>> >>> >>> I crated subinterface with the following command. >>> >>> echo 0 > /sys/class/net/ib0/create_child >>> echo 1 > /sys/class/net/ib0/create_child >> >> >> echo 0x8001 > /sys/class/net/ib0/create_child >> echo 0x8002 > /sys/class/net/ib0/create_child >> >>> >>> and assign ip address with ifconfig >>> >>> * Logical Diagram >>> >>> +----------- 192.168.120.0/24 --- >>> | >>> | +---- 192.168.130.0/24 >>> eth0.120| | eth0.130 >>> +----------+ >>> | Router | >>> +----------+ >>> ib0.8000| |ib0.8001 >>> 192.168.0.1 | +---- 192.168.1.0/24 >>> | >>> +----------- 192.168.0.0/24 --- IB Cllient. (ib0.8000) >>> (192.168.0.2) >>> >>> * /etc/opensm/partitions.conf >>> >>> Default=0x7fff : ALL=full >>> Net0=0x00, ipoib : ALL=full >>> Net1=0x01, ipoib : ALL=full >> >> >> Default=0x7fff: ALL=full >> Net0=0x0001, ipoib: ALL=full >> Net1=0x0002, ipoib: ALL=full >> >> >>> >>> Coule you please tell me what is wrong?? >> >> >> The one thing in the above I'm not sure of is whether the default partition >> also needs the ipoib flag above so you might need to add that too if the >> above doesn't work. >> >> -- Hal >> >>> >>> Sincerely. >>> >>> -- >>> Hiroyuki Sato. >>> _______________________________________________ >>> ewg mailing list >>> ewg@lists.openfabrics.org >>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg >> >> > > > > -- > Hiroyuki Sato > -- Hiroyuki Sato _______________________________________________ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg