On Thu, Oct 27, 2011 at 7:05 AM, Hiroyuki Sato <hiroys...@gmail.com> wrote:
> Hello Hal and Richard. > > I found what is the problem. > > Q: Can I create subinterface with 0 (0x8000)??? > That uses an invalid pkey so this is not a valid configuration. -- Hal > > My Test result. > > partitions.conf > > Default=0x7fff, : ALL=full ; > Net0=0x0000, ipoib : ALL=full ; > Net1=0x0001, ipoib : ALL=full ; > Net2=0x0002, ipoib : ALL=full ; > > 1) Server1 ib.8000 -> Server1 ib.8000 > Ping NG > > 2) Server1 ib.8001 -> Server1 ib.8001 > > The differece is subinterfae number. > > Yesterday, I always tested subinterface 0x8000. > > > > And about previous post, > > The partitions.conf which I posted few hous ago, > was semi colon missing. > > > * partitions.conf is the following > > Default=0x7fff, ipoib : ALL=full > > Net0=0x0001, ipoib : ALL=full > > Should be > > Default=0x7fff, ipoib : ALL=full ; > Net0=0x0001, ipoib : ALL=full ; > > Thank you again. > > > 2011/10/27 Hiroyuki Sato <hiroys...@gmail.com>: > > Hello Hal > > > > Thank you for your information. > > > > I simplified my test environment > > The environment is the folloing. > > > > I'm not sure what is wrong Test2 (Partition test) > > And Could you please tell me how to check the problem?? > > (tool, logfile, etc.. ) > > > > * Environment > > > > OS: Scientific Linux6.1 > > OFED: 1.5.3. > > > > * Diagram > > > > +----------+ib0 ib0+----------+ > > | Server1 |------------------| Server2 | > > +----------+ +----------+ > > > > > > > > > > 1) Test1 (Simple IPoIB no partition Ping test) > > > > Server1 > > > > a) /sbin/ifconfig ib0 inet 192.168.1.1/24 > > b) /sbn/service opensmd start > > 3) /sbin/service opensmd start > > * no /etc/opensm/partitions.conf > > > > Server2 > > /sbin/ifconfig ib0 inet 192.168.1.1/24 > > > > Test from server1(192.168.1.1) to server2(192.168.1.2) > > ping 192.168.1.2 > > The result was OK. > > > > 2) Test2 (Add partition by hand and Ping test) > > > > reboot Server 1 and Server 2 > > > > Server 1 > > 1) echo 0x8001 > /sys/class/net/ib0/create_child > > 2) /sbin/ifconfig ib0.8001 inet 192.168.0.1/24 > > 3) /sbin/service opensmd start > > > > * partitions.conf is the following > > Default=0x7fff, ipoib : ALL=full > > Net0=0x0001, ipoib : ALL=full > > > > Server2 > > 1) echo 0x8001 > /sys/class/net/ib0/create_child > > 2) /sbin/ifconfig ib0.8001 inet 192.168.0.2/24 > > > > Test from server1(ib0.8001/192.168.0.1) to server2(ib0.8001/192.168.0.2 > ) > > > > ping 192.168.0.2 > > no response. > > > > 3) Test2 Log > > > > I enabled debug_log parameter in ib_ipoib modules. > > > > cd /sys/module/ib_ipoib/parameters > > # cat debug_level > > 1 > > > > Could you please tell me what is wrong?? > > > > ib0: enabling connected mode will cause multicast packet drops > > ib0: mtu > 2044 will cause multicast packet drops. > > ib0: mtu > 2044 will cause multicast packet drops. > > ib1: enabling connected mode will cause multicast packet drops > > ib1: mtu > 2044 will cause multicast packet drops. > > ib1: mtu > 2044 will cause multicast packet drops. > > ib0: Event 17 on device mthca0 port 1 > > ib0: Not flushing - IPOIB_FLAG_INITIALIZED not set. > > ib0: Event 11 on device mthca0 port 1 > > ib0: Not flushing - IPOIB_FLAG_INITIALIZED not set. > > ib0: Event 9 on device mthca0 port 1 > > ib0: Not flushing - IPOIB_FLAG_INITIALIZED not set. > > ib0.8001: max_srq_sge=27 > > ib0.8001: max_cm_mtu = 0xfff0, num_frags=16 > > ib0.8001: bringing up interface > > ib0.8001: IPOIB_FLAG_OPER_UP not set > > ib0.8001: IPOIB_FLAG_OPER_UP not set > > ADDRCONF(NETDEV_UP): ib0.8001: link is not ready > > ib0.8001: IPOIB_FLAG_OPER_UP not set > > > > regards. > > > > 2011/10/27 Hal Rosenstock <hal.rosenst...@gmail.com>: > >> > >> > >> On Wed, Oct 26, 2011 at 10:03 AM, Hiroyuki Sato <hiroys...@gmail.com> > wrote: > >>> > >>> Dear members. > >>> > >>> I have some question about Infiniband Partitions. > >>> > >>> I would like to build Linux box with IPoIB Router. > >>> > >>> * Questions > >>> > >>> My Question is the following. > >>> > >>> a) Can I create the following IPoIB network ?? > >>> > >>> b) If so, how to setup /etc/opensm/partitions.conf?? > >>> > >>> * Problem. > >>> > >>> Ping IB Client to Router : I got the followig error. > >>> > >>> Network is unreachable. > >>> > >>> * Environment > >>> > >>> OFED: OFED-1.5.3.2 > >>> OS: Scientific Linux 6.1 > >>> > >>> * Linux box (Router) > >>> > >>> 1 ethernet interface (eth0) > >>> 1 Infiniband interfae (ib0) > >>> Run subnet manager. > >>> > >>> * Ethernet setup > >>> > >>> a) add VLAN ID 120 eth0.120 (192.168.120.0/24) > >>> b) add VLAN ID 130 eth0.130 (192.168.130.0/24) > >>> > >>> * Infiniband Setup > >>> a) add subinterface 0 ib0.8000 (192.168.0.0/24) > >>> b) add subinterface 1 ib0.8001 (192.168.1.0/24) > >>> c) Run opensm > >>> > >>> > >>> I crated subinterface with the following command. > >>> > >>> echo 0 > /sys/class/net/ib0/create_child > >>> echo 1 > /sys/class/net/ib0/create_child > >> > >> > >> echo 0x8001 > /sys/class/net/ib0/create_child > >> echo 0x8002 > /sys/class/net/ib0/create_child > >> > >>> > >>> and assign ip address with ifconfig > >>> > >>> * Logical Diagram > >>> > >>> +----------- 192.168.120.0/24 --- > >>> | > >>> | +---- 192.168.130.0/24 > >>> eth0.120| | eth0.130 > >>> +----------+ > >>> | Router | > >>> +----------+ > >>> ib0.8000| |ib0.8001 > >>> 192.168.0.1 | +---- 192.168.1.0/24 > >>> | > >>> +----------- 192.168.0.0/24 --- IB Cllient. (ib0.8000) > >>> (192.168.0.2) > >>> > >>> * /etc/opensm/partitions.conf > >>> > >>> Default=0x7fff : ALL=full > >>> Net0=0x00, ipoib : ALL=full > >>> Net1=0x01, ipoib : ALL=full > >> > >> > >> Default=0x7fff: ALL=full > >> Net0=0x0001, ipoib: ALL=full > >> Net1=0x0002, ipoib: ALL=full > >> > >> > >>> > >>> Coule you please tell me what is wrong?? > >> > >> > >> The one thing in the above I'm not sure of is whether the default > partition > >> also needs the ipoib flag above so you might need to add that too if the > >> above doesn't work. > >> > >> -- Hal > >> > >>> > >>> Sincerely. > >>> > >>> -- > >>> Hiroyuki Sato. > >>> _______________________________________________ > >>> ewg mailing list > >>> ewg@lists.openfabrics.org > >>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg > >> > >> > > > > > > > > -- > > Hiroyuki Sato > > > > > > -- > Hiroyuki Sato >
_______________________________________________ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
