Re: opensm with multiple IB subnets

2010-04-22 Thread Justin Clift 

We should really put this on the wiki. :)


On 04/22/2010 12:56 AM, Yevgeny Kliteynik wrote:
snip

OpenSM dumps various files to /var/log and /var/cache/opensm folders.
When you have more than one OpenSM process, they will all dump the
same files, which is probably not a good idea.

To change the output directories, set the OSM_TMP_DIR and
OSM_CACHE_DIR env. variables to some other place.
In addition, you need to make sure that each SM instance
prints its log in a different place. You need to do
something like this:

foreach guid in guid_list
export OSM_TMP_DIR=/tmp/osm_dump_dir${guid}
export OSM_CACHE_DIR=/tmp/osm_dump_dir${guid}
opensm --log_file /tmp/osm_dump_dir${guid}/osm.log -g ${guid} [your
other options]

-- Yevgeny

snip

--
Salasaga  -  Open Source eLearning IDE
  http://www.salasaga.org
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: opensm with multiple IB subnets

2010-04-22 Thread Yevgeny Kliteynik 

On 4/22/2010 5:23 PM, Justin Clift wrote:

We should really put this on the wiki. :)


Good idea :)

-- Yevgeny
 


On 04/22/2010 12:56 AM, Yevgeny Kliteynik wrote:
snip

OpenSM dumps various files to /var/log and /var/cache/opensm folders.
When you have more than one OpenSM process, they will all dump the
same files, which is probably not a good idea.

To change the output directories, set the OSM_TMP_DIR and
OSM_CACHE_DIR env. variables to some other place.
In addition, you need to make sure that each SM instance
prints its log in a different place. You need to do
something like this:

foreach guid in guid_list
export OSM_TMP_DIR=/tmp/osm_dump_dir${guid}
export OSM_CACHE_DIR=/tmp/osm_dump_dir${guid}
opensm --log_file /tmp/osm_dump_dir${guid}/osm.log -g ${guid} [your
other options]

-- Yevgeny

snip



--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: opensm with multiple IB subnets

2010-04-21 Thread Yevgeny Kliteynik 

Ken,

On 4/21/2010 3:07 AM, Ken Teague wrote:

On Tue, Apr 20, 2010 at 2:13 PM, Ken Teaguektea...@pobox.com  wrote:

I have a 17-node cluster and each node has a single IB card that has
2x IB ports (ib0 and ib1).


After doing a little more research, I confirmed that my understanding
of the manual page is correct.  To run opensm for each GUID, I
modified my init script to run a for loop based on the information
returned from ibstat -p.


I added this near the beginning of the script where the other
environment variables are located:
snip
OFA_HOME=/usr/local/sbin
IBSTAT_BIN=${OFA_HOME}/ibstat
IBSTAT_ARG=-p
OPENSM_BIN=${OFA_HOME}/opensm
OPENSM_ARG=-B -g
snip


I replaced the single line which started opensm with this for loop:
for i in `${IBSTAT_BIN} ${IBSTAT_ARG}`
do
 ${OPENSM_BIN} ${OPENSM_ARG} ${i}
done
snip

If anyone has a more elegant way to handle this, I'm open to
suggestions.  Many thanks.


OpenSM dumps various files to /var/log and /var/cache/opensm folders.
When you have more than one OpenSM process, they will all dump the
same files, which is probably not a good idea.

To change the output directories, set the OSM_TMP_DIR and
OSM_CACHE_DIR env. variables to some other place.
In addition, you need to make sure that each SM instance
prints its log in a different place. You need to do
something like this:

foreach guid in guid_list
export OSM_TMP_DIR=/tmp/osm_dump_dir${guid}
export OSM_CACHE_DIR=/tmp/osm_dump_dir${guid}
opensm --log_file /tmp/osm_dump_dir${guid}/osm.log -g ${guid} [your 
other options]

-- Yevgeny


Ken
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html



--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: opensm with multiple IB subnets

2010-04-20 Thread Ken Teague 
On Tue, Apr 20, 2010 at 2:13 PM, Ken Teague ktea...@pobox.com wrote:
 I have a 17-node cluster and each node has a single IB card that has
 2x IB ports (ib0 and ib1).

After doing a little more research, I confirmed that my understanding
of the manual page is correct.  To run opensm for each GUID, I
modified my init script to run a for loop based on the information
returned from ibstat -p.


I added this near the beginning of the script where the other
environment variables are located:
snip
OFA_HOME=/usr/local/sbin
IBSTAT_BIN=${OFA_HOME}/ibstat
IBSTAT_ARG=-p
OPENSM_BIN=${OFA_HOME}/opensm
OPENSM_ARG=-B -g
snip


I replaced the single line which started opensm with this for loop:
for i in `${IBSTAT_BIN} ${IBSTAT_ARG}`
do
${OPENSM_BIN} ${OPENSM_ARG} ${i}
done
snip

If anyone has a more elegant way to handle this, I'm open to
suggestions.  Many thanks.

Ken
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html