Hi,
I noticed that specifying the receive_queues through an mca param (-mca
btl_openib_receive_queues ....) doesn't always override the
mca-btl-openib-device-params.ini setting.
If for whatever reason we want to bypass the
mca-btl-openib-device-params.ini file setting for the receive_queues, we
should be able to specify a value through an mca param.
But if the string provided in the mca param is the same as the default
one (default_qps in btl_openib_register_mca_params()), this does not
work: we still get the receive_queues from the .ini file.
This is due to the way the
mca_btl_openib_component.receive_queues_source (where did we get the
receive_queues value from) is computed:
1) in btl_openib_register_mca_params() we register
btl_openib_receive_queues, providing default_qps as a default
value.
2) mca_btl_openib_component.receive_queues_source is set to
BTL_OPENIB_RQ_SOURCE_MCA only if the registered string
is different from default_qps
(if both strings are equal, the source is set to
BTL_OPENIB_RQ_SOURCE_DEFAULT).
3) then, in init_one_device(),
mca_btl_openib_component.receive_queues_source is checked:
. if its value is BTL_OPENIB_RQ_SOURCE_MCA, we bypass any other
setting (this is the behaviour I expected)
. otherwise, we go on, getting the .ini file settings (this is the
behaviour I got)
I wanted to know if this behaviour is intentional and the reason for it.
If ever it is not, the attached trivial patch fixes it.
Regards,
--
Nadia Derbey
# HG changeset patch
# Parent 4cb09323aca44faec7d027586ffa94e7d9681989
btl/openib: when specifying the receive_queues as an mca param to bypass the XRC settings, the XRC settings in the .ini file are taken into account nevertheless if we use the default QPs value
diff -r 4cb09323aca4 ompi/mca/btl/openib/btl_openib_component.c
--- a/ompi/mca/btl/openib/btl_openib_component.c Fri Jul 11 05:05:19 2014 +0000
+++ b/ompi/mca/btl/openib/btl_openib_component.c Fri Jul 11 11:46:56 2014 +0200
@@ -268,6 +268,17 @@ static int btl_openib_component_close(vo
ompi_btl_openib_fd_finalize();
ompi_btl_openib_ini_finalize();
+ if (NULL != mca_btl_openib_component.receive_queues
+ && BTL_OPENIB_RQ_SOURCE_DEFAULT ==
+ mca_btl_openib_component.receive_queues_source) {
+ /*
+ * In that case, the string has not been duplicated during variable
+ * registration. So it won't be freed by the mca_base_var system.
+ * Free it here.
+ */
+ free(mca_btl_openib_component.receive_queues);
+ }
+
if (NULL != mca_btl_openib_component.default_recv_qps) {
free(mca_btl_openib_component.default_recv_qps);
}
diff -r 4cb09323aca4 ompi/mca/btl/openib/btl_openib_mca.c
--- a/ompi/mca/btl/openib/btl_openib_mca.c Fri Jul 11 05:05:19 2014 +0000
+++ b/ompi/mca/btl/openib/btl_openib_mca.c Fri Jul 11 11:46:56 2014 +0200
@@ -661,12 +661,14 @@ int btl_openib_register_mca_params(void)
mca_btl_openib_component.default_recv_qps = default_qps;
CHECK(reg_string("receive_queues", NULL,
"Colon-delimited, comma-delimited list of receive queues: P,4096,8,6,4:P,32768,8,6,4",
- default_qps, &mca_btl_openib_component.receive_queues,
+ NULL, &mca_btl_openib_component.receive_queues,
0));
- mca_btl_openib_component.receive_queues_source =
- (0 == strcmp(default_qps,
- mca_btl_openib_component.receive_queues)) ?
- BTL_OPENIB_RQ_SOURCE_DEFAULT : BTL_OPENIB_RQ_SOURCE_MCA;
+ if (NULL == mca_btl_openib_component.receive_queues) {
+ mca_btl_openib_component.receive_queues = strdup(default_qps);
+ mca_btl_openib_component.receive_queues_source = BTL_OPENIB_RQ_SOURCE_DEFAULT;
+ } else {
+ mca_btl_openib_component.receive_queues_source = BTL_OPENIB_RQ_SOURCE_MCA;
+ }
CHECK(reg_string("if_include", NULL,
"Comma-delimited list of devices/ports to be used (e.g. \"mthca0,mthca1:2\"; empty value means to use all ports found). Mutually exclusive with btl_openib_if_exclude.",
