> Am 30.05.2017 um 17:19 schrieb Roberto Nunnari <[email protected]>:
>
> Hi Reuti.
>
>
> On 28.05.2017 19:56, Reuti wrote:
>> Hi,
>> Am 26.05.2017 um 17:16 schrieb Roberto Nunnari:
>>> Hello.
>>>
>>> I need to upgrade our cluster HPC and would like to hear from the experts
>>> the caveats and thinks to keep in mind.
>>>
>>> For instance, I heard that at a certain point openmpi (after 1.5.4)
>> Unless you are the only user on a node, it's usually advisable to turn off
>> the automatic core binding in Open MPI: "--bind-to-none" or "--bind-to none"
>> (please check the man page for your version). The latter one applies for
>> 2.1.1.
>> The rationale is, that otherwise each Open MPI job doesn't know anything
>> about any other one, and they may bind to the same cores (both counting from
>> core zero on). Some cores would be oversubscribed, others are idling.
>> The Tight Integration of Open MPI into SGE was not automatically detected
>> with 2.0.x and 2.1.0, but 2.1.1 is fine again. Workarounds exist to get the
>> broken ones functional again.
>
> Good. I'll deinstall the CentOS distribution version and install 2.1.1 trying
> to build from source.
>
>
>>> introduced some changes that cause incompatibilities with my present
>>> version of OGE...
>>>
>>> At present:
>>> queuing system: Oracle Grid Engine 6.2u7
>>> OS: CentOS 6.4
>>> openmpi: 1.5.4
>>> Parallel Envs: smp(only on one node) and orte(can span multiple nodes)
>>>
>>> After upgrade:
>>> queuing system: Son of Grid Engine 8.1.9
>>> OS: CentOS 7.3
>>> openmpi: 1.10.3
>> Why 1.10.3? Nowadays I would jump to 2.1.1. It's even straight forward to
>> compile it and put it in a place of your choice, like /usr/local or
>> /opt/local to avoid to mix up with any library from the distribution.
>
> Because it's what comes with CentOS 7.3
Yeah, the distributions sometimes cover not the most recent version – for
understandable reasons.
>
>>> Parallel Envs: smp(only on one node) and orte(can span multiple nodes)
>>>
>>> I never worked with Son of Grid Engine before.. I'll need to build it all..
>>> then import the configuration from OGE.. and make sure integration with
>>> openmpi is working right..
>> There is a tool to save and load the configuration. But as you are new to
>> SGE, I think the best could be to save the text files, but implement all the
>> settings by hand. Then you know exactly what was changed where (i.e.: not to
>> simply load the configuration, but check the differences compared to the
>> default, and then make the necessary adjustments). But the differences
>> between OGE and SGE are not huge.
>> $SGE_ROOT/util/upgrade_modules/{load_sge_config.sh,save_sge_config.sh}
>
> Yes.. I already used that tools in december.. but I could not import back
> everything into GE2011.11p1 (tomorrow I'll try with SoGE 8.1.9).. As a wild
> guess I think it was because I had it compiled without berkeleydb while the
> OGE was with berkeleydb..
I don't think so. The saved files are plain text files and that's independent
how they are finally stored in an SGE installation. But often you have to
import the configuration twice or even tripple due to mutual dependencies.
I.e.: you can not enter a subordinated queue before she is created. Even worse
in case both depend on each other. Then you can do it only by hand.
1. Define all.q
2. Define extra.q with subordinate setting all.q
3. Adjust all.q to have a subordinate setting for extra.q
(In case you want to have only one queue at a time running on an exechosts).
Therefore my suggestion: read the saved settings manually and define all inside
by hand.
-- Reuti
> Anyways.. I built Soge 8.1.9 with
> cd source
> sh scripts/bootstrap.sh && ./aimk
>
> and now? how will I install that? it scattered the files in several places
> under source.. Is there a make install?
_______________________________________________
users mailing list
[email protected]
https://gridengine.org/mailman/listinfo/users
