[ https://issues.apache.org/jira/browse/TRAFODION-2884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392057#comment-16392057 ]
Gonzalo E Correa edited comment on TRAFODION-2884 at 3/9/18 9:37 PM: --------------------------------------------------------------------- There is no flow chart, but starting a Trafodion instance is as follows: # MPIRUN or supervisord creates the monitor process across the configured nodes. # The monitor process initialization logic determines if it is running in AGENT mode or as a MPI collective ** There are the two code paths in which the monitor processes determine who the cluster members are. That is, the operational view of the cluster composed of monitor processes. ** We are working on making the AGENT mode the only way the monitor processes initialize so that we can remove the MPI collective initialization logic. # The monitor process initialization logic determines if the Name Server is enabled or not. I have added an overview document which provides some pictures of monitor process creation and initialization activities which you can refer to it as I try to answer your other questions. # The monitors processes are created by mpirun in a Python installation and in the future by the supervisord in a Cloudera Manager installation. Who creates the process is not that important. How the monitor processes join together to create a Trafodion cluster is the change which is addressed by the MASTER/non-MASTER functionality. I have added logic in the monitor which is enabled by environment variable that will cause the monitor to run in AGENT mode which uses the MASTER/non-MASTER method of creating a cluster. Meaning non-MASTER monitor processes join the cluster through the MASTER monitor process. The environment variables will be documented as follows: {quote}NOTE: This is a work in process so it may change! Monitor process creator: # MPIRUN - monitor process is created by mpirun # Uncomment SQ_MON_CREATOR when running monitor in AGENT mode #export SQ_MON_CREATOR=MPIRUN Monitor process run mode: # AGENT - monitor process runs in agent mode versus MPI collective # Uncomment the next three environment variables #export SQ_MON_RUN_MODE=AGENT #export MONITOR_COMM_PORT=23399 #export MONITOR_SYNC_PORT=23398 # NAME-SERVER - to disable process replication and enable name-server # Uncomment the next four environment variables #export NAMESERVER_ENABLE=1 #export NS_COMM_PORT=23397 #export NS_SYNC_PORT=23396 #export NS_M2N_COMM_PORT=23395 So a Python installation can set the MPIRUN and AGENT environment variables which will tell the monitor MPIRUN is the creator and it is to run in AGENT mode. In addition, the Name Server logic can be enabled or disabled. This is separate from the monitor executing in AGENT mode. {quote} # The Name Server processes will always behave as if they were in AGENT mode, meaning that the MASTER/non-MASTER method is how they initialize the set. The non-MASTER Name Server processes join the set though the MASTER Name Server Process. # They don't. The monitor processes at initialization time will create a Name Server process as the first 'primitive process if there is a Name Server configured to run in its node and connect to it. Otherwise, the monitor process will select an existing Name Server process to connect to. All monitor processes must connect to a Name Server process when NAMESERVER_ENABLE=1; otherwise, the monitor will terminate with an error. was (Author: zcorrea): There is no flow chart, but starting a Trafodion instance is as follows: # MPIRUN or supervisord creates the monitor process across the configured nodes. # The monitor process initialization logic determines if it is running in AGENT mode or as a MPI collective ** There are the two code paths in which the monitor processes determine who the cluster members are. That is, the operational view of the cluster composed of monitor processes. ** We are working on making the AGENT mode the only way the monitor processes initialize so that we can remove the MPI collective initialization logic. # The monitor process initialization logic determines if the Name Server is enabled or not. I have added an overview document which provides some pictures of monitor process creation and initialization activities which you can refer to it as I try to answer your other questions. # The monitors processes are created by mpirun in a Python installation and in the future by the supervisord in a Cloudera Manager installation. Who creates the process is not that important. How the monitor processes join together to create a Trafodion cluster is the change which is addressed by the MASTER/non-MASTER functionality. I have added logic in the monitor which is enabled by environment variable that will cause the monitor to run in AGENT mode which uses the MASTER/non-MASTER method of creating a cluster. Meaning non-MASTER monitor processes join the cluster through the MASTER monitor process. The environment variables will be documented as follows: {quote}NOTE: This is a work in process so it may change! Monitor process creator: # MPIRUN - monitor process is created by mpirun # Uncomment SQ_MON_CREATOR when running monitor in AGENT mode #export SQ_MON_CREATOR=MPIRUN Monitor process run mode: # AGENT - monitor process runs in agent mode versus MPI collective # Uncomment the next three environment variables #export SQ_MON_RUN_MODE=AGENT #export MONITOR_COMM_PORT=23399 #export MONITOR_SYNC_PORT=23398 # NAME-SERVER - to disable process replication and enable name-server # Uncomment the next four environment variables #export NAMESERVER_ENABLE=1 #export NS_COMM_PORT=23397 #export NS_SYNC_PORT=23396 #export NS_M2N_COMM_PORT=23395 So a Python installation can set the MPIRUN and AGENT environment variables which will tell the monitor MPIRUN is the creator and it is to run in AGENT mode. In addition, the Name Server logic can be enabled or disabled. This is separate from the monitor executing in AGENT mode. {quote} # The Name Server processes will always behave as if they were in AGENT mode, meaning that the MASTER/non-MASTER method is how they initialize the set. The non-MASTER Name Server processes join the set though the MASTER Name Server Process. # They don't. The monitor processes at initialization time will create a Name Server process as the first 'primitive process if there is a Name Server configured to run in its node and connect to it. Otherwise, the monitor process will select an existing Name Server process to connect to. All monitor processes must connect to a Name Server process when NAMESERVER_ENABLE=1; otherwise, the monitor will terminate with an error. > Trafodion Foundation Scalability Enhancements > --------------------------------------------- > > Key: TRAFODION-2884 > URL: https://issues.apache.org/jira/browse/TRAFODION-2884 > Project: Apache Trafodion > Issue Type: Improvement > Components: dtm, foundation, installer > Affects Versions: 2.3 > Reporter: Gonzalo E Correa > Assignee: Gonzalo E Correa > Priority: Major > Fix For: 2.3 > > Attachments: > TRAFODION-2884-Scalable_Name_Space-Architecure-review.v2.2.pdf, > TRAFODION-2884-Scalable_Name_Space-Architecure.v1.0.pdf, > TRAFODION-2884-Scalable_Name_Space-Architecure.v1.1.pdf, > TRAFODION-2884-Scalable_Name_Space-Architecure.v2.0.pdf, > TRAFODION-2884-Scalable_Name_Space-Architecure.v2.1.pdf, > TRAFODION-2884-Scalable_Name_Space-Architecure.v2.2.pdf, > TRAFODION-2884-Scalable_Name_Space-DesignNotes-review.v1.0.pdf, > TRAFODION-2884-Scalable_Name_Space-DesignNotes.v0.1.pdf, > TRAFODION-2884-Scalable_Name_Space-DesignNotes.v1.0.pdf, > TRAFODION-2884-Scalable_Name_Space-Overview-20180308.pptx > > > Architectural changes are needed in the Trafodion Foundation components to > effectively scale above 256 servers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)