Richard Chatterton created CLOUDSTACK-4770: ----------------------------------------------
Summary: Management server fails to start with "Unable to get the management server node i" due to downed interface with no MAC address Key: CLOUDSTACK-4770 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4770 Project: CloudStack Issue Type: Bug Security Level: Public (Anyone can view this level - this is the default.) Components: Management Server Affects Versions: 4.1.1, 4.2.0 Environment: CentOS 6.4 Reporter: Richard Chatterton Priority: Minor Installing CloudStack 4.1.1 today, the management server failed to start with the following errors: {noformat} 2013-09-30 14:43:43,436 INFO [utils.component.ComponentContext] (Timer-2:null) Running SystemIntegrityChecker managementServerNode 2013-09-30 14:43:43,438 ERROR [utils.component.ComponentContext] (Timer-2:null) System integrity check failed. Refuse to startup {noformat} Due to CLOUDSTACK-4170, there wasn't any useful information provided, so I attempted to reinstall with CloudStack 4.2.0. This produced the following error messages: {noformat} 2013-09-30 14:43:43,436 INFO [utils.component.ComponentContext] (Timer-2:null) Running SystemIntegrityChecker managementServerNode 2013-09-30 14:43:43,438 ERROR [utils.component.ComponentContext] (Timer-2:null) System integrity check failed. Refuse to startup com.cloud.utils.exception.CloudRuntimeException: Unable to get the management server node id at com.cloud.cluster.ManagementServerNode.check(ManagementServerNode.java:46) at com.cloud.utils.component.ComponentContext.initComponentsLifeCycle(ComponentContext.java:90) at com.cloud.servlet.CloudStartupServlet$1.run(CloudStartupServlet.java:54) at java.util.TimerThread.mainLoop(Timer.java:534) at java.util.TimerThread.run(Timer.java:484) {noformat} I found a Chinese mailing list post which mentioned that this might be due to missing MAC address, and recommended testing by running the class directly. This produced the following output for me: {noformat} $ java -classpath /usr/share/cloudstack-management/webapps/client/WEB-INF/lib/cloud-utils-4.2.0.jar com.cloud.utils.net.MacAddress addr in integer is 0 addr in bytes is 0 0 0 0 0 0 addr in char is 00:00:00:00:00:00 {noformat} This was odd to me, as the output of {{ifconfig}} didn't have any MAC addresses that were 00:00:00:00:00:00. Looking into the code for that module, I found that what it appears to be doing is parsing {{ifconfig -a}} and returning the first string that looks like a MAC address. Reviewing the {{ifconfig -a}} output on the server, there was indeed a downed bond interface which was printed first in {{ifconfig -a}} with a MAC address of 00:00:00:00:00:00. When this interface is up, it automatically gets a MAC address from one of its slave physical interfaces. After assigning a dummy IP address to that interface and upping it, it received a MAC address and I was able to start the management server normally. I expect that there may be a better way to determine what the node id for the management server should be, or logic could be implemented to look for another MAC address if the first once it receives in invalid. -- This message was sent by Atlassian JIRA (v6.1#6144)