[Bug 1762555] Re: ipmi-config connection timeout
[Expired for maas (Ubuntu) because there has been no activity for 60 days.] ** Changed in: maas (Ubuntu) Status: Incomplete => Expired -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1762555 Title: ipmi-config connection timeout To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/maas/+bug/1762555/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1762555] Re: ipmi-config connection timeout
Apologies, I thought I responded to this a while ago. After switching the cable to the dedicated IPMI nic, everything seemed to work out without an issue. Kind of sucks that the shared IPMI flakes out but it is what it is. Thanks for your help! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1762555 Title: ipmi-config connection timeout To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/maas/+bug/1762555/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1762555] Re: ipmi-config connection timeout
Andres, Thanks for the quick reply and the info on the MaaS subsystem. A couple things I have noticed: 1. MaaS can power on/off the machine correctly and can detect that the system is either power on/off. 2. I believe the servers I'm trying to boot are using shared nics which seem to flap when the system is either starting up/shutting down, which might explain why the MaaS check might fail. It looks like the BMC deactivates for about a minute: # bash ./ipmi.sh server1 user Mon Apr 9 23:41:52 EDT 2018 server1: ok Mon Apr 9 23:41:52 EDT 2018 server1: on Mon Apr 9 23:41:52 EDT 2018 # bash ./ipmi.sh server1 user Mon Apr 9 23:41:53 EDT 2018 server1: ok Mon Apr 9 23:41:53 EDT 2018 server1: on Mon Apr 9 23:41:53 EDT 2018 # bash ./ipmi.sh server1 user Mon Apr 9 23:41:53 EDT 2018 server1: ok Mon Apr 9 23:41:53 EDT 2018 server1: on Mon Apr 9 23:41:53 EDT 2018 # bash ./ipmi.sh server1 user Mon Apr 9 23:41:59 EDT 2018 server1: connection timeout Mon Apr 9 23:42:19 EDT 2018 server1: connection timeout Mon Apr 9 23:42:39 EDT 2018 # bash ./ipmi.sh server1 user Mon Apr 9 23:42:52 EDT 2018 server1: ok Mon Apr 9 23:43:05 EDT 2018 server1: on Mon Apr 9 23:43:05 EDT 2018 Even just continually sending those same commands it looks like the BMC is locking up/flapping: # for i in `seq 1 100`;do bash ./ipmi.sh server1 user; sleep 3; done Tue Apr 10 00:09:31 EDT 2018 server1: ok Tue Apr 10 00:09:31 EDT 2018 server1: off Tue Apr 10 00:09:31 EDT 2018 Tue Apr 10 00:09:34 EDT 2018 server1: ok Tue Apr 10 00:09:34 EDT 2018 server1: off Tue Apr 10 00:09:35 EDT 2018 Tue Apr 10 00:09:38 EDT 2018 server1: connection timeout Tue Apr 10 00:09:58 EDT 2018 server1: connection timeout Tue Apr 10 00:10:18 EDT 2018 Tue Apr 10 00:10:21 EDT 2018 server1: ok Tue Apr 10 00:10:32 EDT 2018 server1: on Tue Apr 10 00:10:32 EDT 2018 Tue Apr 10 00:10:35 EDT 2018 server1: ok Tue Apr 10 00:10:36 EDT 2018 server1: on Tue Apr 10 00:10:36 EDT 2018 Tue Apr 10 00:10:39 EDT 2018 server1: connection timeout Tue Apr 10 00:10:59 EDT 2018 server1: connection timeout Tue Apr 10 00:11:19 EDT 2018 Tue Apr 10 00:11:22 EDT 2018 server1: connection timeout Tue Apr 10 00:11:42 EDT 2018 server1: on Tue Apr 10 00:11:46 EDT 2018 Tue Apr 10 00:11:49 EDT 2018 server1: ok Tue Apr 10 00:11:49 EDT 2018 server1: on Tue Apr 10 00:11:49 EDT 2018 Tue Apr 10 00:11:52 EDT 2018 server1: ok Tue Apr 10 00:11:52 EDT 2018 server1: on Tue Apr 10 00:11:52 EDT 2018 Tue Apr 10 00:11:55 EDT 2018 server1: connection timeout Tue Apr 10 00:12:15 EDT 2018 server1: connection timeout Tue Apr 10 00:12:36 EDT 2018 Tue Apr 10 00:12:39 EDT 2018 server1: connection timeout Tue Apr 10 00:12:59 EDT 2018 server1: on Tue Apr 10 00:13:02 EDT 2018 And unfortunately, I think this server has the latest BMC firmware. I'll try and use the dedicated BMC NIC and see if that helps. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1762555 Title: ipmi-config connection timeout To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/maas/+bug/1762555/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1762555] Re: ipmi-config connection timeout
Hi Douglas, If MAAS cannot set the boot order to PXE, it doesn't matter, because that's a best-effort and not something that causes a failure. The failure, however, is that your BMC is either reporting that it is failing ti power on, failing to report it was powered on correctly, or not reporting it at all. MAAS does this: 1. MAAS attempts to set the machine to PXE boot. If it fails, it doens't matter, it continues. 2. MAAS tells the machine to power on and checks if it powered on. If it didn't power on, it re-attempts to power on and check if it powered on. MAAS does 2 in an interval of (1, 2, 2, 4, 6, 8, 12) seconds, unless the tool reports there's fatal errors. That said, the times we have typically seen the issues you are reporting, although very few cases, it have been due to a buggy BMC that locks itself up. As such, I would recommend you try by upgrading the firmware. Once that, could you also provide the output of: ipmipower -W opensesspriv -D LAN_2_0 -u -p -h --cycle --on-if-off ipmipower -W opensesspriv -D LAN_2_0 -u -p -h --stat And repeat that, if you can script it to see if your BMC locks or reports an failure? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1762555 Title: ipmi-config connection timeout To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/maas/+bug/1762555/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1762555] Re: ipmi-config connection timeout
** Changed in: maas (Ubuntu) Status: New => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1762555 Title: ipmi-config connection timeout To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/maas/+bug/1762555/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
