Re: [stable-ish 9] Dell R815 ipmi(4) attach failure
On Friday, April 06, 2012 2:01:15 pm Doug Ambrisko wrote: Alexander Motin writes: | On 04/06/12 20:12, Doug Ambrisko wrote: | Alexander Motin writes: | | On 04/04/12 21:47, John Baldwin wrote: | | On Wednesday, April 04, 2012 12:24:33 pm Doug Ambrisko wrote: | | John Baldwin writes: | | | On Tuesday, April 03, 2012 12:37:50 pm Doug Ambrisko wrote: | | | John Baldwin writes: | | | | On Monday, April 02, 2012 7:27:13 pm Doug Ambrisko wrote: | | | | Doug Ambrisko writes: | | | | | John Baldwin writes: | | | | | | On Saturday, March 31, 2012 3:25:48 pm Doug Ambrisko wrote: | | | | | | Sean Bruno writes: | | | | | | | Noting a failure to attach to the onboard IPMI controller | | with | | | this | | | | dell | | | | | | | R815. Not sure what to start poking at and thought I'd | | though | | | this | | | | over | | | | | | | here for comment. | | | | | | | | | | | | | | -bash-4.2$ dmesg |grep ipmi | | | | | | | ipmi0: KCS mode found at io 0xca8 on acpi | | | | | | | ipmi1:IPMI System Interface on isa0 | | | | | | | device_attach: ipmi1 attach returned 16 | | | | | | | ipmi1:IPMI System Interface on isa0 | | | | | | | device_attach: ipmi1 attach returned 16 | | | | | | | ipmi0: Timed out waiting for GET_DEVICE_ID | | | | | | | | | | | | I've run into this recently. A quick hack to fix it is: | | | | | | | | | | | | Index: ipmi.c | | | | | | |[snip] | | | If you use -ct then you get a file you can feed into schedgraph. | | | However, just reading the log, it seems that IRQ 20 keeps preempting | | | the KCS worker thread preventing it from getting anything done. Also, | | | there seem to be a lot of threads on CPU 0's runqueue waiting for a | | | chance to run (load average of 12 or 13 the entire time). You can try | | | just bumping up the max timeout from 3 seconds to higher perhaps. Not | | | sure why IRQ 20 keeps firing though. It might be related to USB, so | | | you could try fiddling with USB options in the BIOS perhaps, or disabling | | | the USB drivers to see if that fixes IPMI. | | | | Tried without USB in kernel: | | http://people.freebsd.org/~ambrisko/ipmi_ktr_dump_no_usb.txt | | | | Hmm, it's still just running constantly (note that the idle thread is | | _never_ scheduled). The lion's share of the time seems to be spent in | | xpt_thrd. Note that there are several places where nothing happens except | | that xpt_thrd runs constantly (spinning) during 10's of statclock ticks. I | | would maybe start debugging that to see what in the world it is doing. Maybe | | it is polling some hardware down in xpt_action() (i.e., xpt_action() for a | | single bus called down into a driver and it is just spinning using polling | | instead of sleeping and waiting for an interrupt). | | | | xpt_thrd is a bus scanner thread. It is scheduled by CAM for every bus | | on attach and by controller driver on hot-plug events. For some | | controllers it may be quite CPU-hungry. For example, for legacy ATA | | controllers, where bus reset may take many seconds of hardware polling, | | while devices just spinning up. For ahci(4) it was improved about year | | ago to not use polling when possible, but it still may loop for some | | time if controller is not responding on reset. What mfi(4), mentioned in | | log, does during scanning, I am not sure. | | I thought that mfi(4) could be an issue. There are some ata controllers | with nothing attached. I built a GENERIC with USB and mfi commented out | and then the timeout issue went away: | ipmi0: KCS mode found at io 0xca8 on acpi | ipmi1:IPMI System Interface on isa0 | device_attach: ipmi1 attach returned 16 | ipmi1:IPMI System Interface on isa0 | device_attach: ipmi1 attach returned 16 | ipmi0: DEBUG ipmi_submit_driver_request 551 before msleep 1 | ipmi0: DEBUG ipmi_complete_request 527 before wakeup 2211 | ipmi0: DEBUG ipmi_complete_request 529 after wakeup 2272 | ipmi0: DEBUG ipmi_submit_driver_request 553 after msleep 2332 | ipmi0: IPMI device rev. 0, firmware rev. 1.61, version 2.0 | | Without mfi and with USB and it had issues: | ipmi0: KCS mode found at io 0xca8 on acpi | ipmi1:IPMI System Interface on isa0 | device_attach: ipmi1 attach returned 16 | ipmi1:IPMI System Interface on isa0 | device_attach: ipmi1 attach returned 16 | ipmi0: DEBUG ipmi_submit_driver_request 551 before msleep 2 | ipmi0: DEBUG ipmi_complete_request 527 before wakeup 3137 | ipmi0: DEBUG ipmi_complete_request 529 after wakeup 3199 | ipmi0: DEBUG ipmi_submit_driver_request 553 after msleep 3259 | ipmi0: Timed out waiting for GET_DEVICE_ID | ipmi0:
Re: [stable-ish 9] Dell R815 ipmi(4) attach failure
On 04/04/12 21:47, John Baldwin wrote:
On Wednesday, April 04, 2012 12:24:33 pm Doug Ambrisko wrote:
John Baldwin writes:
| On Tuesday, April 03, 2012 12:37:50 pm Doug Ambrisko wrote:
| John Baldwin writes:
| | On Monday, April 02, 2012 7:27:13 pm Doug Ambrisko wrote:
| | Doug Ambrisko writes:
| | | John Baldwin writes:
| | | | On Saturday, March 31, 2012 3:25:48 pm Doug Ambrisko wrote:
| | | | Sean Bruno writes:
| | | | | Noting a failure to attach to the onboard IPMI controller
with
| this
| | dell
| | | | | R815. Not sure what to start poking at and thought I'd
though
| this
| | over
| | | | | here for comment.
| | | | |
| | | | | -bash-4.2$ dmesg |grep ipmi
| | | | | ipmi0: KCS mode found at io 0xca8 on acpi
| | | | | ipmi1:IPMI System Interface on isa0
| | | | | device_attach: ipmi1 attach returned 16
| | | | | ipmi1:IPMI System Interface on isa0
| | | | | device_attach: ipmi1 attach returned 16
| | | | | ipmi0: Timed out waiting for GET_DEVICE_ID
| | | |
| | | | I've run into this recently. A quick hack to fix it is:
| | | |
| | | | Index: ipmi.c
| | | |
| ===
| | | | RCS file: /cvs/src/sys/dev/ipmi/ipmi.c,v
| | | | retrieving revision 1.14
| | | | diff -u -p -r1.14 ipmi.c
| | | | --- ipmi.c 14 Apr 2011 07:14:22 - 1.14
| | | | +++ ipmi.c 31 Mar 2012 19:18:35 -
| | | | @@ -695,7 +695,6 @@ ipmi_startup(void *arg)
| | | | if (error == EWOULDBLOCK) {
| | | | device_printf(dev, Timed out waiting for
| GET_DEVICE_ID\n);
| | | | ipmi_free_request(req);
| | | | -return;
| | | | } else if (error) {
| | | | device_printf(dev, Failed GET_DEVICE_ID: %d\n,
error);
| | | | ipmi_free_request(req);
| | | |
| | | | The issue is that the wakeup doesn't actually wake up the
msleep
| | | | in ipmi_submit_driver_request. The error being reported is
that
| | | | the msleep timed out. This doesn't seem to be critical
problem
| | | | since after this things seemed to work work. I saw this on
9.X.
| | | | Haven't seen it on 8.2. Not sure about -current.
| | | |
| | | | It doesn't happen on all machines.
| | | |
| | | | Hmm, are you seeing the KCS thread manage the request but the
| wakeup()
| | is
| | | | lost?
| | |
| | | It was a couple of weeks ago that I played with it. I put
printf's
| | | around the msleep and wakeup. I saw the wakeup called but the
sleep
| | | not get it. I can try the test again later today. Right now my
main
| | | work machine is recovering from a power outage. This was with 9.0
| | | when I first saw it. This issue seems to only happen at boot
time.
| | | If I kldload the module after the system is booted then it seems
to
| work
| | | okay. The KCS part was working fine and got the data okay from
the
| | | request. I haven't seen or heard any issues with 8.2.
| |
| | With -current I patched ipmi.c with:
| | Index: ipmi.c
| | ===
| | --- ipmi.c (revision 233806)
| | +++ ipmi.c (working copy)
| | @@ -523,7 +523,11 @@
| | * waiter that we awaken.
| | */
| | if (req-ir_owner == NULL)
| | +{
| | +device_printf(sc-ipmi_dev, DEBUG %s %d before wakeup
| | %d\n,__FUNCTION__,__LINE__,ticks);
| | wakeup(req);
| | +device_printf(sc-ipmi_dev, DEBUG %s %d after wakeup
| | %d\n,__FUNCTION__,__LINE__,ticks);
| | +}
| | else {
| | dev = req-ir_owner;
| | TAILQ_INSERT_TAIL(dev-ipmi_completed_requests,
req,
| | ir_link);
| | @@ -543,7 +547,11 @@
| | IPMI_LOCK(sc);
| | error = sc-ipmi_enqueue_request(sc, req);
| | if (error == 0)
| | +{
| | +device_printf(sc-ipmi_dev, DEBUG %s %d before msleep
| | %d\n,__FUNCTION__,__LINE__,ticks);
| | error = msleep(req,sc-ipmi_lock, 0, ipmireq,
timo);
| | +device_printf(sc-ipmi_dev, DEBUG %s %d after msleep
| | %d\n,__FUNCTION__,__LINE__,ticks);
| | +}
| | if (error == 0)
| | error = req-ir_error;
| | IPMI_UNLOCK(sc);
| | @@ -695,8 +703,11 @@
| | error = ipmi_submit_driver_request(sc, req, MAX_TIMEOUT);
| | if (error == EWOULDBLOCK) {
| | device_printf(dev, Timed out waiting for
| GET_DEVICE_ID\n);
| | + printf(DJA\n);
| | +/*
| | ipmi_free_request(req);
| | return;
| | +*/
| | } else if (error) {
| | device_printf(dev, Failed GET_DEVICE_ID: %d\n,
error);
| | ipmi_free_request(req);
| |
| | and get
| |# dmesg | grep ipmi
| |ipmi0: KCS mode found at io 0xca8 on acpi
Re: [stable-ish 9] Dell R815 ipmi(4) attach failure
Alexander Motin writes: [ Charset ISO-8859-1 unsupported, converting... ] | On 04/04/12 21:47, John Baldwin wrote: | On Wednesday, April 04, 2012 12:24:33 pm Doug Ambrisko wrote: | John Baldwin writes: | | On Tuesday, April 03, 2012 12:37:50 pm Doug Ambrisko wrote: | | John Baldwin writes: | | | On Monday, April 02, 2012 7:27:13 pm Doug Ambrisko wrote: | | | Doug Ambrisko writes: | | | | John Baldwin writes: | | | | | On Saturday, March 31, 2012 3:25:48 pm Doug Ambrisko wrote: | | | | | Sean Bruno writes: | | | | | | Noting a failure to attach to the onboard IPMI controller | with | | this | | | dell | | | | | | R815. Not sure what to start poking at and thought I'd | though | | this | | | over | | | | | | here for comment. | | | | | | | | | | | | -bash-4.2$ dmesg |grep ipmi | | | | | | ipmi0: KCS mode found at io 0xca8 on acpi | | | | | | ipmi1:IPMI System Interface on isa0 | | | | | | device_attach: ipmi1 attach returned 16 | | | | | | ipmi1:IPMI System Interface on isa0 | | | | | | device_attach: ipmi1 attach returned 16 | | | | | | ipmi0: Timed out waiting for GET_DEVICE_ID | | | | | | | | | | I've run into this recently. A quick hack to fix it is: | | | | | | | | | | Index: ipmi.c | | | | | [snip] | | If you use -ct then you get a file you can feed into schedgraph. | | However, just reading the log, it seems that IRQ 20 keeps preempting | | the KCS worker thread preventing it from getting anything done. Also, | | there seem to be a lot of threads on CPU 0's runqueue waiting for a | | chance to run (load average of 12 or 13 the entire time). You can try | | just bumping up the max timeout from 3 seconds to higher perhaps. Not | | sure why IRQ 20 keeps firing though. It might be related to USB, so | | you could try fiddling with USB options in the BIOS perhaps, or disabling | | the USB drivers to see if that fixes IPMI. | | Tried without USB in kernel: | http://people.freebsd.org/~ambrisko/ipmi_ktr_dump_no_usb.txt | | Hmm, it's still just running constantly (note that the idle thread is | _never_ scheduled). The lion's share of the time seems to be spent in | xpt_thrd. Note that there are several places where nothing happens except | that xpt_thrd runs constantly (spinning) during 10's of statclock ticks. I | would maybe start debugging that to see what in the world it is doing. Maybe | it is polling some hardware down in xpt_action() (i.e., xpt_action() for a | single bus called down into a driver and it is just spinning using polling | instead of sleeping and waiting for an interrupt). | | xpt_thrd is a bus scanner thread. It is scheduled by CAM for every bus | on attach and by controller driver on hot-plug events. For some | controllers it may be quite CPU-hungry. For example, for legacy ATA | controllers, where bus reset may take many seconds of hardware polling, | while devices just spinning up. For ahci(4) it was improved about year | ago to not use polling when possible, but it still may loop for some | time if controller is not responding on reset. What mfi(4), mentioned in | log, does during scanning, I am not sure. I thought that mfi(4) could be an issue. There are some ata controllers with nothing attached. I built a GENERIC with USB and mfi commented out and then the timeout issue went away: ipmi0: KCS mode found at io 0xca8 on acpi ipmi1: IPMI System Interface on isa0 device_attach: ipmi1 attach returned 16 ipmi1: IPMI System Interface on isa0 device_attach: ipmi1 attach returned 16 ipmi0: DEBUG ipmi_submit_driver_request 551 before msleep 1 ipmi0: DEBUG ipmi_complete_request 527 before wakeup 2211 ipmi0: DEBUG ipmi_complete_request 529 after wakeup 2272 ipmi0: DEBUG ipmi_submit_driver_request 553 after msleep 2332 ipmi0: IPMI device rev. 0, firmware rev. 1.61, version 2.0 Without mfi and with USB and it had issues: ipmi0: KCS mode found at io 0xca8 on acpi ipmi1: IPMI System Interface on isa0 device_attach: ipmi1 attach returned 16 ipmi1: IPMI System Interface on isa0 device_attach: ipmi1 attach returned 16 ipmi0: DEBUG ipmi_submit_driver_request 551 before msleep 2 ipmi0: DEBUG ipmi_complete_request 527 before wakeup 3137 ipmi0: DEBUG ipmi_complete_request 529 after wakeup 3199 ipmi0: DEBUG ipmi_submit_driver_request 553 after msleep 3259 ipmi0: Timed out waiting for GET_DEVICE_ID ipmi0: IPMI device rev. 0, firmware rev. 1.61, version 2.0 I can post more ktrdump traces if needed. A 1U Dell machine without mfi also has this problem. As John mentioned it might be good to bump up the timeout from 3s to 6s. I did that with the USB no mfi kernel and that passed: % dmesg | grep ipmi ipmi0: KCS mode found at io 0xca8 on acpi ipmi1: IPMI System Interface on isa0 device_attach: ipmi1 attach returned 16 ipmi1: IPMI System Interface on isa0 device_attach: ipmi1 attach returned 16 ipmi0: DEBUG
Re: [stable-ish 9] Dell R815 ipmi(4) attach failure
On 04/06/12 20:12, Doug Ambrisko wrote: Alexander Motin writes: [ Charset ISO-8859-1 unsupported, converting... ] | On 04/04/12 21:47, John Baldwin wrote: | On Wednesday, April 04, 2012 12:24:33 pm Doug Ambrisko wrote: | John Baldwin writes: | | On Tuesday, April 03, 2012 12:37:50 pm Doug Ambrisko wrote: | | John Baldwin writes: | | | On Monday, April 02, 2012 7:27:13 pm Doug Ambrisko wrote: | | | Doug Ambrisko writes: | | | | John Baldwin writes: | | | | | On Saturday, March 31, 2012 3:25:48 pm Doug Ambrisko wrote: | | | | | Sean Bruno writes: | | | | | | Noting a failure to attach to the onboard IPMI controller | with | | this | | | dell | | | | | | R815. Not sure what to start poking at and thought I'd | though | | this | | | over | | | | | | here for comment. | | | | | | | | | | | | -bash-4.2$ dmesg |grep ipmi | | | | | | ipmi0: KCS mode found at io 0xca8 on acpi | | | | | | ipmi1:IPMI System Interface on isa0 | | | | | | device_attach: ipmi1 attach returned 16 | | | | | | ipmi1:IPMI System Interface on isa0 | | | | | | device_attach: ipmi1 attach returned 16 | | | | | | ipmi0: Timed out waiting for GET_DEVICE_ID | | | | | | | | | | I've run into this recently. A quick hack to fix it is: | | | | | | | | | | Index: ipmi.c | | | | | [snip] | | If you use -ct then you get a file you can feed into schedgraph. | | However, just reading the log, it seems that IRQ 20 keeps preempting | | the KCS worker thread preventing it from getting anything done. Also, | | there seem to be a lot of threads on CPU 0's runqueue waiting for a | | chance to run (load average of 12 or 13 the entire time). You can try | | just bumping up the max timeout from 3 seconds to higher perhaps. Not | | sure why IRQ 20 keeps firing though. It might be related to USB, so | | you could try fiddling with USB options in the BIOS perhaps, or disabling | | the USB drivers to see if that fixes IPMI. | | Tried without USB in kernel: | http://people.freebsd.org/~ambrisko/ipmi_ktr_dump_no_usb.txt | | Hmm, it's still just running constantly (note that the idle thread is | _never_ scheduled). The lion's share of the time seems to be spent in | xpt_thrd. Note that there are several places where nothing happens except | that xpt_thrd runs constantly (spinning) during 10's of statclock ticks. I | would maybe start debugging that to see what in the world it is doing. Maybe | it is polling some hardware down in xpt_action() (i.e., xpt_action() for a | single bus called down into a driver and it is just spinning using polling | instead of sleeping and waiting for an interrupt). | | xpt_thrd is a bus scanner thread. It is scheduled by CAM for every bus | on attach and by controller driver on hot-plug events. For some | controllers it may be quite CPU-hungry. For example, for legacy ATA | controllers, where bus reset may take many seconds of hardware polling, | while devices just spinning up. For ahci(4) it was improved about year | ago to not use polling when possible, but it still may loop for some | time if controller is not responding on reset. What mfi(4), mentioned in | log, does during scanning, I am not sure. I thought that mfi(4) could be an issue. There are some ata controllers with nothing attached. I built a GENERIC with USB and mfi commented out and then the timeout issue went away: ipmi0: KCS mode found at io 0xca8 on acpi ipmi1:IPMI System Interface on isa0 device_attach: ipmi1 attach returned 16 ipmi1:IPMI System Interface on isa0 device_attach: ipmi1 attach returned 16 ipmi0: DEBUG ipmi_submit_driver_request 551 before msleep 1 ipmi0: DEBUG ipmi_complete_request 527 before wakeup 2211 ipmi0: DEBUG ipmi_complete_request 529 after wakeup 2272 ipmi0: DEBUG ipmi_submit_driver_request 553 after msleep 2332 ipmi0: IPMI device rev. 0, firmware rev. 1.61, version 2.0 Without mfi and with USB and it had issues: ipmi0: KCS mode found at io 0xca8 on acpi ipmi1:IPMI System Interface on isa0 device_attach: ipmi1 attach returned 16 ipmi1:IPMI System Interface on isa0 device_attach: ipmi1 attach returned 16 ipmi0: DEBUG ipmi_submit_driver_request 551 before msleep 2 ipmi0: DEBUG ipmi_complete_request 527 before wakeup 3137 ipmi0: DEBUG ipmi_complete_request 529 after wakeup 3199 ipmi0: DEBUG ipmi_submit_driver_request 553 after msleep 3259 ipmi0: Timed out waiting for GET_DEVICE_ID ipmi0: IPMI device rev. 0, firmware rev. 1.61, version 2.0 I can post more ktrdump traces if needed. A 1U Dell machine without mfi also has this problem. As John mentioned it might be good to bump up the timeout from 3s to 6s. I did that with the USB no mfi kernel and that passed: % dmesg | grep ipmi ipmi0: KCS mode found at io 0xca8 on acpi ipmi1:IPMI System Interface on isa0 device_attach:
Re: [stable-ish 9] Dell R815 ipmi(4) attach failure
Alexander Motin writes: | On 04/06/12 20:12, Doug Ambrisko wrote: | Alexander Motin writes: | | On 04/04/12 21:47, John Baldwin wrote: | | On Wednesday, April 04, 2012 12:24:33 pm Doug Ambrisko wrote: | | John Baldwin writes: | | | On Tuesday, April 03, 2012 12:37:50 pm Doug Ambrisko wrote: | | | John Baldwin writes: | | | | On Monday, April 02, 2012 7:27:13 pm Doug Ambrisko wrote: | | | | Doug Ambrisko writes: | | | | | John Baldwin writes: | | | | | | On Saturday, March 31, 2012 3:25:48 pm Doug Ambrisko wrote: | | | | | | Sean Bruno writes: | | | | | | | Noting a failure to attach to the onboard IPMI controller | | with | | | this | | | | dell | | | | | | | R815. Not sure what to start poking at and thought I'd | | though | | | this | | | | over | | | | | | | here for comment. | | | | | | | | | | | | | | -bash-4.2$ dmesg |grep ipmi | | | | | | | ipmi0: KCS mode found at io 0xca8 on acpi | | | | | | | ipmi1:IPMI System Interface on isa0 | | | | | | | device_attach: ipmi1 attach returned 16 | | | | | | | ipmi1:IPMI System Interface on isa0 | | | | | | | device_attach: ipmi1 attach returned 16 | | | | | | | ipmi0: Timed out waiting for GET_DEVICE_ID | | | | | | | | | | | | I've run into this recently. A quick hack to fix it is: | | | | | | | | | | | | Index: ipmi.c | | | | | | | [snip] | | | If you use -ct then you get a file you can feed into schedgraph. | | | However, just reading the log, it seems that IRQ 20 keeps preempting | | | the KCS worker thread preventing it from getting anything done. Also, | | | there seem to be a lot of threads on CPU 0's runqueue waiting for a | | | chance to run (load average of 12 or 13 the entire time). You can try | | | just bumping up the max timeout from 3 seconds to higher perhaps. Not | | | sure why IRQ 20 keeps firing though. It might be related to USB, so | | | you could try fiddling with USB options in the BIOS perhaps, or disabling | | | the USB drivers to see if that fixes IPMI. | | | | Tried without USB in kernel: | | http://people.freebsd.org/~ambrisko/ipmi_ktr_dump_no_usb.txt | | | | Hmm, it's still just running constantly (note that the idle thread is | | _never_ scheduled). The lion's share of the time seems to be spent in | | xpt_thrd. Note that there are several places where nothing happens except | | that xpt_thrd runs constantly (spinning) during 10's of statclock ticks. I | | would maybe start debugging that to see what in the world it is doing. Maybe | | it is polling some hardware down in xpt_action() (i.e., xpt_action() for a | | single bus called down into a driver and it is just spinning using polling | | instead of sleeping and waiting for an interrupt). | | | | xpt_thrd is a bus scanner thread. It is scheduled by CAM for every bus | | on attach and by controller driver on hot-plug events. For some | | controllers it may be quite CPU-hungry. For example, for legacy ATA | | controllers, where bus reset may take many seconds of hardware polling, | | while devices just spinning up. For ahci(4) it was improved about year | | ago to not use polling when possible, but it still may loop for some | | time if controller is not responding on reset. What mfi(4), mentioned in | | log, does during scanning, I am not sure. | | I thought that mfi(4) could be an issue. There are some ata controllers | with nothing attached. I built a GENERIC with USB and mfi commented out | and then the timeout issue went away: | ipmi0: KCS mode found at io 0xca8 on acpi | ipmi1:IPMI System Interface on isa0 | device_attach: ipmi1 attach returned 16 | ipmi1:IPMI System Interface on isa0 | device_attach: ipmi1 attach returned 16 | ipmi0: DEBUG ipmi_submit_driver_request 551 before msleep 1 | ipmi0: DEBUG ipmi_complete_request 527 before wakeup 2211 | ipmi0: DEBUG ipmi_complete_request 529 after wakeup 2272 | ipmi0: DEBUG ipmi_submit_driver_request 553 after msleep 2332 | ipmi0: IPMI device rev. 0, firmware rev. 1.61, version 2.0 | | Without mfi and with USB and it had issues: | ipmi0: KCS mode found at io 0xca8 on acpi | ipmi1:IPMI System Interface on isa0 | device_attach: ipmi1 attach returned 16 | ipmi1:IPMI System Interface on isa0 | device_attach: ipmi1 attach returned 16 | ipmi0: DEBUG ipmi_submit_driver_request 551 before msleep 2 | ipmi0: DEBUG ipmi_complete_request 527 before wakeup 3137 | ipmi0: DEBUG ipmi_complete_request 529 after wakeup 3199 | ipmi0: DEBUG ipmi_submit_driver_request 553 after msleep 3259 | ipmi0: Timed out waiting for GET_DEVICE_ID | ipmi0: IPMI device rev. 0, firmware rev. 1.61, version 2.0 | | I can post more ktrdump traces if needed. A 1U Dell machine without | mfi also has this
Re: [stable-ish 9] Dell R815 ipmi(4) attach failure
John Baldwin writes:
| On Tuesday, April 03, 2012 12:37:50 pm Doug Ambrisko wrote:
| John Baldwin writes:
| | On Monday, April 02, 2012 7:27:13 pm Doug Ambrisko wrote:
| | Doug Ambrisko writes:
| | | John Baldwin writes:
| | | | On Saturday, March 31, 2012 3:25:48 pm Doug Ambrisko wrote:
| | | | Sean Bruno writes:
| | | | | Noting a failure to attach to the onboard IPMI controller with
| this
| | dell
| | | | | R815. Not sure what to start poking at and thought I'd though
| this
| | over
| | | | | here for comment.
| | | | |
| | | | | -bash-4.2$ dmesg |grep ipmi
| | | | | ipmi0: KCS mode found at io 0xca8 on acpi
| | | | | ipmi1: IPMI System Interface on isa0
| | | | | device_attach: ipmi1 attach returned 16
| | | | | ipmi1: IPMI System Interface on isa0
| | | | | device_attach: ipmi1 attach returned 16
| | | | | ipmi0: Timed out waiting for GET_DEVICE_ID
| | | |
| | | | I've run into this recently. A quick hack to fix it is:
| | | |
| | | | Index: ipmi.c
| | | |
| ===
| | | | RCS file: /cvs/src/sys/dev/ipmi/ipmi.c,v
| | | | retrieving revision 1.14
| | | | diff -u -p -r1.14 ipmi.c
| | | | --- ipmi.c14 Apr 2011 07:14:22 - 1.14
| | | | +++ ipmi.c31 Mar 2012 19:18:35 -
| | | | @@ -695,7 +695,6 @@ ipmi_startup(void *arg)
| | | |if (error == EWOULDBLOCK) {
| | | |device_printf(dev, Timed out waiting for
| GET_DEVICE_ID\n);
| | | |ipmi_free_request(req);
| | | | - return;
| | | |} else if (error) {
| | | |device_printf(dev, Failed GET_DEVICE_ID: %d\n, error);
| | | |ipmi_free_request(req);
| | | |
| | | | The issue is that the wakeup doesn't actually wake up the msleep
| | | | in ipmi_submit_driver_request. The error being reported is that
| | | | the msleep timed out. This doesn't seem to be critical problem
| | | | since after this things seemed to work work. I saw this on 9.X.
| | | | Haven't seen it on 8.2. Not sure about -current.
| | | |
| | | | It doesn't happen on all machines.
| | | |
| | | | Hmm, are you seeing the KCS thread manage the request but the
| wakeup()
| | is
| | | | lost?
| | |
| | | It was a couple of weeks ago that I played with it. I put printf's
| | | around the msleep and wakeup. I saw the wakeup called but the sleep
| | | not get it. I can try the test again later today. Right now my main
| | | work machine is recovering from a power outage. This was with 9.0
| | | when I first saw it. This issue seems to only happen at boot time.
| | | If I kldload the module after the system is booted then it seems to
| work
| | | okay. The KCS part was working fine and got the data okay from the
| | | request. I haven't seen or heard any issues with 8.2.
| |
| | With -current I patched ipmi.c with:
| | Index: ipmi.c
| | ===
| | --- ipmi.c (revision 233806)
| | +++ ipmi.c (working copy)
| | @@ -523,7 +523,11 @@
| | * waiter that we awaken.
| | */
| | if (req-ir_owner == NULL)
| | +{
| | +device_printf(sc-ipmi_dev, DEBUG %s %d before wakeup
| | %d\n,__FUNCTION__,__LINE__,ticks);
| | wakeup(req);
| | +device_printf(sc-ipmi_dev, DEBUG %s %d after wakeup
| | %d\n,__FUNCTION__,__LINE__,ticks);
| | +}
| | else {
| | dev = req-ir_owner;
| | TAILQ_INSERT_TAIL(dev-ipmi_completed_requests, req,
| | ir_link);
| | @@ -543,7 +547,11 @@
| | IPMI_LOCK(sc);
| | error = sc-ipmi_enqueue_request(sc, req);
| | if (error == 0)
| | +{
| | +device_printf(sc-ipmi_dev, DEBUG %s %d before msleep
| | %d\n,__FUNCTION__,__LINE__,ticks);
| | error = msleep(req, sc-ipmi_lock, 0, ipmireq, timo);
| | +device_printf(sc-ipmi_dev, DEBUG %s %d after msleep
| | %d\n,__FUNCTION__,__LINE__,ticks);
| | +}
| | if (error == 0)
| | error = req-ir_error;
| | IPMI_UNLOCK(sc);
| | @@ -695,8 +703,11 @@
| | error = ipmi_submit_driver_request(sc, req, MAX_TIMEOUT);
| | if (error == EWOULDBLOCK) {
| | device_printf(dev, Timed out waiting for
| GET_DEVICE_ID\n);
| | + printf(DJA\n);
| | +/*
| | ipmi_free_request(req);
| | return;
| | +*/
| | } else if (error) {
| | device_printf(dev, Failed GET_DEVICE_ID: %d\n, error);
| | ipmi_free_request(req);
| |
| | and get
| |# dmesg | grep ipmi
| |ipmi0: KCS mode found at io 0xca8 on acpi
| |ipmi1: IPMI System Interface on isa0
| |device_attach: ipmi1 attach returned 16
| |ipmi1: IPMI System Interface on isa0
| |
Re: [stable-ish 9] Dell R815 ipmi(4) attach failure
On Wednesday, April 04, 2012 12:24:33 pm Doug Ambrisko wrote:
John Baldwin writes:
| On Tuesday, April 03, 2012 12:37:50 pm Doug Ambrisko wrote:
| John Baldwin writes:
| | On Monday, April 02, 2012 7:27:13 pm Doug Ambrisko wrote:
| | Doug Ambrisko writes:
| | | John Baldwin writes:
| | | | On Saturday, March 31, 2012 3:25:48 pm Doug Ambrisko wrote:
| | | | Sean Bruno writes:
| | | | | Noting a failure to attach to the onboard IPMI controller
with
| this
| | dell
| | | | | R815. Not sure what to start poking at and thought I'd
though
| this
| | over
| | | | | here for comment.
| | | | |
| | | | | -bash-4.2$ dmesg |grep ipmi
| | | | | ipmi0: KCS mode found at io 0xca8 on acpi
| | | | | ipmi1: IPMI System Interface on isa0
| | | | | device_attach: ipmi1 attach returned 16
| | | | | ipmi1: IPMI System Interface on isa0
| | | | | device_attach: ipmi1 attach returned 16
| | | | | ipmi0: Timed out waiting for GET_DEVICE_ID
| | | |
| | | | I've run into this recently. A quick hack to fix it is:
| | | |
| | | | Index: ipmi.c
| | | |
| ===
| | | | RCS file: /cvs/src/sys/dev/ipmi/ipmi.c,v
| | | | retrieving revision 1.14
| | | | diff -u -p -r1.14 ipmi.c
| | | | --- ipmi.c 14 Apr 2011 07:14:22 - 1.14
| | | | +++ ipmi.c 31 Mar 2012 19:18:35 -
| | | | @@ -695,7 +695,6 @@ ipmi_startup(void *arg)
| | | | if (error == EWOULDBLOCK) {
| | | | device_printf(dev, Timed out waiting for
| GET_DEVICE_ID\n);
| | | | ipmi_free_request(req);
| | | | - return;
| | | | } else if (error) {
| | | | device_printf(dev, Failed GET_DEVICE_ID:
%d\n,
error);
| | | | ipmi_free_request(req);
| | | |
| | | | The issue is that the wakeup doesn't actually wake up the
msleep
| | | | in ipmi_submit_driver_request. The error being reported is
that
| | | | the msleep timed out. This doesn't seem to be critical
problem
| | | | since after this things seemed to work work. I saw this on
9.X.
| | | | Haven't seen it on 8.2. Not sure about -current.
| | | |
| | | | It doesn't happen on all machines.
| | | |
| | | | Hmm, are you seeing the KCS thread manage the request but the
| wakeup()
| | is
| | | | lost?
| | |
| | | It was a couple of weeks ago that I played with it. I put
printf's
| | | around the msleep and wakeup. I saw the wakeup called but the
sleep
| | | not get it. I can try the test again later today. Right now my
main
| | | work machine is recovering from a power outage. This was with 9.0
| | | when I first saw it. This issue seems to only happen at boot
time.
| | | If I kldload the module after the system is booted then it seems
to
| work
| | | okay. The KCS part was working fine and got the data okay from
the
| | | request. I haven't seen or heard any issues with 8.2.
| |
| | With -current I patched ipmi.c with:
| | Index: ipmi.c
| | ===
| | --- ipmi.c (revision 233806)
| | +++ ipmi.c (working copy)
| | @@ -523,7 +523,11 @@
| | * waiter that we awaken.
| | */
| | if (req-ir_owner == NULL)
| | +{
| | +device_printf(sc-ipmi_dev, DEBUG %s %d before wakeup
| | %d\n,__FUNCTION__,__LINE__,ticks);
| | wakeup(req);
| | +device_printf(sc-ipmi_dev, DEBUG %s %d after wakeup
| | %d\n,__FUNCTION__,__LINE__,ticks);
| | +}
| | else {
| | dev = req-ir_owner;
| | TAILQ_INSERT_TAIL(dev-ipmi_completed_requests,
req,
| | ir_link);
| | @@ -543,7 +547,11 @@
| | IPMI_LOCK(sc);
| | error = sc-ipmi_enqueue_request(sc, req);
| | if (error == 0)
| | +{
| | +device_printf(sc-ipmi_dev, DEBUG %s %d before msleep
| | %d\n,__FUNCTION__,__LINE__,ticks);
| | error = msleep(req, sc-ipmi_lock, 0, ipmireq,
timo);
| | +device_printf(sc-ipmi_dev, DEBUG %s %d after msleep
| | %d\n,__FUNCTION__,__LINE__,ticks);
| | +}
| | if (error == 0)
| | error = req-ir_error;
| | IPMI_UNLOCK(sc);
| | @@ -695,8 +703,11 @@
| | error = ipmi_submit_driver_request(sc, req, MAX_TIMEOUT);
| | if (error == EWOULDBLOCK) {
| | device_printf(dev, Timed out waiting for
| GET_DEVICE_ID\n);
| | + printf(DJA\n);
| | +/*
| | ipmi_free_request(req);
| | return;
| | +*/
| | } else if (error) {
| | device_printf(dev, Failed GET_DEVICE_ID: %d\n,
error);
| | ipmi_free_request(req);
| |
| |
Re: [stable-ish 9] Dell R815 ipmi(4) attach failure
On Monday, April 02, 2012 7:27:13 pm Doug Ambrisko wrote:
Doug Ambrisko writes:
| John Baldwin writes:
| | On Saturday, March 31, 2012 3:25:48 pm Doug Ambrisko wrote:
| | Sean Bruno writes:
| | | Noting a failure to attach to the onboard IPMI controller with this
dell
| | | R815. Not sure what to start poking at and thought I'd though this
over
| | | here for comment.
| | |
| | | -bash-4.2$ dmesg |grep ipmi
| | | ipmi0: KCS mode found at io 0xca8 on acpi
| | | ipmi1: IPMI System Interface on isa0
| | | device_attach: ipmi1 attach returned 16
| | | ipmi1: IPMI System Interface on isa0
| | | device_attach: ipmi1 attach returned 16
| | | ipmi0: Timed out waiting for GET_DEVICE_ID
| |
| | I've run into this recently. A quick hack to fix it is:
| |
| | Index: ipmi.c
| | ===
| | RCS file: /cvs/src/sys/dev/ipmi/ipmi.c,v
| | retrieving revision 1.14
| | diff -u -p -r1.14 ipmi.c
| | --- ipmi.c 14 Apr 2011 07:14:22 - 1.14
| | +++ ipmi.c 31 Mar 2012 19:18:35 -
| | @@ -695,7 +695,6 @@ ipmi_startup(void *arg)
| | if (error == EWOULDBLOCK) {
| | device_printf(dev, Timed out waiting for
GET_DEVICE_ID\n);
| | ipmi_free_request(req);
| | - return;
| | } else if (error) {
| | device_printf(dev, Failed GET_DEVICE_ID: %d\n, error);
| | ipmi_free_request(req);
| |
| | The issue is that the wakeup doesn't actually wake up the msleep
| | in ipmi_submit_driver_request. The error being reported is that
| | the msleep timed out. This doesn't seem to be critical problem
| | since after this things seemed to work work. I saw this on 9.X.
| | Haven't seen it on 8.2. Not sure about -current.
| |
| | It doesn't happen on all machines.
| |
| | Hmm, are you seeing the KCS thread manage the request but the wakeup()
is
| | lost?
|
| It was a couple of weeks ago that I played with it. I put printf's
| around the msleep and wakeup. I saw the wakeup called but the sleep
| not get it. I can try the test again later today. Right now my main
| work machine is recovering from a power outage. This was with 9.0
| when I first saw it. This issue seems to only happen at boot time.
| If I kldload the module after the system is booted then it seems to work
| okay. The KCS part was working fine and got the data okay from the
| request. I haven't seen or heard any issues with 8.2.
With -current I patched ipmi.c with:
Index: ipmi.c
===
--- ipmi.c (revision 233806)
+++ ipmi.c (working copy)
@@ -523,7 +523,11 @@
* waiter that we awaken.
*/
if (req-ir_owner == NULL)
+{
+device_printf(sc-ipmi_dev, DEBUG %s %d before wakeup
%d\n,__FUNCTION__,__LINE__,ticks);
wakeup(req);
+device_printf(sc-ipmi_dev, DEBUG %s %d after wakeup
%d\n,__FUNCTION__,__LINE__,ticks);
+}
else {
dev = req-ir_owner;
TAILQ_INSERT_TAIL(dev-ipmi_completed_requests, req,
ir_link);
@@ -543,7 +547,11 @@
IPMI_LOCK(sc);
error = sc-ipmi_enqueue_request(sc, req);
if (error == 0)
+{
+device_printf(sc-ipmi_dev, DEBUG %s %d before msleep
%d\n,__FUNCTION__,__LINE__,ticks);
error = msleep(req, sc-ipmi_lock, 0, ipmireq, timo);
+device_printf(sc-ipmi_dev, DEBUG %s %d after msleep
%d\n,__FUNCTION__,__LINE__,ticks);
+}
if (error == 0)
error = req-ir_error;
IPMI_UNLOCK(sc);
@@ -695,8 +703,11 @@
error = ipmi_submit_driver_request(sc, req, MAX_TIMEOUT);
if (error == EWOULDBLOCK) {
device_printf(dev, Timed out waiting for GET_DEVICE_ID\n);
+ printf(DJA\n);
+/*
ipmi_free_request(req);
return;
+*/
} else if (error) {
device_printf(dev, Failed GET_DEVICE_ID: %d\n, error);
ipmi_free_request(req);
and get
# dmesg | grep ipmi
ipmi0: KCS mode found at io 0xca8 on acpi
ipmi1: IPMI System Interface on isa0
device_attach: ipmi1 attach returned 16
ipmi1: IPMI System Interface on isa0
device_attach: ipmi1 attach returned 16
ipmi0: DEBUG ipmi_submit_driver_request 551 before msleep 2
ipmi0: DEBUG ipmi_complete_request 527 before wakeup 6201
ipmi0: DEBUG ipmi_complete_request 529 after wakeup 6263
ipmi0: DEBUG ipmi_submit_driver_request 553 after msleep 6323
Actually, can you compile with:
options KTR
options KTR_COMPILE=KTR_SCHED
options KTR_MASK=KTR_SCHED
and then add a temporary hack to ipmi.c to set ktr_mask to 0 after
ipmi_submit_driver_request() returns in ipmi_startup()? You can
then use 'ktrdump -ct' after boot to capture a log of what the scheduler
did including if it timed
Re: [stable-ish 9] Dell R815 ipmi(4) attach failure
John Baldwin writes:
| On Monday, April 02, 2012 7:27:13 pm Doug Ambrisko wrote:
| Doug Ambrisko writes:
| | John Baldwin writes:
| | | On Saturday, March 31, 2012 3:25:48 pm Doug Ambrisko wrote:
| | | Sean Bruno writes:
| | | | Noting a failure to attach to the onboard IPMI controller with this
| dell
| | | | R815. Not sure what to start poking at and thought I'd though this
| over
| | | | here for comment.
| | | |
| | | | -bash-4.2$ dmesg |grep ipmi
| | | | ipmi0: KCS mode found at io 0xca8 on acpi
| | | | ipmi1: IPMI System Interface on isa0
| | | | device_attach: ipmi1 attach returned 16
| | | | ipmi1: IPMI System Interface on isa0
| | | | device_attach: ipmi1 attach returned 16
| | | | ipmi0: Timed out waiting for GET_DEVICE_ID
| | |
| | | I've run into this recently. A quick hack to fix it is:
| | |
| | | Index: ipmi.c
| | | ===
| | | RCS file: /cvs/src/sys/dev/ipmi/ipmi.c,v
| | | retrieving revision 1.14
| | | diff -u -p -r1.14 ipmi.c
| | | --- ipmi.c14 Apr 2011 07:14:22 - 1.14
| | | +++ ipmi.c31 Mar 2012 19:18:35 -
| | | @@ -695,7 +695,6 @@ ipmi_startup(void *arg)
| | |if (error == EWOULDBLOCK) {
| | |device_printf(dev, Timed out waiting for
GET_DEVICE_ID\n);
| | |ipmi_free_request(req);
| | | - return;
| | |} else if (error) {
| | |device_printf(dev, Failed GET_DEVICE_ID: %d\n, error);
| | |ipmi_free_request(req);
| | |
| | | The issue is that the wakeup doesn't actually wake up the msleep
| | | in ipmi_submit_driver_request. The error being reported is that
| | | the msleep timed out. This doesn't seem to be critical problem
| | | since after this things seemed to work work. I saw this on 9.X.
| | | Haven't seen it on 8.2. Not sure about -current.
| | |
| | | It doesn't happen on all machines.
| | |
| | | Hmm, are you seeing the KCS thread manage the request but the wakeup()
| is
| | | lost?
| |
| | It was a couple of weeks ago that I played with it. I put printf's
| | around the msleep and wakeup. I saw the wakeup called but the sleep
| | not get it. I can try the test again later today. Right now my main
| | work machine is recovering from a power outage. This was with 9.0
| | when I first saw it. This issue seems to only happen at boot time.
| | If I kldload the module after the system is booted then it seems to work
| | okay. The KCS part was working fine and got the data okay from the
| | request. I haven't seen or heard any issues with 8.2.
|
| With -current I patched ipmi.c with:
| Index: ipmi.c
| ===
| --- ipmi.c (revision 233806)
| +++ ipmi.c (working copy)
| @@ -523,7 +523,11 @@
| * waiter that we awaken.
| */
| if (req-ir_owner == NULL)
| +{
| +device_printf(sc-ipmi_dev, DEBUG %s %d before wakeup
| %d\n,__FUNCTION__,__LINE__,ticks);
| wakeup(req);
| +device_printf(sc-ipmi_dev, DEBUG %s %d after wakeup
| %d\n,__FUNCTION__,__LINE__,ticks);
| +}
| else {
| dev = req-ir_owner;
| TAILQ_INSERT_TAIL(dev-ipmi_completed_requests, req,
| ir_link);
| @@ -543,7 +547,11 @@
| IPMI_LOCK(sc);
| error = sc-ipmi_enqueue_request(sc, req);
| if (error == 0)
| +{
| +device_printf(sc-ipmi_dev, DEBUG %s %d before msleep
| %d\n,__FUNCTION__,__LINE__,ticks);
| error = msleep(req, sc-ipmi_lock, 0, ipmireq, timo);
| +device_printf(sc-ipmi_dev, DEBUG %s %d after msleep
| %d\n,__FUNCTION__,__LINE__,ticks);
| +}
| if (error == 0)
| error = req-ir_error;
| IPMI_UNLOCK(sc);
| @@ -695,8 +703,11 @@
| error = ipmi_submit_driver_request(sc, req, MAX_TIMEOUT);
| if (error == EWOULDBLOCK) {
| device_printf(dev, Timed out waiting for GET_DEVICE_ID\n);
| + printf(DJA\n);
| +/*
| ipmi_free_request(req);
| return;
| +*/
| } else if (error) {
| device_printf(dev, Failed GET_DEVICE_ID: %d\n, error);
| ipmi_free_request(req);
|
| and get
|# dmesg | grep ipmi
|ipmi0: KCS mode found at io 0xca8 on acpi
|ipmi1: IPMI System Interface on isa0
|device_attach: ipmi1 attach returned 16
|ipmi1: IPMI System Interface on isa0
|device_attach: ipmi1 attach returned 16
|ipmi0: DEBUG ipmi_submit_driver_request 551 before msleep 2
|ipmi0: DEBUG ipmi_complete_request 527 before wakeup 6201
|ipmi0: DEBUG ipmi_complete_request 529 after wakeup 6263
|ipmi0: DEBUG ipmi_submit_driver_request 553 after msleep 6323
|
| Actually, can you compile with:
|
| options KTR
| options KTR_COMPILE=KTR_SCHED
| options
Re: [stable-ish 9] Dell R815 ipmi(4) attach failure
On Friday, March 30, 2012 5:30:34 pm Sean Bruno wrote: This is the relevant bits: Handle 0x2600, DMI type 38, 18 bytes IPMI Device Information Interface Type: KCS (Keyboard Control Style) Specification Version: 2.0 I2C Slave Address: 0x10 NV Storage Device: Not Present Base Address: 0x0CA8 (I/O) Register Spacing: 32-bit Boundaries Note the '32-bit' boundaries. I think ACPI doesn't support that for its attachment (well, it does if they specify each port as a separate thing in _CRS). Can you get acpidump -d output? Aye, here ya go. http://people.freebsd.org/~sbruno/acpidump_r815.txt Hmm, that actually looks correct. Try sabotaging the ACPI attach just for testing (i.e. add a 'return (ENXIO)' to the top of the probe routine in ipmi_acpi.c) just to see if that makes a difference. If not, then I think the BMC is just broken or your BIOS is lying. -- John Baldwin ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: [stable-ish 9] Dell R815 ipmi(4) attach failure
On Saturday, March 31, 2012 3:25:48 pm Doug Ambrisko wrote:
Sean Bruno writes:
| Noting a failure to attach to the onboard IPMI controller with this dell
| R815. Not sure what to start poking at and thought I'd though this over
| here for comment.
|
| -bash-4.2$ dmesg |grep ipmi
| ipmi0: KCS mode found at io 0xca8 on acpi
| ipmi1: IPMI System Interface on isa0
| device_attach: ipmi1 attach returned 16
| ipmi1: IPMI System Interface on isa0
| device_attach: ipmi1 attach returned 16
| ipmi0: Timed out waiting for GET_DEVICE_ID
I've run into this recently. A quick hack to fix it is:
Index: ipmi.c
===
RCS file: /cvs/src/sys/dev/ipmi/ipmi.c,v
retrieving revision 1.14
diff -u -p -r1.14 ipmi.c
--- ipmi.c14 Apr 2011 07:14:22 - 1.14
+++ ipmi.c31 Mar 2012 19:18:35 -
@@ -695,7 +695,6 @@ ipmi_startup(void *arg)
if (error == EWOULDBLOCK) {
device_printf(dev, Timed out waiting for GET_DEVICE_ID\n);
ipmi_free_request(req);
- return;
} else if (error) {
device_printf(dev, Failed GET_DEVICE_ID: %d\n, error);
ipmi_free_request(req);
The issue is that the wakeup doesn't actually wake up the msleep
in ipmi_submit_driver_request. The error being reported is that
the msleep timed out. This doesn't seem to be critical problem
since after this things seemed to work work. I saw this on 9.X.
Haven't seen it on 8.2. Not sure about -current.
It doesn't happen on all machines.
Hmm, are you seeing the KCS thread manage the request but the wakeup() is
lost?
--
John Baldwin
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: [stable-ish 9] Dell R815 ipmi(4) attach failure
John Baldwin writes:
| On Saturday, March 31, 2012 3:25:48 pm Doug Ambrisko wrote:
| Sean Bruno writes:
| | Noting a failure to attach to the onboard IPMI controller with this dell
| | R815. Not sure what to start poking at and thought I'd though this over
| | here for comment.
| |
| | -bash-4.2$ dmesg |grep ipmi
| | ipmi0: KCS mode found at io 0xca8 on acpi
| | ipmi1: IPMI System Interface on isa0
| | device_attach: ipmi1 attach returned 16
| | ipmi1: IPMI System Interface on isa0
| | device_attach: ipmi1 attach returned 16
| | ipmi0: Timed out waiting for GET_DEVICE_ID
|
| I've run into this recently. A quick hack to fix it is:
|
| Index: ipmi.c
| ===
| RCS file: /cvs/src/sys/dev/ipmi/ipmi.c,v
| retrieving revision 1.14
| diff -u -p -r1.14 ipmi.c
| --- ipmi.c 14 Apr 2011 07:14:22 - 1.14
| +++ ipmi.c 31 Mar 2012 19:18:35 -
| @@ -695,7 +695,6 @@ ipmi_startup(void *arg)
| if (error == EWOULDBLOCK) {
| device_printf(dev, Timed out waiting for GET_DEVICE_ID\n);
| ipmi_free_request(req);
| - return;
| } else if (error) {
| device_printf(dev, Failed GET_DEVICE_ID: %d\n, error);
| ipmi_free_request(req);
|
| The issue is that the wakeup doesn't actually wake up the msleep
| in ipmi_submit_driver_request. The error being reported is that
| the msleep timed out. This doesn't seem to be critical problem
| since after this things seemed to work work. I saw this on 9.X.
| Haven't seen it on 8.2. Not sure about -current.
|
| It doesn't happen on all machines.
|
| Hmm, are you seeing the KCS thread manage the request but the wakeup() is
| lost?
It was a couple of weeks ago that I played with it. I put printf's
around the msleep and wakeup. I saw the wakeup called but the sleep
not get it. I can try the test again later today. Right now my main
work machine is recovering from a power outage. This was with 9.0
when I first saw it. This issue seems to only happen at boot time.
If I kldload the module after the system is booted then it seems to work
okay. The KCS part was working fine and got the data okay from the
request. I haven't seen or heard any issues with 8.2.
Thanks,
Doug A.
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: [stable-ish 9] Dell R815 ipmi(4) attach failure
Doug Ambrisko writes:
| John Baldwin writes:
| | On Saturday, March 31, 2012 3:25:48 pm Doug Ambrisko wrote:
| | Sean Bruno writes:
| | | Noting a failure to attach to the onboard IPMI controller with this dell
| | | R815. Not sure what to start poking at and thought I'd though this over
| | | here for comment.
| | |
| | | -bash-4.2$ dmesg |grep ipmi
| | | ipmi0: KCS mode found at io 0xca8 on acpi
| | | ipmi1: IPMI System Interface on isa0
| | | device_attach: ipmi1 attach returned 16
| | | ipmi1: IPMI System Interface on isa0
| | | device_attach: ipmi1 attach returned 16
| | | ipmi0: Timed out waiting for GET_DEVICE_ID
| |
| | I've run into this recently. A quick hack to fix it is:
| |
| | Index: ipmi.c
| | ===
| | RCS file: /cvs/src/sys/dev/ipmi/ipmi.c,v
| | retrieving revision 1.14
| | diff -u -p -r1.14 ipmi.c
| | --- ipmi.c14 Apr 2011 07:14:22 - 1.14
| | +++ ipmi.c31 Mar 2012 19:18:35 -
| | @@ -695,7 +695,6 @@ ipmi_startup(void *arg)
| |if (error == EWOULDBLOCK) {
| |device_printf(dev, Timed out waiting for GET_DEVICE_ID\n);
| |ipmi_free_request(req);
| | - return;
| |} else if (error) {
| |device_printf(dev, Failed GET_DEVICE_ID: %d\n, error);
| |ipmi_free_request(req);
| |
| | The issue is that the wakeup doesn't actually wake up the msleep
| | in ipmi_submit_driver_request. The error being reported is that
| | the msleep timed out. This doesn't seem to be critical problem
| | since after this things seemed to work work. I saw this on 9.X.
| | Haven't seen it on 8.2. Not sure about -current.
| |
| | It doesn't happen on all machines.
| |
| | Hmm, are you seeing the KCS thread manage the request but the wakeup() is
| | lost?
|
| It was a couple of weeks ago that I played with it. I put printf's
| around the msleep and wakeup. I saw the wakeup called but the sleep
| not get it. I can try the test again later today. Right now my main
| work machine is recovering from a power outage. This was with 9.0
| when I first saw it. This issue seems to only happen at boot time.
| If I kldload the module after the system is booted then it seems to work
| okay. The KCS part was working fine and got the data okay from the
| request. I haven't seen or heard any issues with 8.2.
With -current I patched ipmi.c with:
Index: ipmi.c
===
--- ipmi.c (revision 233806)
+++ ipmi.c (working copy)
@@ -523,7 +523,11 @@
* waiter that we awaken.
*/
if (req-ir_owner == NULL)
+{
+device_printf(sc-ipmi_dev, DEBUG %s %d before wakeup
%d\n,__FUNCTION__,__LINE__,ticks);
wakeup(req);
+device_printf(sc-ipmi_dev, DEBUG %s %d after wakeup
%d\n,__FUNCTION__,__LINE__,ticks);
+}
else {
dev = req-ir_owner;
TAILQ_INSERT_TAIL(dev-ipmi_completed_requests, req, ir_link);
@@ -543,7 +547,11 @@
IPMI_LOCK(sc);
error = sc-ipmi_enqueue_request(sc, req);
if (error == 0)
+{
+device_printf(sc-ipmi_dev, DEBUG %s %d before msleep
%d\n,__FUNCTION__,__LINE__,ticks);
error = msleep(req, sc-ipmi_lock, 0, ipmireq, timo);
+device_printf(sc-ipmi_dev, DEBUG %s %d after msleep
%d\n,__FUNCTION__,__LINE__,ticks);
+}
if (error == 0)
error = req-ir_error;
IPMI_UNLOCK(sc);
@@ -695,8 +703,11 @@
error = ipmi_submit_driver_request(sc, req, MAX_TIMEOUT);
if (error == EWOULDBLOCK) {
device_printf(dev, Timed out waiting for GET_DEVICE_ID\n);
+ printf(DJA\n);
+/*
ipmi_free_request(req);
return;
+*/
} else if (error) {
device_printf(dev, Failed GET_DEVICE_ID: %d\n, error);
ipmi_free_request(req);
and get
# dmesg | grep ipmi
ipmi0: KCS mode found at io 0xca8 on acpi
ipmi1: IPMI System Interface on isa0
device_attach: ipmi1 attach returned 16
ipmi1: IPMI System Interface on isa0
device_attach: ipmi1 attach returned 16
ipmi0: DEBUG ipmi_submit_driver_request 551 before msleep 2
ipmi0: DEBUG ipmi_complete_request 527 before wakeup 6201
ipmi0: DEBUG ipmi_complete_request 529 after wakeup 6263
ipmi0: DEBUG ipmi_submit_driver_request 553 after msleep 6323
ipmi0: Timed out waiting for GET_DEVICE_ID
ipmi0: IPMI device rev. 0, firmware rev. 1.61, version 2.0
ipmi0: DEBUG ipmi_submit_driver_request 551 before msleep 6503
ipmi0: DEBUG ipmi_complete_request 527 before wakeup 6620
ipmi0: DEBUG ipmi_complete_request 529 after wakeup 6682
ipmi0: DEBUG ipmi_submit_driver_request 553 after msleep 6742
...
# sysctl kern.clockrate
kern.clockrate: { hz = 1000, tick = 1000, profhz = 8126, stathz = 127 }
#
The 6000+ ticks before calling wakeup looks to be a problem. Maybe
a few things are getting
Re: [stable-ish 9] Dell R815 ipmi(4) attach failure
Sean Bruno writes:
| Noting a failure to attach to the onboard IPMI controller with this dell
| R815. Not sure what to start poking at and thought I'd though this over
| here for comment.
|
| -bash-4.2$ dmesg |grep ipmi
| ipmi0: KCS mode found at io 0xca8 on acpi
| ipmi1: IPMI System Interface on isa0
| device_attach: ipmi1 attach returned 16
| ipmi1: IPMI System Interface on isa0
| device_attach: ipmi1 attach returned 16
| ipmi0: Timed out waiting for GET_DEVICE_ID
I've run into this recently. A quick hack to fix it is:
Index: ipmi.c
===
RCS file: /cvs/src/sys/dev/ipmi/ipmi.c,v
retrieving revision 1.14
diff -u -p -r1.14 ipmi.c
--- ipmi.c 14 Apr 2011 07:14:22 - 1.14
+++ ipmi.c 31 Mar 2012 19:18:35 -
@@ -695,7 +695,6 @@ ipmi_startup(void *arg)
if (error == EWOULDBLOCK) {
device_printf(dev, Timed out waiting for GET_DEVICE_ID\n);
ipmi_free_request(req);
- return;
} else if (error) {
device_printf(dev, Failed GET_DEVICE_ID: %d\n, error);
ipmi_free_request(req);
The issue is that the wakeup doesn't actually wake up the msleep
in ipmi_submit_driver_request. The error being reported is that
the msleep timed out. This doesn't seem to be critical problem
since after this things seemed to work work. I saw this on 9.X.
Haven't seen it on 8.2. Not sure about -current.
It doesn't happen on all machines.
Doug A.
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: [stable-ish 9] Dell R815 ipmi(4) attach failure
Doug Ambrisko writes:
| Sean Bruno writes:
| | Noting a failure to attach to the onboard IPMI controller with this dell
| | R815. Not sure what to start poking at and thought I'd though this over
| | here for comment.
| |
| | -bash-4.2$ dmesg |grep ipmi
| | ipmi0: KCS mode found at io 0xca8 on acpi
| | ipmi1: IPMI System Interface on isa0
| | device_attach: ipmi1 attach returned 16
| | ipmi1: IPMI System Interface on isa0
| | device_attach: ipmi1 attach returned 16
| | ipmi0: Timed out waiting for GET_DEVICE_ID
|
| I've run into this recently. A quick hack to fix it is:
|
| Index: ipmi.c
| ===
| RCS file: /cvs/src/sys/dev/ipmi/ipmi.c,v
| retrieving revision 1.14
| diff -u -p -r1.14 ipmi.c
| --- ipmi.c14 Apr 2011 07:14:22 - 1.14
| +++ ipmi.c31 Mar 2012 19:18:35 -
| @@ -695,7 +695,6 @@ ipmi_startup(void *arg)
| if (error == EWOULDBLOCK) {
| device_printf(dev, Timed out waiting for GET_DEVICE_ID\n);
| ipmi_free_request(req);
| - return;
Correction get rid of the ipmi_free_request as well.
If you kldload then it doesn't have this issue. I've been doing that
on -current for a while so I didn't notice the regression when it happened.
| } else if (error) {
| device_printf(dev, Failed GET_DEVICE_ID: %d\n, error);
| ipmi_free_request(req);
|
| The issue is that the wakeup doesn't actually wake up the msleep
| in ipmi_submit_driver_request. The error being reported is that
| the msleep timed out. This doesn't seem to be critical problem
| since after this things seemed to work work. I saw this on 9.X.
| Haven't seen it on 8.2. Not sure about -current.
|
| It doesn't happen on all machines.
|
| Doug A.
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: [stable-ish 9] Dell R815 ipmi(4) attach failure
On Thursday, March 29, 2012 12:48:39 pm Sean Bruno wrote: Noting a failure to attach to the onboard IPMI controller with this dell R815. Not sure what to start poking at and thought I'd though this over here for comment. -bash-4.2$ dmesg |grep ipmi ipmi0: KCS mode found at io 0xca8 on acpi ipmi1: IPMI System Interface on isa0 device_attach: ipmi1 attach returned 16 ipmi1: IPMI System Interface on isa0 device_attach: ipmi1 attach returned 16 ipmi0: Timed out waiting for GET_DEVICE_ID -bash-4.2$ sysctl -a|grep ipmi device ipmi# IPMI hw.ipmi.on: 1 dev.ipmi.0.%desc: IPMI System Interface dev.ipmi.0.%driver: ipmi dev.ipmi.0.%location: handle=\_SB_.PCI0.ISA_.NIPM dev.ipmi.0.%pnpinfo: _HID=IPI0001 _UID=5 dev.ipmi.0.%parent: acpi0 Can you get dmidecode output? -- John Baldwin ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: [stable-ish 9] Dell R815 ipmi(4) attach failure
On Fri, 2012-03-30 at 09:29 -0700, John Baldwin wrote: On Thursday, March 29, 2012 12:48:39 pm Sean Bruno wrote: Noting a failure to attach to the onboard IPMI controller with this dell R815. Not sure what to start poking at and thought I'd though this over here for comment. -bash-4.2$ dmesg |grep ipmi ipmi0: KCS mode found at io 0xca8 on acpi ipmi1: IPMI System Interface on isa0 device_attach: ipmi1 attach returned 16 ipmi1: IPMI System Interface on isa0 device_attach: ipmi1 attach returned 16 ipmi0: Timed out waiting for GET_DEVICE_ID -bash-4.2$ sysctl -a|grep ipmi device ipmi# IPMI hw.ipmi.on: 1 dev.ipmi.0.%desc: IPMI System Interface dev.ipmi.0.%driver: ipmi dev.ipmi.0.%location: handle=\_SB_.PCI0.ISA_.NIPM dev.ipmi.0.%pnpinfo: _HID=IPI0001 _UID=5 dev.ipmi.0.%parent: acpi0 Can you get dmidecode output? http://people.freebsd.org/~sbruno/dmidecode_r815.txt http://people.freebsd.org/~sbruno/pciconf_r815.txt Sean ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: [stable-ish 9] Dell R815 ipmi(4) attach failure
On Friday, March 30, 2012 4:03:54 pm Sean Bruno wrote: On Fri, 2012-03-30 at 09:29 -0700, John Baldwin wrote: On Thursday, March 29, 2012 12:48:39 pm Sean Bruno wrote: Noting a failure to attach to the onboard IPMI controller with this dell R815. Not sure what to start poking at and thought I'd though this over here for comment. -bash-4.2$ dmesg |grep ipmi ipmi0: KCS mode found at io 0xca8 on acpi ipmi1: IPMI System Interface on isa0 device_attach: ipmi1 attach returned 16 ipmi1: IPMI System Interface on isa0 device_attach: ipmi1 attach returned 16 ipmi0: Timed out waiting for GET_DEVICE_ID -bash-4.2$ sysctl -a|grep ipmi device ipmi# IPMI hw.ipmi.on: 1 dev.ipmi.0.%desc: IPMI System Interface dev.ipmi.0.%driver: ipmi dev.ipmi.0.%location: handle=\_SB_.PCI0.ISA_.NIPM dev.ipmi.0.%pnpinfo: _HID=IPI0001 _UID=5 dev.ipmi.0.%parent: acpi0 Can you get dmidecode output? http://people.freebsd.org/~sbruno/dmidecode_r815.txt This is the relevant bits: Handle 0x2600, DMI type 38, 18 bytes IPMI Device Information Interface Type: KCS (Keyboard Control Style) Specification Version: 2.0 I2C Slave Address: 0x10 NV Storage Device: Not Present Base Address: 0x0CA8 (I/O) Register Spacing: 32-bit Boundaries Note the '32-bit' boundaries. I think ACPI doesn't support that for its attachment (well, it does if they specify each port as a separate thing in _CRS). Can you get acpidump -d output? -- John Baldwin ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: [stable-ish 9] Dell R815 ipmi(4) attach failure
This is the relevant bits: Handle 0x2600, DMI type 38, 18 bytes IPMI Device Information Interface Type: KCS (Keyboard Control Style) Specification Version: 2.0 I2C Slave Address: 0x10 NV Storage Device: Not Present Base Address: 0x0CA8 (I/O) Register Spacing: 32-bit Boundaries Note the '32-bit' boundaries. I think ACPI doesn't support that for its attachment (well, it does if they specify each port as a separate thing in _CRS). Can you get acpidump -d output? Aye, here ya go. http://people.freebsd.org/~sbruno/acpidump_r815.txt sean ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
