Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
On Tue, Apr 27, 2010 at 7:06 PM, Maxim Sobolev wrote: > Alexander Sack wrote: >> >> On Tue, Apr 27, 2010 at 5:25 PM, John Baldwin wrote: >>> >>> On Tuesday 27 April 2010 5:06:37 pm Maxim Sobolev wrote: John Baldwin wrote: > > On Tuesday 27 April 2010 4:26:09 pm Maxim Sobolev wrote: >> >> John Baldwin wrote: >>> >>> Hmm, I think you should definitely commit the atkbdc_isa.c change >>> first of >>> all. I'm still thinking about the other change. I wonder if we can >>> figure >>> out that a keyboard isn't present sooner somehow? Do you know if the >>> keyboard >>> appears to be present but just slow vs if the keyboard is eventually >>> found to >>> not be present? >> >> Our syscons does keyboard probing two times - once early during kernel >> initialization before most of the subsystems have been initialized >> yet, >> and then "real" probing later in boot process. Interesting thing is >> that >> initially keyboard looks present. Reading status port in >> atkbdc_configure() gives value other than 0xff, although reading is >> thousand times slower than usually. This causes syscons try attaching >> it. Even though reading status port works, apparently either emulation >> is not complete or there is some other issue, so that it never >> responds >> to some commands. Slow access and lack of response results in >> wait_for_data() function waiting several minutes instead of 200ms as >> designed. This what causes that 6-10 minutes delay in boot process. > > I believe the USB driver has disabled the keyboard emulation by the > time the > second probe happens in syscons. Can you try disabling legacy USB > support in > the BIOS just to make sure that is what causes the delay? Unfortunately it's not possible. Hosting provider doesn't allow me to have access to BIOS settings. >> >> Stunt double: I tried it and it has no effect. The waits in atkdbd >> kills it with or without USB legacy support on. The wait on this >> machine is about 1-2 minutes before boot. Just another data point. > > Have you tried my patch? Does it help? Maxim, yes I have. Works much better. Wait time is nominal. I would definitely document the time delay code as its non-intuitive looking at it. -aps ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
Alexander Sack wrote: On Tue, Apr 27, 2010 at 5:25 PM, John Baldwin wrote: On Tuesday 27 April 2010 5:06:37 pm Maxim Sobolev wrote: John Baldwin wrote: On Tuesday 27 April 2010 4:26:09 pm Maxim Sobolev wrote: John Baldwin wrote: Hmm, I think you should definitely commit the atkbdc_isa.c change first of all. I'm still thinking about the other change. I wonder if we can figure out that a keyboard isn't present sooner somehow? Do you know if the keyboard appears to be present but just slow vs if the keyboard is eventually found to not be present? Our syscons does keyboard probing two times - once early during kernel initialization before most of the subsystems have been initialized yet, and then "real" probing later in boot process. Interesting thing is that initially keyboard looks present. Reading status port in atkbdc_configure() gives value other than 0xff, although reading is thousand times slower than usually. This causes syscons try attaching it. Even though reading status port works, apparently either emulation is not complete or there is some other issue, so that it never responds to some commands. Slow access and lack of response results in wait_for_data() function waiting several minutes instead of 200ms as designed. This what causes that 6-10 minutes delay in boot process. I believe the USB driver has disabled the keyboard emulation by the time the second probe happens in syscons. Can you try disabling legacy USB support in the BIOS just to make sure that is what causes the delay? Unfortunately it's not possible. Hosting provider doesn't allow me to have access to BIOS settings. Stunt double: I tried it and it has no effect. The waits in atkdbd kills it with or without USB legacy support on. The wait on this machine is about 1-2 minutes before boot. Just another data point. Have you tried my patch? Does it help? -Maxim ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
On Tue, Apr 27, 2010 at 5:25 PM, John Baldwin wrote: > On Tuesday 27 April 2010 5:06:37 pm Maxim Sobolev wrote: >> John Baldwin wrote: >> > On Tuesday 27 April 2010 4:26:09 pm Maxim Sobolev wrote: >> >> John Baldwin wrote: >> >>> Hmm, I think you should definitely commit the atkbdc_isa.c change first >> >>> of >> >>> all. I'm still thinking about the other change. I wonder if we can >> >>> figure >> >>> out that a keyboard isn't present sooner somehow? Do you know if the >> >>> keyboard >> >>> appears to be present but just slow vs if the keyboard is eventually >> >>> found to >> >>> not be present? >> >> Our syscons does keyboard probing two times - once early during kernel >> >> initialization before most of the subsystems have been initialized yet, >> >> and then "real" probing later in boot process. Interesting thing is that >> >> initially keyboard looks present. Reading status port in >> >> atkbdc_configure() gives value other than 0xff, although reading is >> >> thousand times slower than usually. This causes syscons try attaching >> >> it. Even though reading status port works, apparently either emulation >> >> is not complete or there is some other issue, so that it never responds >> >> to some commands. Slow access and lack of response results in >> >> wait_for_data() function waiting several minutes instead of 200ms as >> >> designed. This what causes that 6-10 minutes delay in boot process. >> > >> > I believe the USB driver has disabled the keyboard emulation by the time >> > the >> > second probe happens in syscons. Can you try disabling legacy USB support >> > in >> > the BIOS just to make sure that is what causes the delay? >> >> Unfortunately it's not possible. Hosting provider doesn't allow me to >> have access to BIOS settings. Stunt double: I tried it and it has no effect. The waits in atkdbd kills it with or without USB legacy support on. The wait on this machine is about 1-2 minutes before boot. Just another data point. -aps ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
On Tuesday 27 April 2010 5:06:37 pm Maxim Sobolev wrote: > John Baldwin wrote: > > On Tuesday 27 April 2010 4:26:09 pm Maxim Sobolev wrote: > >> John Baldwin wrote: > >>> Hmm, I think you should definitely commit the atkbdc_isa.c change first > >>> of > >>> all. I'm still thinking about the other change. I wonder if we can > >>> figure > >>> out that a keyboard isn't present sooner somehow? Do you know if the > >>> keyboard > >>> appears to be present but just slow vs if the keyboard is eventually > >>> found to > >>> not be present? > >> Our syscons does keyboard probing two times - once early during kernel > >> initialization before most of the subsystems have been initialized yet, > >> and then "real" probing later in boot process. Interesting thing is that > >> initially keyboard looks present. Reading status port in > >> atkbdc_configure() gives value other than 0xff, although reading is > >> thousand times slower than usually. This causes syscons try attaching > >> it. Even though reading status port works, apparently either emulation > >> is not complete or there is some other issue, so that it never responds > >> to some commands. Slow access and lack of response results in > >> wait_for_data() function waiting several minutes instead of 200ms as > >> designed. This what causes that 6-10 minutes delay in boot process. > > > > I believe the USB driver has disabled the keyboard emulation by the time the > > second probe happens in syscons. Can you try disabling legacy USB support > > in > > the BIOS just to make sure that is what causes the delay? > > Unfortunately it's not possible. Hosting provider doesn't allow me to > have access to BIOS settings. Hmm, ok. > >> In fact I've also tried sending 0xAA command instead of just checking > >> that value read from the status port is not 0xFF, and it actually > >> completed fine at this point. The controller has returned 0x55 as > >> expected. Therefore, I believe measuring inb() delay is the only > >> reasonable way to avoid that big boot delay at this point. > >> > >> Later on, when we probe/attach keyboard second time (in atkbdc_isa.c) > >> reading status port returns 0xff, so the we can fail attachment process > >> immediately. However, this is not a big issue since at this point > >> reading from status port is as fast as any other ISA inb() operations, > >> so it doesn't cause any noticeable delay. > >> > >> Here is the latest version of the patch: > >> > >> http://sobomax.sippysoft.com/atkbdc.diff > > > > Hmm, still has the issue that we can't assume a TSC on i386. I would still > > commit the easy bits to change various '#if defined(__i386__)' to > > '#if defined(__i386__) || defined(__amd64__)' now. > > > > One thing that could make this cleaner would be to have a macro defined > > something like this in atkbdc.c: > > > > /* CPU will stay inside loops for 100msec at most. */ > > #ifdef __amd64__ > > #define KBD_RETRY(kbd) (10 / ((KBDD_DELAYTIME * 2) + > > kbdc->read_delay)) > > #else > > #define KBD_RETRY(kbd) (5000) > > #endif > > > > and then use 'KBD_RETRY(kbd)' to initialize 'retry' in various places. > > Please take a closer look. Use of rdtsc() in this version is conditonal > on __amd64__ only. Ok, that does look a lot better. I don't like having to use rdtsc() to time the delay but I don't have any better suggestions. I think you should add a block comment above the calibration section to explain why you are doing it (i.e. some BIOSes' PS/2 emulation take a really long time to execute). -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
John Baldwin wrote: On Tuesday 27 April 2010 4:26:09 pm Maxim Sobolev wrote: John Baldwin wrote: Hmm, I think you should definitely commit the atkbdc_isa.c change first of all. I'm still thinking about the other change. I wonder if we can figure out that a keyboard isn't present sooner somehow? Do you know if the keyboard appears to be present but just slow vs if the keyboard is eventually found to not be present? Our syscons does keyboard probing two times - once early during kernel initialization before most of the subsystems have been initialized yet, and then "real" probing later in boot process. Interesting thing is that initially keyboard looks present. Reading status port in atkbdc_configure() gives value other than 0xff, although reading is thousand times slower than usually. This causes syscons try attaching it. Even though reading status port works, apparently either emulation is not complete or there is some other issue, so that it never responds to some commands. Slow access and lack of response results in wait_for_data() function waiting several minutes instead of 200ms as designed. This what causes that 6-10 minutes delay in boot process. I believe the USB driver has disabled the keyboard emulation by the time the second probe happens in syscons. Can you try disabling legacy USB support in the BIOS just to make sure that is what causes the delay? Unfortunately it's not possible. Hosting provider doesn't allow me to have access to BIOS settings. In fact I've also tried sending 0xAA command instead of just checking that value read from the status port is not 0xFF, and it actually completed fine at this point. The controller has returned 0x55 as expected. Therefore, I believe measuring inb() delay is the only reasonable way to avoid that big boot delay at this point. Later on, when we probe/attach keyboard second time (in atkbdc_isa.c) reading status port returns 0xff, so the we can fail attachment process immediately. However, this is not a big issue since at this point reading from status port is as fast as any other ISA inb() operations, so it doesn't cause any noticeable delay. Here is the latest version of the patch: http://sobomax.sippysoft.com/atkbdc.diff Hmm, still has the issue that we can't assume a TSC on i386. I would still commit the easy bits to change various '#if defined(__i386__)' to '#if defined(__i386__) || defined(__amd64__)' now. One thing that could make this cleaner would be to have a macro defined something like this in atkbdc.c: /* CPU will stay inside loops for 100msec at most. */ #ifdef __amd64__ #define KBD_RETRY(kbd) (10 / ((KBDD_DELAYTIME * 2) + kbdc->read_delay)) #else #define KBD_RETRY(kbd) (5000) #endif and then use 'KBD_RETRY(kbd)' to initialize 'retry' in various places. Please take a closer look. Use of rdtsc() in this version is conditonal on __amd64__ only. -Maxim ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
On Tuesday 27 April 2010 4:26:09 pm Maxim Sobolev wrote: > John Baldwin wrote: > > Hmm, I think you should definitely commit the atkbdc_isa.c change first of > > all. I'm still thinking about the other change. I wonder if we can figure > > out that a keyboard isn't present sooner somehow? Do you know if the > > keyboard > > appears to be present but just slow vs if the keyboard is eventually found > > to > > not be present? > > Our syscons does keyboard probing two times - once early during kernel > initialization before most of the subsystems have been initialized yet, > and then "real" probing later in boot process. Interesting thing is that > initially keyboard looks present. Reading status port in > atkbdc_configure() gives value other than 0xff, although reading is > thousand times slower than usually. This causes syscons try attaching > it. Even though reading status port works, apparently either emulation > is not complete or there is some other issue, so that it never responds > to some commands. Slow access and lack of response results in > wait_for_data() function waiting several minutes instead of 200ms as > designed. This what causes that 6-10 minutes delay in boot process. I believe the USB driver has disabled the keyboard emulation by the time the second probe happens in syscons. Can you try disabling legacy USB support in the BIOS just to make sure that is what causes the delay? > In fact I've also tried sending 0xAA command instead of just checking > that value read from the status port is not 0xFF, and it actually > completed fine at this point. The controller has returned 0x55 as > expected. Therefore, I believe measuring inb() delay is the only > reasonable way to avoid that big boot delay at this point. > > Later on, when we probe/attach keyboard second time (in atkbdc_isa.c) > reading status port returns 0xff, so the we can fail attachment process > immediately. However, this is not a big issue since at this point > reading from status port is as fast as any other ISA inb() operations, > so it doesn't cause any noticeable delay. > > Here is the latest version of the patch: > > http://sobomax.sippysoft.com/atkbdc.diff Hmm, still has the issue that we can't assume a TSC on i386. I would still commit the easy bits to change various '#if defined(__i386__)' to '#if defined(__i386__) || defined(__amd64__)' now. One thing that could make this cleaner would be to have a macro defined something like this in atkbdc.c: /* CPU will stay inside loops for 100msec at most. */ #ifdef __amd64__ #define KBD_RETRY(kbd) (10 / ((KBDD_DELAYTIME * 2) + kbdc->read_delay)) #else #define KBD_RETRY(kbd) (5000) #endif and then use 'KBD_RETRY(kbd)' to initialize 'retry' in various places. -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
John Baldwin wrote: Hmm, I think you should definitely commit the atkbdc_isa.c change first of all. I'm still thinking about the other change. I wonder if we can figure out that a keyboard isn't present sooner somehow? Do you know if the keyboard appears to be present but just slow vs if the keyboard is eventually found to not be present? Our syscons does keyboard probing two times - once early during kernel initialization before most of the subsystems have been initialized yet, and then "real" probing later in boot process. Interesting thing is that initially keyboard looks present. Reading status port in atkbdc_configure() gives value other than 0xff, although reading is thousand times slower than usually. This causes syscons try attaching it. Even though reading status port works, apparently either emulation is not complete or there is some other issue, so that it never responds to some commands. Slow access and lack of response results in wait_for_data() function waiting several minutes instead of 200ms as designed. This what causes that 6-10 minutes delay in boot process. In fact I've also tried sending 0xAA command instead of just checking that value read from the status port is not 0xFF, and it actually completed fine at this point. The controller has returned 0x55 as expected. Therefore, I believe measuring inb() delay is the only reasonable way to avoid that big boot delay at this point. Later on, when we probe/attach keyboard second time (in atkbdc_isa.c) reading status port returns 0xff, so the we can fail attachment process immediately. However, this is not a big issue since at this point reading from status port is as fast as any other ISA inb() operations, so it doesn't cause any noticeable delay. Here is the latest version of the patch: http://sobomax.sippysoft.com/atkbdc.diff -Maxim ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
On Tue, Apr 27, 2010 at 12:04 PM, John Baldwin wrote: > On Thursday 22 April 2010 4:07:37 pm Maxim Sobolev wrote: >> John Baldwin wrote: >> > On Thursday 22 April 2010 2:28:26 pm Maxim Sobolev wrote: >> >> John Baldwin wrote: >> >>> On Thursday 22 April 2010 6:05:04 am Maxim Sobolev wrote: >> Maxim Sobolev wrote: >> > There is already a code to detect non-existing AT keyboard and avoid >> > attaching atkbd to it. The code is i386-only at the moment, I am > trying >> > to figure out how to modify it so that it works on amd64 as well. >> Looks like this huge delay is caused by the inb() being astonishingly >> slow, which is not factored by the timeout routines. Reading keyboard >> status port once takes about 0.003s! I am not sure if it's common >> behaviour of the platform, or something specific to this particular >> model. Do you know by any chance? >> >>> Well, many BIOSes trigger an SMI# when doing inb/outb to the keyboard > ports so >> >>> they can emulate a PS/2 keyboard when a USB keyboard is inserted. Do > you have >> >>> any BIOS options related to the USB legacy compat? I know of the > Nehalem >> >>> systems I've seen they have a separate option for controlling port 60/64 >> >>> emulation which we leave disabled by default. >> >> That makes sense. Unfortunately I don't have access to the BIOS >> >> settings. This is a hosted system, and the provider keeps BIOS password >> >> for themselves. >> >> >> >> I have a patch that fixes that issue by measuring status register >> >> reading time first and then factoring it in the calculations of the >> >> number of retries: >> >> >> >> http://sobomax.sippysoft.com/atkbdc.diff >> >> >> >> It also applies the same logic to detect broken/non-existing keyboard >> >> controller to amd64 as we do to the i386. I'd appreciate if you can do a >> >> review. >> > >> > Hmm, not all i386 CPUs that we support have a TSC. Is the change to >> > atkbdc_isa.c sufficient to fix the hang? If so, I'd rather just commit > that >> > bit and leave out the read_delay changes. >> >> No, it's not sufficient. The problem here is that for some reason that >> test passes on that system (probably emulation works) and so that normal >> keyboard attach routine is invoked early in boot, when we don't even >> have clock initialized. What if I make TSC-related changes amd64? Will >> that be OK? > > Hmm, I think you should definitely commit the atkbdc_isa.c change first of > all. I'm still thinking about the other change. I wonder if we can figure > out that a keyboard isn't present sooner somehow? Do you know if the keyboard > appears to be present but just slow vs if the keyboard is eventually found to > not be present? S5520UR, Intel BIOS 48 (last 2 digits), Build 2/27/10. If I disable 60/64 emulation the box refuses to go into the BIOS or boot anything. The BIOS just simply hangs after the Intel logo screen. This just seems like a bug. On the last generation Alcolu based machines, we usually have 60/64h emulation disabled which works just fine (USB Legacy Mode is still enabled so things like the debugger still/kinda/sorta work). I will work through our Intel channels to have them look at this (we already discovered some other bugs with respect to flashing and RMM3). I am looking at the atkbd driver for the first time. It would seem to me at first glance that John makes a very good point: there is already code to deal with a lack of a keyboard in the i386 code. I would *assume* that turning it on for amd64 would be all that is necessary? Btw these systems also fail other keyboard related tests. I also see: "atkbd: unable to set the command byte." On all Nehalem systems (this might be less serious given the OS has taken over from the BIOS). I am testing a variant of Maxim's patch right now. Stay tuned, -aps ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
On Thursday 22 April 2010 4:07:37 pm Maxim Sobolev wrote: > John Baldwin wrote: > > On Thursday 22 April 2010 2:28:26 pm Maxim Sobolev wrote: > >> John Baldwin wrote: > >>> On Thursday 22 April 2010 6:05:04 am Maxim Sobolev wrote: > Maxim Sobolev wrote: > > There is already a code to detect non-existing AT keyboard and avoid > > attaching atkbd to it. The code is i386-only at the moment, I am trying > > to figure out how to modify it so that it works on amd64 as well. > Looks like this huge delay is caused by the inb() being astonishingly > slow, which is not factored by the timeout routines. Reading keyboard > status port once takes about 0.003s! I am not sure if it's common > behaviour of the platform, or something specific to this particular > model. Do you know by any chance? > >>> Well, many BIOSes trigger an SMI# when doing inb/outb to the keyboard ports so > >>> they can emulate a PS/2 keyboard when a USB keyboard is inserted. Do you have > >>> any BIOS options related to the USB legacy compat? I know of the Nehalem > >>> systems I've seen they have a separate option for controlling port 60/64 > >>> emulation which we leave disabled by default. > >> That makes sense. Unfortunately I don't have access to the BIOS > >> settings. This is a hosted system, and the provider keeps BIOS password > >> for themselves. > >> > >> I have a patch that fixes that issue by measuring status register > >> reading time first and then factoring it in the calculations of the > >> number of retries: > >> > >> http://sobomax.sippysoft.com/atkbdc.diff > >> > >> It also applies the same logic to detect broken/non-existing keyboard > >> controller to amd64 as we do to the i386. I'd appreciate if you can do a > >> review. > > > > Hmm, not all i386 CPUs that we support have a TSC. Is the change to > > atkbdc_isa.c sufficient to fix the hang? If so, I'd rather just commit that > > bit and leave out the read_delay changes. > > No, it's not sufficient. The problem here is that for some reason that > test passes on that system (probably emulation works) and so that normal > keyboard attach routine is invoked early in boot, when we don't even > have clock initialized. What if I make TSC-related changes amd64? Will > that be OK? Hmm, I think you should definitely commit the atkbdc_isa.c change first of all. I'm still thinking about the other change. I wonder if we can figure out that a keyboard isn't present sooner somehow? Do you know if the keyboard appears to be present but just slow vs if the keyboard is eventually found to not be present? -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
John Baldwin wrote: On Thursday 22 April 2010 2:28:26 pm Maxim Sobolev wrote: John Baldwin wrote: On Thursday 22 April 2010 6:05:04 am Maxim Sobolev wrote: Maxim Sobolev wrote: There is already a code to detect non-existing AT keyboard and avoid attaching atkbd to it. The code is i386-only at the moment, I am trying to figure out how to modify it so that it works on amd64 as well. Looks like this huge delay is caused by the inb() being astonishingly slow, which is not factored by the timeout routines. Reading keyboard status port once takes about 0.003s! I am not sure if it's common behaviour of the platform, or something specific to this particular model. Do you know by any chance? Well, many BIOSes trigger an SMI# when doing inb/outb to the keyboard ports so they can emulate a PS/2 keyboard when a USB keyboard is inserted. Do you have any BIOS options related to the USB legacy compat? I know of the Nehalem systems I've seen they have a separate option for controlling port 60/64 emulation which we leave disabled by default. That makes sense. Unfortunately I don't have access to the BIOS settings. This is a hosted system, and the provider keeps BIOS password for themselves. I have a patch that fixes that issue by measuring status register reading time first and then factoring it in the calculations of the number of retries: http://sobomax.sippysoft.com/atkbdc.diff It also applies the same logic to detect broken/non-existing keyboard controller to amd64 as we do to the i386. I'd appreciate if you can do a review. Hmm, not all i386 CPUs that we support have a TSC. Is the change to atkbdc_isa.c sufficient to fix the hang? If so, I'd rather just commit that bit and leave out the read_delay changes. No, it's not sufficient. The problem here is that for some reason that test passes on that system (probably emulation works) and so that normal keyboard attach routine is invoked early in boot, when we don't even have clock initialized. What if I make TSC-related changes amd64? Will that be OK? -Maxim ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
On Thursday 22 April 2010 2:28:26 pm Maxim Sobolev wrote: > John Baldwin wrote: > > On Thursday 22 April 2010 6:05:04 am Maxim Sobolev wrote: > >> Maxim Sobolev wrote: > >>> There is already a code to detect non-existing AT keyboard and avoid > >>> attaching atkbd to it. The code is i386-only at the moment, I am trying > >>> to figure out how to modify it so that it works on amd64 as well. > >> Looks like this huge delay is caused by the inb() being astonishingly > >> slow, which is not factored by the timeout routines. Reading keyboard > >> status port once takes about 0.003s! I am not sure if it's common > >> behaviour of the platform, or something specific to this particular > >> model. Do you know by any chance? > > > > Well, many BIOSes trigger an SMI# when doing inb/outb to the keyboard ports > > so > > they can emulate a PS/2 keyboard when a USB keyboard is inserted. Do you > > have > > any BIOS options related to the USB legacy compat? I know of the Nehalem > > systems I've seen they have a separate option for controlling port 60/64 > > emulation which we leave disabled by default. > > That makes sense. Unfortunately I don't have access to the BIOS > settings. This is a hosted system, and the provider keeps BIOS password > for themselves. > > I have a patch that fixes that issue by measuring status register > reading time first and then factoring it in the calculations of the > number of retries: > > http://sobomax.sippysoft.com/atkbdc.diff > > It also applies the same logic to detect broken/non-existing keyboard > controller to amd64 as we do to the i386. I'd appreciate if you can do a > review. Hmm, not all i386 CPUs that we support have a TSC. Is the change to atkbdc_isa.c sufficient to fix the hang? If so, I'd rather just commit that bit and leave out the read_delay changes. -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
John Baldwin wrote: On Thursday 22 April 2010 6:05:04 am Maxim Sobolev wrote: Maxim Sobolev wrote: There is already a code to detect non-existing AT keyboard and avoid attaching atkbd to it. The code is i386-only at the moment, I am trying to figure out how to modify it so that it works on amd64 as well. Looks like this huge delay is caused by the inb() being astonishingly slow, which is not factored by the timeout routines. Reading keyboard status port once takes about 0.003s! I am not sure if it's common behaviour of the platform, or something specific to this particular model. Do you know by any chance? Well, many BIOSes trigger an SMI# when doing inb/outb to the keyboard ports so they can emulate a PS/2 keyboard when a USB keyboard is inserted. Do you have any BIOS options related to the USB legacy compat? I know of the Nehalem systems I've seen they have a separate option for controlling port 60/64 emulation which we leave disabled by default. That makes sense. Unfortunately I don't have access to the BIOS settings. This is a hosted system, and the provider keeps BIOS password for themselves. I have a patch that fixes that issue by measuring status register reading time first and then factoring it in the calculations of the number of retries: http://sobomax.sippysoft.com/atkbdc.diff It also applies the same logic to detect broken/non-existing keyboard controller to amd64 as we do to the i386. I'd appreciate if you can do a review. -Maxim ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
On Thursday 22 April 2010 6:05:04 am Maxim Sobolev wrote: > Maxim Sobolev wrote: > > There is already a code to detect non-existing AT keyboard and avoid > > attaching atkbd to it. The code is i386-only at the moment, I am trying > > to figure out how to modify it so that it works on amd64 as well. > > Looks like this huge delay is caused by the inb() being astonishingly > slow, which is not factored by the timeout routines. Reading keyboard > status port once takes about 0.003s! I am not sure if it's common > behaviour of the platform, or something specific to this particular > model. Do you know by any chance? Well, many BIOSes trigger an SMI# when doing inb/outb to the keyboard ports so they can emulate a PS/2 keyboard when a USB keyboard is inserted. Do you have any BIOS options related to the USB legacy compat? I know of the Nehalem systems I've seen they have a separate option for controlling port 60/64 emulation which we leave disabled by default. -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
Maxim Sobolev wrote: There is already a code to detect non-existing AT keyboard and avoid attaching atkbd to it. The code is i386-only at the moment, I am trying to figure out how to modify it so that it works on amd64 as well. Looks like this huge delay is caused by the inb() being astonishingly slow, which is not factored by the timeout routines. Reading keyboard status port once takes about 0.003s! I am not sure if it's common behaviour of the platform, or something specific to this particular model. Do you know by any chance? -Maxim ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
John Baldwin wrote: On Tuesday 20 April 2010 7:35:46 pm Matthew Jacob wrote: On 04/20/2010 03:44 PM, Maxim Sobolev wrote: Maxim Sobolev wrote: Maybe try adding hint.atkbdc.0.disabled="1" hint.atkbd.0.disabled="1" to /boot/device.hints? That has reportedly removed minute-long boot delays on some Nehalem machines. No, that have not helped at all. I measured the delay - it's about 6 minutes from boot command to the first "smap" message. Do you or anybody else have other ideas? Actually it helped, thank you very much! The problem was that I have had my hints compiled into the kernel itself. Me too! I can't reproduce this currently, but it would be good to debug this further. My suggestions on how to do this would be to create an array of uint64_t and save TSC values (rdtsc()) into it at specific points in the atkbd/syscons console init. You can then print out the deltas between array entries once the console is fully initialized. Moving the rdtsc() calls around should allow one to determine where in the atkbd/syscons init the long pause is happening. There is already a code to detect non-existing AT keyboard and avoid attaching atkbd to it. The code is i386-only at the moment, I am trying to figure out how to modify it so that it works on amd64 as well. -Maxim ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
On Tuesday 20 April 2010 7:35:46 pm Matthew Jacob wrote: > On 04/20/2010 03:44 PM, Maxim Sobolev wrote: > > Maxim Sobolev wrote: > >>> Maybe try adding > >>> > >>> hint.atkbdc.0.disabled="1" > >>> hint.atkbd.0.disabled="1" > >>> > >>> to /boot/device.hints? That has reportedly removed minute-long boot > >>> delays on some Nehalem machines. > >> > >> No, that have not helped at all. I measured the delay - it's about 6 > >> minutes from boot command to the first "smap" message. Do you or > >> anybody else have other ideas? > > > > Actually it helped, thank you very much! The problem was that I have > > had my hints compiled into the kernel itself. > > Me too! I can't reproduce this currently, but it would be good to debug this further. My suggestions on how to do this would be to create an array of uint64_t and save TSC values (rdtsc()) into it at specific points in the atkbd/syscons console init. You can then print out the deltas between array entries once the console is fully initialized. Moving the rdtsc() calls around should allow one to determine where in the atkbd/syscons init the long pause is happening. -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
> On 21.04.2010 10:01, pluknet wrote: > > Hmm.. That's strange to hear. > > We have in production a number of x3650m2: 7.2-R, 7.3-R (all amd64). > > All runs flawlessly. > > I'll try to boot it from head today if that matters. > > It was about 1.5 hour ago when i entered "autoboot" in loader prompt. > It still show slowly rotated dash line at end of > /boot/kernel/kernel text=x | I've seen this happen when there were problems with the serial port, either missing or miss-configured. HTH, danny ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
On 21.04.2010 10:01, pluknet wrote: Hmm.. That's strange to hear. We have in production a number of x3650m2: 7.2-R, 7.3-R (all amd64). All runs flawlessly. I'll try to boot it from head today if that matters. It was about 1.5 hour ago when i entered "autoboot" in loader prompt. It still show slowly rotated dash line at end of /boot/kernel/kernel text=x | -- WBR, Andrey V. Elsukov ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
2010/4/21 Andrey V. Elsukov : > On 21.04.2010 2:44, Maxim Sobolev wrote: >> >> Maxim Sobolev wrote: Maybe try adding hint.atkbdc.0.disabled="1" hint.atkbd.0.disabled="1" >> >> Actually it helped, thank you very much! The problem was that I have had >> my hints compiled into the kernel itself. > > Hi, Maxim. > > I tried to boot 9.0-CURRENT amd64 snapshot on IBM x3650 M2 and seems have > the > same problem, i set hints from loader prompt, but this didn't help. > Can you explain what you did to boot FreeBSD faster? > Hmm.. That's strange to hear. We have in production a number of x3650m2: 7.2-R, 7.3-R (all amd64). All runs flawlessly. I'll try to boot it from head today if that matters. -- wbr, pluknet ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
On 21.04.2010 2:44, Maxim Sobolev wrote: Maxim Sobolev wrote: Maybe try adding hint.atkbdc.0.disabled="1" hint.atkbd.0.disabled="1" Actually it helped, thank you very much! The problem was that I have had my hints compiled into the kernel itself. Hi, Maxim. I tried to boot 9.0-CURRENT amd64 snapshot on IBM x3650 M2 and seems have the same problem, i set hints from loader prompt, but this didn't help. Can you explain what you did to boot FreeBSD faster? -- WBR, Andrey V. Elsukov ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
On Apr 20, 2010, at 5:48 PM, Maxim Sobolev wrote: > John Baldwin wrote: >> On Tuesday 20 April 2010 8:30:42 am Maxim Sobolev wrote: >>> Update: I've discovered that the 7.3 kernels actually boot after some >>> ridiculously long waiting period after the "boot" command (i.e. 10 minutes >>> or even more). I suspect that it might be caused by the memory probing, >>> which as far as I know the FreeBSD does to determine if the physical memory >>> that has been reported by BIOS really exists. >>> >>> Here is what kernel reports about RAM: >>> >>> http://sobomax.sippysoft.com/ScreenShot605.png >>> >>> Any comments? >> Maybe try adding >> hint.atkbdc.0.disabled="1" >> hint.atkbd.0.disabled="1" >> to /boot/device.hints? That has reportedly removed minute-long boot delays >> on some Nehalem machines. > > No, that have not helped at all. I measured the delay - it's about 6 minutes. > Do you or anybody else have other ideas? I am not sure if this helps but here is my case: I had the same problem with FreeBSD 7.0-RELEASE+ and FreeBSD 8-RELEASE running under vmware server 2.0 with Virtual Hardware Version 7. I was seeing 15min waiting period on boot. I added a serial port in vmware and then disabled it in FreeBSD. This fixed the problem some what but sometimes after rebuilding the kernel to latest -STABLE I see long delays not always though. > > Regards, > -- > Maksym Sobolyev > Sippy Software, Inc. > Internet Telephony (VoIP) Experts > T/F: +1-646-651-1110 > Web: http://www.sippysoft.com > MSN: sa...@sippysoft.com > Skype: SippySoft > ___ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
On 04/20/2010 03:44 PM, Maxim Sobolev wrote: Maxim Sobolev wrote: Maybe try adding hint.atkbdc.0.disabled="1" hint.atkbd.0.disabled="1" to /boot/device.hints? That has reportedly removed minute-long boot delays on some Nehalem machines. No, that have not helped at all. I measured the delay - it's about 6 minutes from boot command to the first "smap" message. Do you or anybody else have other ideas? Actually it helped, thank you very much! The problem was that I have had my hints compiled into the kernel itself. Me too! ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
Maxim Sobolev wrote: Maybe try adding hint.atkbdc.0.disabled="1" hint.atkbd.0.disabled="1" to /boot/device.hints? That has reportedly removed minute-long boot delays on some Nehalem machines. No, that have not helped at all. I measured the delay - it's about 6 minutes from boot command to the first "smap" message. Do you or anybody else have other ideas? Actually it helped, thank you very much! The problem was that I have had my hints compiled into the kernel itself. -Maxim ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
John Baldwin wrote: On Tuesday 20 April 2010 8:30:42 am Maxim Sobolev wrote: Update: I've discovered that the 7.3 kernels actually boot after some ridiculously long waiting period after the "boot" command (i.e. 10 minutes or even more). I suspect that it might be caused by the memory probing, which as far as I know the FreeBSD does to determine if the physical memory that has been reported by BIOS really exists. Here is what kernel reports about RAM: http://sobomax.sippysoft.com/ScreenShot605.png Any comments? Maybe try adding hint.atkbdc.0.disabled="1" hint.atkbd.0.disabled="1" to /boot/device.hints? That has reportedly removed minute-long boot delays on some Nehalem machines. No, that have not helped at all. I measured the delay - it's about 6 minutes. Do you or anybody else have other ideas? Regards, -- Maksym Sobolyev Sippy Software, Inc. Internet Telephony (VoIP) Experts T/F: +1-646-651-1110 Web: http://www.sippysoft.com MSN: sa...@sippysoft.com Skype: SippySoft ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
John Baldwin wrote: On Tuesday 20 April 2010 8:30:42 am Maxim Sobolev wrote: Update: I've discovered that the 7.3 kernels actually boot after some ridiculously long waiting period after the "boot" command (i.e. 10 minutes or even more). I suspect that it might be caused by the memory probing, which as far as I know the FreeBSD does to determine if the physical memory that has been reported by BIOS really exists. Here is what kernel reports about RAM: http://sobomax.sippysoft.com/ScreenShot605.png Any comments? Maybe try adding hint.atkbdc.0.disabled="1" hint.atkbd.0.disabled="1" to /boot/device.hints? That has reportedly removed minute-long boot delays on some Nehalem machines. No, that have not helped at all. I measured the delay - it's about 6 minutes from boot command to the first "smap" message. Do you or anybody else have other ideas? -Maxim ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
On Tuesday 20 April 2010 8:30:42 am Maxim Sobolev wrote: > Update: I've discovered that the 7.3 kernels actually boot after some > ridiculously long waiting period after the "boot" command (i.e. 10 > minutes or even more). I suspect that it might be caused by the memory > probing, which as far as I know the FreeBSD does to determine if the > physical memory that has been reported by BIOS really exists. > > Here is what kernel reports about RAM: > > http://sobomax.sippysoft.com/ScreenShot605.png > > Any comments? Maybe try adding hint.atkbdc.0.disabled="1" hint.atkbd.0.disabled="1" to /boot/device.hints? That has reportedly removed minute-long boot delays on some Nehalem machines. -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
Update: I've discovered that the 7.3 kernels actually boot after some ridiculously long waiting period after the "boot" command (i.e. 10 minutes or even more). I suspect that it might be caused by the memory probing, which as far as I know the FreeBSD does to determine if the physical memory that has been reported by BIOS really exists. Here is what kernel reports about RAM: http://sobomax.sippysoft.com/ScreenShot605.png Any comments? -Maxim ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
Maxim Sobolev wrote: the boot command, HEAD - filled in console with funny blinking characters. ...and hanged machine after that as well. -Maxim ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
FreeBSD kernel doesn't boot on FUJITSU PRIMERGY RX200 S5 server
Hi, For the first time in many years, I've stumbled across a server hardware where FreeBSD kernel refuses to boot. It's FUJITSU PRIMERGY RX200 S5 server with 2x Quad core E5520 processors and 16GB of RAM. Linux boots on that hardware just fine. Linux dmesg is available here: http://sobomax.sippysoft.com/RX200.linux.dmesg I tried 7.3-RELEASE and latest HEAD, both amd64 and i386 - neither worked. HEAD and 7.3 behaved little bit differently. 7.3 hanged after the boot command, HEAD - filled in console with funny blinking characters. Screenshots with some debug information - SMAP, boot environment (7.3/amd64): http://sobomax.sippysoft.com/ScreenShot595.png http://sobomax.sippysoft.com/ScreenShot596.png http://sobomax.sippysoft.com/ScreenShot597.png http://sobomax.sippysoft.com/ScreenShot598.png http://sobomax.sippysoft.com/ScreenShot599.png http://sobomax.sippysoft.com/ScreenShot600.png http://sobomax.sippysoft.com/ScreenShot601.png HEAD/amd64 booting (boot -v): http://sobomax.sippysoft.com/ScreenShot603.png I have control over that machine, so that I can run some tests on it, including building and executing debug kernels. The only thing that I don't have access to at the moment is BIOS. Any ideas on how to investigate and fix that problem are highly appreciated. -Maxim ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
