Carson Tan wrote:
Hi Kyle,
Thanks for your great effort on this. It really make sense, and I
really appreciate it.
I have checked your disassembly of the disconnection point, and it
looks the same as mine. But
it's hard to say what's the root cause right now. I am still waiting
for the document from IBM.
Meanwhile, I am trying to find out which previous build of Nevada
works well, as that will be
much easier for me to find the differences.
Any update, I will let you know.
Just for curiosity's sake, I stepped through the same section of code on
S10u8 (output below) and wouldn't you know it, it did disconnect. So,
while I'm not sure what it means, I think it's telling us something. Why
on S10 would it work when running freely outside the debugger, and
disconnect when stepping through the code? And in NV it disconnects
inside and out of the debugger?
All I can think of is a timing issue. Something in the timing of S10
allows it to avoid disconnecting when running full tilt?
Anyway, it's food for thought.
-Kyle
S10u8 booting:
ucode0 is /pseudo/uc...@0
pseudo-device: fssnap0
fssnap0 is /pseudo/fss...@0
pseudo-device: winlock0
winlock0 is /pseudo/winl...@0
pseudo-device: vol0
vol0 is /pseudo/v...@0
pseudo-device: pm0
pm0 is /pseudo/p...@0
pseudo-device: rsm0
rsm0 is /pseudo/r...@0
pseudo-device: pool0
pool0 is /pseudo/p...@0
Hostname: Einstein03
dump on /dev/zvol/dsk/zroot0/dump size 4096 MB
NIS domain name is Engineering.NIS
Loaded modules: [ crypto cpc ptm ufs sppp lofs logindmux md random ]
kmdb: stop at bge`bge_attach
kmdb: target stopped at:
bge`bge_attach: pushq %rbp
[3]> bge_asf_pre_reset_operations:b
[3]> :c
kmdb: stop at bge`bge_asf_pre_reset_operations
kmdb: stop at bge`bge_asf_pre_reset_operations
kmdb: target stopped at:
bge`bge_asf_pre_reset_operations: pushq %rbp
[3]> ::step over
kmdb: target stopped at:
bge`bge_asf_pre_reset_operations+1: movl $0x2,%edx
[3]> ::step over
kmdb: target stopped at:
bge`bge_asf_pre_reset_operations+6: movq %rsp,%rbp
[3]> ::step over
kmdb: target stopped at:
bge`bge_asf_pre_reset_operations+9: pushq %r13
[3]> ::step over
kmdb: target stopped at:
bge`bge_asf_pre_reset_operations+0xb: movl %esi,%r13d
[3]> ::step over
kmdb: target stopped at:
bge`bge_asf_pre_reset_operations+0xe: movl $0xb78,%esi
[3]> ::step over
kmdb: target stopped at:
bge`bge_asf_pre_reset_operations+0x13: pushq %r12
[3]> ::step over
kmdb: target stopped at:
bge`bge_asf_pre_reset_operations+0x15: movq %rdi,%r12
[3]> ::step over
kmdb: target stopped at:
bge`bge_asf_pre_reset_operations+0x18: pushq %rbx
[3]> ::step over
kmdb: target stopped at:
bge`bge_asf_pre_reset_operations+0x19: xorl %ebx,%ebx
[3]> ::step over
kmdb: target stopped at:
bge`bge_asf_pre_reset_operations+0x1b: subq $0x8,%rsp
[3]> ::step over
kmdb: target stopped at:
bge`bge_asf_pre_reset_operations+0x1f: call -0x448f <bge`bge_nic_put32>
[3]> ::step over
kmdb: target stopped at:
bge`bge_asf_pre_reset_operations+0x24: movl $0x6810,%esi
[3]> ::step over
kmdb: target stopped at:
bge`bge_asf_pre_reset_operations+0x29: movq %r12,%rdi
[3]> ::step over
kmdb: target stopped at:
bge`bge_asf_pre_reset_operations+0x2c: call -0x480c <bge`bge_reg_get32>
[3]> ::step over
kmdb: target stopped at:
bge`bge_asf_pre_reset_operations+0x31: movl %eax,%edx
[3]> ::step over
kmdb: target stopped at:
bge`bge_asf_pre_reset_operations+0x33: movl $0x6810,%esi
[3]> ::step over
kmdb: target stopped at:
bge`bge_asf_pre_reset_operations+0x38: movq %r12,%rdi
[3]> ::step over
kmdb: target stopped at:
bge`bge_asf_pre_reset_operations+0x3b: orb $0x40,%dh
[3]> ::step over
kmdb: target stopped at:
bge`bge_asf_pre_reset_operations+0x3e: call -0x47fe <bge`bge_reg_put32>
[3]>
system> console -T blade[3]
SOL is not ready
system>
Thanks again,
Carson
Kyle McDonald wrote:
Carson Tan wrote:
Hi Minskey and Kyle,
Thanks for all your discussion on this.
I found that the SOL session is gone after executing the following
code in bge_asf_pre_reset_operations:
bge_reg_put32(bgep, RX_RISC_EVENT_REG, event | RRER_ASF_EVENT);
Hi,
I've never done driver development, so if I'm way off base just say
so....
The code line you quote above is part of:
5759 event = bge_reg_get32
<http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/io/bge/bge_chip2.c#bge_reg_get32>(bgep,
RX_RISC_EVENT_REG
<http://src.opensolaris.org/source/s?defs=RX_RISC_EVENT_REG&project=/onnv>);
5760 bge_reg_put32
<http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/io/bge/bge_chip2.c#bge_reg_put32>(bgep,
RX_RISC_EVENT_REG
<http://src.opensolaris.org/source/s?defs=RX_RISC_EVENT_REG&project=/onnv>,
event | RRER_ASF_EVENT
<http://src.opensolaris.org/source/s?defs=RRER_ASF_EVENT&project=/onnv>);
Is this section of code atomic?
Can the HW change the register on it's own?
The failure is 100% reproducible, and not intermittent, so I normally
wouldn't consider a race condition right away, but it occurred to me
that any changes to the register between the get and the put would be
lost by this code.
Poking around, I also noticed this function:
574 *void*
575 bge_reg_set32
<http://src.opensolaris.org/source/s?refs=bge_reg_set32&project=/onnv>(bge_t
<http://src.opensolaris.org/source/s?defs=bge_t&project=/onnv> *bgep,
bge_regno_t
<http://src.opensolaris.org/source/s?defs=bge_regno_t&project=/onnv>
regno, uint32_t
<http://src.opensolaris.org/source/s?defs=uint32_t&project=/onnv>
bits <http://src.opensolaris.org/source/s?defs=bits&project=/onnv>)
576 {
577 uint32_t
<http://src.opensolaris.org/source/s?defs=uint32_t&project=/onnv>
regval <http://src.opensolaris.org/source/s?refs=regval&project=/onnv>;
578 579 BGE_TRACE
<http://src.opensolaris.org/source/s?defs=BGE_TRACE&project=/onnv>(("bge_reg_set32($%p,
0x%lx, 0x%x)",
580 (*void* *)bgep, regno, bits
<http://src.opensolaris.org/source/s?defs=bits&project=/onnv>));
581 582 regval = bge_reg_get32
<http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/io/bge/bge_chip2.c#bge_reg_get32>(bgep,
regno);
583 regval |= bits
<http://src.opensolaris.org/source/s?defs=bits&project=/onnv>;
584 bge_reg_put32
<http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/io/bge/bge_chip2.c#bge_reg_put32>(bgep,
regno, regval);
585 }
586
I don't know if it would be any better protected than the existing
code above, but it seems like the code above could have been
re-written as:
bge_reg_set32
<http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/io/bge/bge_chip2.c#bge_reg_put32>(bgep,
RX_RISC_EVENT_REG
<http://src.opensolaris.org/source/s?defs=RX_RISC_EVENT_REG&project=/onnv>,
RRER_ASF_EVENT
<http://src.opensolaris.org/source/s?defs=RRER_ASF_EVENT&project=/onnv>);
Am I missing something?
Also I noticed several parts of bge_main2.c (line 634) and
bge_chip2.c (lines 4367,4714)that specifically mention problems with
the IBM BladeCenter HS20 blade. Nothing discussed there seemed
immediately obvious to me, but since you said the code in the area
that triggers the disconnect hasn't changed since S10, I'm wondering
if any of these areas that mention the HS20 have changed since S10?
Maybe a problem created by a change in one of them doesn't rear it's
head until we get to the code we're all looking at?
-Kyle
_______________________________________________
driver-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/driver-discuss
