[gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs
That is correct, you should just recompile the DTB; no need to recompile gem5 Kind Regards Giacomo > -Original Message- > From: Md Rubel Ahmed via gem5-users > Sent: 31 July 2021 10:05 > To: gem5-users@gem5.org > Cc: Md Rubel Ahmed > Subject: [gem5-users] VExpress_GEM5_V1, Ethernet, and BARs > > Hi, > I was trying to get ethernet working on my FS simulation, but facing this > problem as follows: > > fatal: system.iobus has two ports responding within range > [0x8000:0x8002]: > system.ethernet.pio > system.iocache.cpu_side_port > > > I changed the third word as discussed in the previous emails of this thread > accordingly. > > old 70: <0x0200 0x0 0x0 0x0 0x4000 0x0 0x4000>; new 70: > <0x0200 0x0 0x0 0x0 0x4000 0x0 0x0>; > > > > > How I added the ethernet(in FSconfig.py): > else: > self.pci_ide = IdeController(disks=disks) > pci_devices.append(self.pci_ide) > > # Added ethernet by rubel > > self.ethernet = IGbE_e1000() > pci_devices.append(self.ethernet) > > > > The command I executed: > ./build/ARM/gem5.opt --debug-flags=Drain,VGIC,Ethernet > configs/example/fs.py --kernel=vmlinux --machine- > type=VExpress_GEM5_V1 --dtb- > file=/home/rubel/gem5/system/arm/dt/armv7_gem5_v1_1cpu.dtb --num- > cpus=1 --cpu-type=TimingSimpleCPU --disk-image=gem5_ubuntu16.img -- > caches --l2cache --mem-size=4GB --l1i_size=16kB --l1d_size=64kB -- > l2_size=256kB > > I believe I don't need to REBUILD gem5 for these changes. I am using a very > recent version of gem5(ea7d012c00e857ef999b88a8ec2bde801a1f). > > So my question is, did I interpreted the word thing correctly? Do I need to > rebuild gem5 for these changes? > > Any help is highly appreciated. > > > Regards, > Rubel Ahmed > IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
[gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs
Giacomo, Replacing the third word on line 70 of the dtsi [1] with `0x0` resolved the collision! Aside, running without the `--dtb` argument caused numerous failures (including an inability to enumerate PCI devices) leading up to a panic. Thanks to you both for your input resolving this, I really appreciate it. [1] https://github.com/gem5/gem5/blob/0d703041fcd5d119012b62287695723a2955b408/system/arm/dt/platforms/vexpress_gem5_v1_base.dtsi#L70 -jrb From: Giacomo Travaglini Sent: Friday, January 22, 2021 9:10 PM To: Bohren, Jonathan; Gabe Black Cc: gem5 users mailing list Subject: RE: [gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs I think the problem lies in the DTB and more specifically in the range mapping https://github.com/gem5/gem5/blob/stable/system/arm/dt/platforms/vexpress_gem5_v1_base.dtsi#L70 Could you replace the third word with 0x0? Otherwise you could rely on DTB autogeneration (simply omit the --dtb argument from cmdline) Please let me know if it works Giacomo > -Original Message- > From: Bohren, Jonathan > Sent: 23 January 2021 01:59 > To: Gabe Black > Cc: Giacomo Travaglini ; gem5 users mailing list > > Subject: Re: [gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs > > Yeah, the first time it gets offset in the PciDevice constructor, and the > second > time is in a call to PciDevice::writeConfig() by the PCI host. Does it look > like the > `GenericArmPciHost` used by VExpress_GEM5_V1 (but not VExpress_EMM) > might be the source of the double-offset? > > > > > -jrb > > > > > From: Gabe Black > Sent: Friday, January 22, 2021 8:21 PM > To: Bohren, Jonathan > Cc: Giacomo Travaglini; gem5 users mailing list > Subject: Re: [gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs > > I haven't looked at this closely, but it's normal for the values in the BARs > to be > displaced based on the platform object's idea of where the PCI ranges are in > the address space, ie there is an offset for config space, etc. It sounds to > me > like this offset is being added in twice, one by the kernel, and then once > again > by the PCI controller in the platform. That would not be a bug in the PCI BAR > system, it would be a configuration bug where only one of those should be > happening, probably the controller but I imagine it would depend somewhat. > > Gabe > > On Fri, Jan 22, 2021 at 1:25 PM Bohren, Jonathan > <mailto:jrboh...@honeybeerobotics.com> > wrote: > > > Gabe, > > It looks like after being initialized properly to 0x4000'000 range, the > BAR0 start for the Ethernet device is getting changed from 0x4000' to > 0x8000' in a call to `PciMemBar::write()` [1] once the driver loads. > [1] > https://github.com/gem5/gem5/blob/develop/src/dev/pci/device.hh#L195- > L213 > <https://github.com/gem5/gem5/blob/develop/src/dev/pci/device.hh#L195- > L213> > > Let me know if this conversation should migrate to the gem5-dev list. > > The stack trace below shows from where the call is being made: > ``` > #0 PciMemBar::write (this=, host=..., val= out>) at build/ARM/dev/pci/device.hh:218 > #1 0x57659221 in PciDevice::writeConfig > (this=0x5d8a6000, pkt=0x7fffb540) at build/ARM/dev/pci/device.cc:381 > #2 0x575a500c in IGbE::writeConfig (this=0x5d8a6000, > pkt=) at build/ARM/dev/net/i8254xGBe.cc:152 > #3 0x5765f0b8 in GenericPciHost::write > (this=0x5a048100, pkt=0x7fffb540) at build/ARM/dev/pci/host.cc:175 > #4 0x574870b6 in non-virtual thunk to > PioPort::recvAtomic(Packet*) () > #5 0x57282617 in RequestPort::sendAtomic > (pkt=0x7fffb540, this=0x5d859030) at build/ARM/mem/port.hh:464 > #6 NoncoherentXBar::recvAtomicBackdoor (this=0x5a047e40, > pkt=0x7fffb540, cpu_side_port_id=, backdoor=0x0) at > build/ARM/mem/noncoherent_xbar.cc:264 > #7 0x5725626a in RequestPort::sendAtomic (pkt= out>, this=0x59f67e50) at build/ARM/mem/port.hh:464 > #8 Bridge::BridgeResponsePort::recvAtomic (this=, > pkt=) at build/ARM/mem/bridge.cc:344 > #9 0x5725f897 in RequestPort::sendAtomic > (pkt=0x7fffb540, this=0x5d85aa00) at build/ARM/mem/port.hh:464 > #10 CoherentXBar::recvAtomicBackdoor (this=0x5a016d00, > pkt=, cpu_side_port_id=, > backdoor=) at build/ARM/mem/coherent_xbar.cc:813 > #11 0x57679db4 in RequestPort::sendAtomic > (pkt=0x7fffb540, this=0x59649a50) at build/ARM/mem/port.hh:464 > #12 AtomicSimpleCPU::sendPacket (pkt=@0x7fffb530: > 0x7fffb540, port=..., this=0x59649600) at > build/ARM/cpu/simple/atomic.cc:276 > #13 AtomicSimpleCPU::writeMem (this=, > data=0x7fffb674 "", s
[gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs
I think the problem lies in the DTB and more specifically in the range mapping
https://github.com/gem5/gem5/blob/stable/system/arm/dt/platforms/vexpress_gem5_v1_base.dtsi#L70
Could you replace the third word with 0x0? Otherwise you could rely on DTB
autogeneration (simply omit the --dtb argument from cmdline)
Please let me know if it works
Giacomo
> -Original Message-
> From: Bohren, Jonathan
> Sent: 23 January 2021 01:59
> To: Gabe Black
> Cc: Giacomo Travaglini ; gem5 users mailing list
>
> Subject: Re: [gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs
>
> Yeah, the first time it gets offset in the PciDevice constructor, and the
> second
> time is in a call to PciDevice::writeConfig() by the PCI host. Does it look
> like the
> `GenericArmPciHost` used by VExpress_GEM5_V1 (but not VExpress_EMM)
> might be the source of the double-offset?
>
>
>
>
> -jrb
>
>
>
>
> From: Gabe Black
> Sent: Friday, January 22, 2021 8:21 PM
> To: Bohren, Jonathan
> Cc: Giacomo Travaglini; gem5 users mailing list
> Subject: Re: [gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs
>
> I haven't looked at this closely, but it's normal for the values in the BARs
> to be
> displaced based on the platform object's idea of where the PCI ranges are in
> the address space, ie there is an offset for config space, etc. It sounds to
> me
> like this offset is being added in twice, one by the kernel, and then once
> again
> by the PCI controller in the platform. That would not be a bug in the PCI BAR
> system, it would be a configuration bug where only one of those should be
> happening, probably the controller but I imagine it would depend somewhat.
>
> Gabe
>
> On Fri, Jan 22, 2021 at 1:25 PM Bohren, Jonathan
> <mailto:jrboh...@honeybeerobotics.com> > wrote:
>
>
> Gabe,
>
> It looks like after being initialized properly to 0x4000'000 range, the
> BAR0 start for the Ethernet device is getting changed from 0x4000' to
> 0x8000' in a call to `PciMemBar::write()` [1] once the driver loads.
> [1]
> https://github.com/gem5/gem5/blob/develop/src/dev/pci/device.hh#L195-
> L213
> <https://github.com/gem5/gem5/blob/develop/src/dev/pci/device.hh#L195-
> L213>
>
> Let me know if this conversation should migrate to the gem5-dev list.
>
> The stack trace below shows from where the call is being made:
> ```
> #0 PciMemBar::write (this=, host=..., val= out>) at build/ARM/dev/pci/device.hh:218
> #1 0x57659221 in PciDevice::writeConfig
> (this=0x5d8a6000, pkt=0x7fffb540) at build/ARM/dev/pci/device.cc:381
> #2 0x575a500c in IGbE::writeConfig (this=0x5d8a6000,
> pkt=) at build/ARM/dev/net/i8254xGBe.cc:152
> #3 0x5765f0b8 in GenericPciHost::write
> (this=0x5a048100, pkt=0x7fffb540) at build/ARM/dev/pci/host.cc:175
> #4 0x574870b6 in non-virtual thunk to
> PioPort::recvAtomic(Packet*) ()
> #5 0x57282617 in RequestPort::sendAtomic
> (pkt=0x7fffb540, this=0x5d859030) at build/ARM/mem/port.hh:464
> #6 NoncoherentXBar::recvAtomicBackdoor (this=0x5a047e40,
> pkt=0x7fffb540, cpu_side_port_id=, backdoor=0x0) at
> build/ARM/mem/noncoherent_xbar.cc:264
> #7 0x5725626a in RequestPort::sendAtomic (pkt= out>, this=0x59f67e50) at build/ARM/mem/port.hh:464
> #8 Bridge::BridgeResponsePort::recvAtomic (this=,
> pkt=) at build/ARM/mem/bridge.cc:344
> #9 0x5725f897 in RequestPort::sendAtomic
> (pkt=0x7fffb540, this=0x5d85aa00) at build/ARM/mem/port.hh:464
> #10 CoherentXBar::recvAtomicBackdoor (this=0x5a016d00,
> pkt=, cpu_side_port_id=,
> backdoor=) at build/ARM/mem/coherent_xbar.cc:813
> #11 0x57679db4 in RequestPort::sendAtomic
> (pkt=0x7fffb540, this=0x59649a50) at build/ARM/mem/port.hh:464
> #12 AtomicSimpleCPU::sendPacket (pkt=@0x7fffb530:
> 0x7fffb540, port=..., this=0x59649600) at
> build/ARM/cpu/simple/atomic.cc:276
> #13 AtomicSimpleCPU::writeMem (this=,
> data=0x7fffb674 "", size=, addr=, flags=...,
> res=, byte_enable=...) at build/ARM/cpu/simple/atomic.cc:505
> #14 0x5768bb02 in SimpleExecContext::writeMem
> (this=, data=, size=,
> addr=, flags=..., res=0x0, byte_enable=std::vector of
> length 4, capacity 64 = {...})
> at build/ARM/cpu/simple/exec_context.hh:553
> #15 0x569d137d in writeMemAtomic
> (byte_enable=std::vector of length 4, capacity 64 = {...}, res=0x0,
> flags=..., size=4, addr=269008, mem=0x7fffb674 "", xc=0x59d856c0)
> at build/ARM/arch/generic/memhelpers.hh:190
> #16 writeMemAtomic<(ByteOrder)1, ExecContext, unsigned int>
> (xc=xc@entry=0x59d8
[gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs
Yeah, the first time it gets offset in the PciDevice constructor, and the
second time is in a call to PciDevice::writeConfig() by the PCI host. Does it
look like the `GenericArmPciHost` used by VExpress_GEM5_V1 (but not
VExpress_EMM) might be the source of the double-offset?
-jrb
From: Gabe Black
Sent: Friday, January 22, 2021 8:21 PM
To: Bohren, Jonathan
Cc: Giacomo Travaglini; gem5 users mailing list
Subject: Re: [gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs
I haven't looked at this closely, but it's normal for the values in the BARs to
be displaced based on the platform object's idea of where the PCI ranges are in
the address space, ie there is an offset for config space, etc. It sounds to me
like this offset is being added in twice, one by the kernel, and then once
again by the PCI controller in the platform. That would not be a bug in the PCI
BAR system, it would be a configuration bug where only one of those should be
happening, probably the controller but I imagine it would depend somewhat.
Gabe
On Fri, Jan 22, 2021 at 1:25 PM Bohren, Jonathan
mailto:jrboh...@honeybeerobotics.com>> wrote:
Gabe,
It looks like after being initialized properly to 0x4000'000 range, the BAR0
start for the Ethernet device is getting changed from 0x4000' to
0x8000' in a call to `PciMemBar::write()` [1] once the driver loads.
[1] https://github.com/gem5/gem5/blob/develop/src/dev/pci/device.hh#L195-L213
Let me know if this conversation should migrate to the gem5-dev list.
The stack trace below shows from where the call is being made:
```
#0 PciMemBar::write (this=, host=..., val=) at
build/ARM/dev/pci/device.hh:218
#1 0x57659221 in PciDevice::writeConfig (this=0x5d8a6000,
pkt=0x7fffb540) at build/ARM/dev/pci/device.cc:381
#2 0x575a500c in IGbE::writeConfig (this=0x5d8a6000,
pkt=) at build/ARM/dev/net/i8254xGBe.cc:152
#3 0x5765f0b8 in GenericPciHost::write (this=0x5a048100,
pkt=0x7fffb540) at build/ARM/dev/pci/host.cc:175
#4 0x574870b6 in non-virtual thunk to
PioPort::recvAtomic(Packet*) ()
#5 0x57282617 in RequestPort::sendAtomic (pkt=0x7fffb540,
this=0x5d859030) at build/ARM/mem/port.hh:464
#6 NoncoherentXBar::recvAtomicBackdoor (this=0x5a047e40,
pkt=0x7fffb540, cpu_side_port_id=, backdoor=0x0) at
build/ARM/mem/noncoherent_xbar.cc:264
#7 0x5725626a in RequestPort::sendAtomic (pkt=,
this=0x59f67e50) at build/ARM/mem/port.hh:464
#8 Bridge::BridgeResponsePort::recvAtomic (this=,
pkt=) at build/ARM/mem/bridge.cc:344
#9 0x5725f897 in RequestPort::sendAtomic (pkt=0x7fffb540,
this=0x5d85aa00) at build/ARM/mem/port.hh:464
#10 CoherentXBar::recvAtomicBackdoor (this=0x5a016d00, pkt=,
cpu_side_port_id=, backdoor=) at
build/ARM/mem/coherent_xbar.cc:813
#11 0x57679db4 in RequestPort::sendAtomic (pkt=0x7fffb540,
this=0x59649a50) at build/ARM/mem/port.hh:464
#12 AtomicSimpleCPU::sendPacket (pkt=@0x7fffb530: 0x7fffb540, port=...,
this=0x59649600) at build/ARM/cpu/simple/atomic.cc:276
#13 AtomicSimpleCPU::writeMem (this=, data=0x7fffb674 "",
size=, addr=, flags=..., res=,
byte_enable=...) at build/ARM/cpu/simple/atomic.cc:505
#14 0x5768bb02 in SimpleExecContext::writeMem (this=,
data=, size=, addr=, flags=...,
res=0x0, byte_enable=std::vector of length 4, capacity 64 = {...})
at build/ARM/cpu/simple/exec_context.hh:553
#15 0x569d137d in writeMemAtomic
(byte_enable=std::vector of length 4, capacity 64 = {...}, res=0x0,
flags=..., size=4, addr=269008, mem=0x7fffb674 "", xc=0x59d856c0)
at build/ARM/arch/generic/memhelpers.hh:190
#16 writeMemAtomic<(ByteOrder)1, ExecContext, unsigned int>
(xc=xc@entry=0x59d856c0, traceData=traceData@entry=0x0,
mem=@0x7fffb6ec: 1073741824, addr=addr@entry=269008, flags=...,
flags@entry=..., res=0x0)
at build/ARM/arch/generic/memhelpers.hh:203
#17 0x56477426 in writeMemAtomicLE (res=0x0,
flags=..., addr=269008, mem=@0x7fffb6ec: 1073741824, traceData=0x0,
xc=0x59d856c0) at build/ARM/arch/generic/memhelpers.hh:220
#18 ArmISAInst::STORE_IMM_PN_AY_WN_SN_UN_SZ4::execute (this=,
xc=0x59d856c0, traceData=0x0) at
build/ARM/arch/arm/generated/exec-ns.cc.inc:21605
#19 0x576770dc in AtomicSimpleCPU::tick (this=0x59649600) at
build/ARM/cpu/simple/atomic.cc:685
#20 0x5715ffcd in std::function::operator()() const
(this=0x596499a0) at /usr/include/c++/7/bits/std_function.h:706
#21 EventFunctionWrapper::process (this=0x59649968) at
build/ARM/sim/eventq.hh:1138
#22 EventQueue::serviceOne (this=this@entry=0x5900b200) at
build/ARM/sim/eventq.cc:223
#23 0x57181cd0 in doSimLoop (eventq=0x5900b200) at
build/ARM/sim/simulate.cc:216
#24 0x57182eda in simulate (num_cycles=) at
build/ARM/sim/simulate.cc:129
```
-jrb
From:
[gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs
I haven't looked at this closely, but it's normal for the values in the
BARs to be displaced based on the platform object's idea of where the PCI
ranges are in the address space, ie there is an offset for config space,
etc. It sounds to me like this offset is being added in twice, one by the
kernel, and then once again by the PCI controller in the platform. That
would not be a bug in the PCI BAR system, it would be a configuration bug
where only one of those should be happening, probably the controller but I
imagine it would depend somewhat.
Gabe
On Fri, Jan 22, 2021 at 1:25 PM Bohren, Jonathan <
jrboh...@honeybeerobotics.com> wrote:
> Gabe,
>
> It looks like after being initialized properly to 0x4000'000 range, the
> BAR0 start for the Ethernet device is getting changed from 0x4000' to
> 0x8000' in a call to `PciMemBar::write()` [1] once the driver loads.
> [1]
> https://github.com/gem5/gem5/blob/develop/src/dev/pci/device.hh#L195-L213
>
> Let me know if this conversation should migrate to the gem5-dev list.
>
> The stack trace below shows from where the call is being made:
> ```
> #0 PciMemBar::write (this=, host=..., val=)
> at build/ARM/dev/pci/device.hh:218
> #1 0x57659221 in PciDevice::writeConfig (this=0x5d8a6000,
> pkt=0x7fffb540) at build/ARM/dev/pci/device.cc:381
> #2 0x575a500c in IGbE::writeConfig (this=0x5d8a6000,
> pkt=) at build/ARM/dev/net/i8254xGBe.cc:152
> #3 0x5765f0b8 in GenericPciHost::write (this=0x5a048100,
> pkt=0x7fffb540) at build/ARM/dev/pci/host.cc:175
> #4 0x574870b6 in non-virtual thunk to
> PioPort::recvAtomic(Packet*) ()
> #5 0x57282617 in RequestPort::sendAtomic (pkt=0x7fffb540,
> this=0x5d859030) at build/ARM/mem/port.hh:464
> #6 NoncoherentXBar::recvAtomicBackdoor (this=0x5a047e40,
> pkt=0x7fffb540, cpu_side_port_id=, backdoor=0x0) at
> build/ARM/mem/noncoherent_xbar.cc:264
> #7 0x5725626a in RequestPort::sendAtomic (pkt=,
> this=0x59f67e50) at build/ARM/mem/port.hh:464
> #8 Bridge::BridgeResponsePort::recvAtomic (this=,
> pkt=) at build/ARM/mem/bridge.cc:344
> #9 0x5725f897 in RequestPort::sendAtomic (pkt=0x7fffb540,
> this=0x5d85aa00) at build/ARM/mem/port.hh:464
> #10 CoherentXBar::recvAtomicBackdoor (this=0x5a016d00, pkt= out>, cpu_side_port_id=, backdoor=) at
> build/ARM/mem/coherent_xbar.cc:813
> #11 0x57679db4 in RequestPort::sendAtomic (pkt=0x7fffb540,
> this=0x59649a50) at build/ARM/mem/port.hh:464
> #12 AtomicSimpleCPU::sendPacket (pkt=@0x7fffb530: 0x7fffb540,
> port=..., this=0x59649600) at build/ARM/cpu/simple/atomic.cc:276
> #13 AtomicSimpleCPU::writeMem (this=, data=0x7fffb674
> "", size=, addr=, flags=..., res= out>, byte_enable=...) at build/ARM/cpu/simple/atomic.cc:505
> #14 0x5768bb02 in SimpleExecContext::writeMem (this= out>, data=, size=, addr=,
> flags=..., res=0x0, byte_enable=std::vector of length 4, capacity 64
> = {...})
> at build/ARM/cpu/simple/exec_context.hh:553
> #15 0x569d137d in writeMemAtomic
> (byte_enable=std::vector of length 4, capacity 64 = {...}, res=0x0,
> flags=..., size=4, addr=269008, mem=0x7fffb674 "",
> xc=0x59d856c0)
> at build/ARM/arch/generic/memhelpers.hh:190
> #16 writeMemAtomic<(ByteOrder)1, ExecContext, unsigned int>
> (xc=xc@entry=0x59d856c0,
> traceData=traceData@entry=0x0, mem=@0x7fffb6ec: 1073741824,
> addr=addr@entry=269008, flags=..., flags@entry=..., res=0x0)
> at build/ARM/arch/generic/memhelpers.hh:203
> #17 0x56477426 in writeMemAtomicLE
> (res=0x0, flags=..., addr=269008, mem=@0x7fffb6ec: 1073741824,
> traceData=0x0, xc=0x59d856c0) at
> build/ARM/arch/generic/memhelpers.hh:220
> #18 ArmISAInst::STORE_IMM_PN_AY_WN_SN_UN_SZ4::execute (this= out>, xc=0x59d856c0, traceData=0x0) at
> build/ARM/arch/arm/generated/exec-ns.cc.inc:21605
> #19 0x576770dc in AtomicSimpleCPU::tick (this=0x59649600) at
> build/ARM/cpu/simple/atomic.cc:685
> #20 0x5715ffcd in std::function::operator()() const
> (this=0x596499a0) at /usr/include/c++/7/bits/std_function.h:706
> #21 EventFunctionWrapper::process (this=0x59649968) at
> build/ARM/sim/eventq.hh:1138
> #22 EventQueue::serviceOne (this=this@entry=0x5900b200) at
> build/ARM/sim/eventq.cc:223
> #23 0x57181cd0 in doSimLoop (eventq=0x5900b200) at
> build/ARM/sim/simulate.cc:216
> #24 0x57182eda in simulate (num_cycles=) at
> build/ARM/sim/simulate.cc:129
> ```
>
> -jrb
>
>
>
>
>
>
>
> From: Bohren, Jonathan via gem5-users
> Sent: Friday, January 22, 2021 10:07 AM
> To: gem5 users mailing list; G
[gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs
Gabe,
It looks like after being initialized properly to 0x4000'000 range, the BAR0
start for the Ethernet device is getting changed from 0x4000' to
0x8000' in a call to `PciMemBar::write()` [1] once the driver loads.
[1] https://github.com/gem5/gem5/blob/develop/src/dev/pci/device.hh#L195-L213
Let me know if this conversation should migrate to the gem5-dev list.
The stack trace below shows from where the call is being made:
```
#0 PciMemBar::write (this=, host=..., val=) at
build/ARM/dev/pci/device.hh:218
#1 0x57659221 in PciDevice::writeConfig (this=0x5d8a6000,
pkt=0x7fffb540) at build/ARM/dev/pci/device.cc:381
#2 0x575a500c in IGbE::writeConfig (this=0x5d8a6000,
pkt=) at build/ARM/dev/net/i8254xGBe.cc:152
#3 0x5765f0b8 in GenericPciHost::write (this=0x5a048100,
pkt=0x7fffb540) at build/ARM/dev/pci/host.cc:175
#4 0x574870b6 in non-virtual thunk to
PioPort::recvAtomic(Packet*) ()
#5 0x57282617 in RequestPort::sendAtomic (pkt=0x7fffb540,
this=0x5d859030) at build/ARM/mem/port.hh:464
#6 NoncoherentXBar::recvAtomicBackdoor (this=0x5a047e40,
pkt=0x7fffb540, cpu_side_port_id=, backdoor=0x0) at
build/ARM/mem/noncoherent_xbar.cc:264
#7 0x5725626a in RequestPort::sendAtomic (pkt=,
this=0x59f67e50) at build/ARM/mem/port.hh:464
#8 Bridge::BridgeResponsePort::recvAtomic (this=,
pkt=) at build/ARM/mem/bridge.cc:344
#9 0x5725f897 in RequestPort::sendAtomic (pkt=0x7fffb540,
this=0x5d85aa00) at build/ARM/mem/port.hh:464
#10 CoherentXBar::recvAtomicBackdoor (this=0x5a016d00, pkt=,
cpu_side_port_id=, backdoor=) at
build/ARM/mem/coherent_xbar.cc:813
#11 0x57679db4 in RequestPort::sendAtomic (pkt=0x7fffb540,
this=0x59649a50) at build/ARM/mem/port.hh:464
#12 AtomicSimpleCPU::sendPacket (pkt=@0x7fffb530: 0x7fffb540, port=...,
this=0x59649600) at build/ARM/cpu/simple/atomic.cc:276
#13 AtomicSimpleCPU::writeMem (this=, data=0x7fffb674 "",
size=, addr=, flags=..., res=,
byte_enable=...) at build/ARM/cpu/simple/atomic.cc:505
#14 0x5768bb02 in SimpleExecContext::writeMem (this=,
data=, size=, addr=, flags=...,
res=0x0, byte_enable=std::vector of length 4, capacity 64 = {...})
at build/ARM/cpu/simple/exec_context.hh:553
#15 0x569d137d in writeMemAtomic
(byte_enable=std::vector of length 4, capacity 64 = {...}, res=0x0,
flags=..., size=4, addr=269008, mem=0x7fffb674 "", xc=0x59d856c0)
at build/ARM/arch/generic/memhelpers.hh:190
#16 writeMemAtomic<(ByteOrder)1, ExecContext, unsigned int>
(xc=xc@entry=0x59d856c0, traceData=traceData@entry=0x0,
mem=@0x7fffb6ec: 1073741824, addr=addr@entry=269008, flags=...,
flags@entry=..., res=0x0)
at build/ARM/arch/generic/memhelpers.hh:203
#17 0x56477426 in writeMemAtomicLE (res=0x0,
flags=..., addr=269008, mem=@0x7fffb6ec: 1073741824, traceData=0x0,
xc=0x59d856c0) at build/ARM/arch/generic/memhelpers.hh:220
#18 ArmISAInst::STORE_IMM_PN_AY_WN_SN_UN_SZ4::execute (this=,
xc=0x59d856c0, traceData=0x0) at
build/ARM/arch/arm/generated/exec-ns.cc.inc:21605
#19 0x576770dc in AtomicSimpleCPU::tick (this=0x59649600) at
build/ARM/cpu/simple/atomic.cc:685
#20 0x5715ffcd in std::function::operator()() const
(this=0x596499a0) at /usr/include/c++/7/bits/std_function.h:706
#21 EventFunctionWrapper::process (this=0x59649968) at
build/ARM/sim/eventq.hh:1138
#22 EventQueue::serviceOne (this=this@entry=0x5900b200) at
build/ARM/sim/eventq.cc:223
#23 0x57181cd0 in doSimLoop (eventq=0x5900b200) at
build/ARM/sim/simulate.cc:216
#24 0x57182eda in simulate (num_cycles=) at
build/ARM/sim/simulate.cc:129
```
-jrb
From: Bohren, Jonathan via gem5-users
Sent: Friday, January 22, 2021 10:07 AM
To: gem5 users mailing list; Giacomo Travaglini
Cc: Gabe Black; Bohren, Jonathan
Subject: [gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs
Giacomo,
That's good to know.
We're running `fs.py` with `system/arm/dt/armv7_gem5_v1_1cpu.dtb` which we've
copied into our working directory.
Full invocation:
```
./gem5/build/ARM/gem5.opt \
gem5/configs/example/fs.py \
--bootloader ./boot.arm \
--kernel=./vmlinux \
--machine-type=VExpress_GEM5_V1 \
--dtb-file=./armv7_gem5_v1_1cpu.dtb \
--disk-image=../image/ubuntu-base-18.04.5-base-armhf.img \
--root-device=/dev/sda
```
-jrb
From: Giacomo Travaglini
Sent: Friday, January 22, 2021 8:16:48 AM
To: gem5 users mailing list
Cc: Gabe Black ; Bohren, Jonathan
Subject: RE: [gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs
Out of curiosity, which DTB are you using?
IMO the kernel shouldn't write 800 to the PCI Bar, knowing there is DRAM
there and this is notified via the DTB
Kind Regards
Giacomo
> -Original Message-
> From: Bohren, Jonathan via gem5-users
> Sent: 22 January 2021 12:51
> T
[gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs
Giacomo, That's good to know. We're running `fs.py` with `system/arm/dt/armv7_gem5_v1_1cpu.dtb` which we've copied into our working directory. Full invocation: ``` ./gem5/build/ARM/gem5.opt \ gem5/configs/example/fs.py<http://fs.py> \ --bootloader ./boot.arm \ --kernel=./vmlinux \ --machine-type=VExpress_GEM5_V1 \ --dtb-file=./armv7_gem5_v1_1cpu.dtb \ --disk-image=../image/ubuntu-base-18.04.5-base-armhf.img \ --root-device=/dev/sda ``` -jrb From: Giacomo Travaglini Sent: Friday, January 22, 2021 8:16:48 AM To: gem5 users mailing list Cc: Gabe Black ; Bohren, Jonathan Subject: RE: [gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs Out of curiosity, which DTB are you using? IMO the kernel shouldn't write 800 to the PCI Bar, knowing there is DRAM there and this is notified via the DTB Kind Regards Giacomo > -Original Message- > From: Bohren, Jonathan via gem5-users > Sent: 22 January 2021 12:51 > To: gem5 users mailing list > Cc: Gabe Black ; Bohren, Jonathan > > Subject: [gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs > > Gabe, > > > > Yeah, this is a very recent version of `develop`. This is after rebasing a > recent > proposed change `38796` [1] (which disables the HDLcd) onto `develop` HEAD > from 2020-01-20: > > > > ``` > > c6e3fd79f (develop-rebase-38796) dev-arm, system-arm: Remove HDLcd from > VExpress_GEM5_VX platforms > > b9ce8848e system-arm: Move display node into a shared DTS file > > 71932a360 (upstream/develop, develop) dev: Fixing EtherDevice stats > initialization order > > ``` > > > > The two changesets in CL `38796` [1] were based on `3059c6d` which was the > HEAD of `develop` on 2020-01-05, and it didn’t appear that CL `38796` reverted > any changes made in the interim. > > > > I came across a similar discussion in the archives [2], which did seem > similar, > but didn’t address the problem described below. Is the bug you’re referring to > the one addressed by CL `35516` [3]? > > > > Thanks for taking a look at this! > > > > [1] https://gem5-review.googlesource.com/c/public/gem5/+/38796 > <https://gem5-review.googlesource.com/c/public/gem5/+/38796> > > [2] https://www.mail-archive.com/gem5-users@gem5.org/msg18647.html > <https://www.mail-archive.com/gem5-users@gem5.org/msg18647.html> > > [3] https://gem5-review.googlesource.com/c/public/gem5/+/35516 > <https://gem5-review.googlesource.com/c/public/gem5/+/35516> > > > > -jrb > > > > From: Gabe Black via gem5-users > Sent: Thursday, January 21, 2021 11:59 PM > To: gem5 users mailing list > Cc: Gabe Black > Subject: [gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs > > > > Are you using up to date develop? There was a bug like this a while ago, but > it's > been fixed on develop for a while as well. > > > > Gabe > > > > On Thu, Jan 21, 2021 at 6:35 PM Bohren, Jonathan via gem5-users us...@gem5.org <mailto:gem5-users@gem5.org> > wrote: > > We've been using the `VExpress_GEM5_V1` platform but it was failing > on both `stable` and `develop` when adding an `IGbE_e1000` PCI device to the > `makeArmSystem()` function in `FSConfig.py`. We've worked around it, but > looking for the "right way" to add such a device. > > For reference, the change is shown here: > ``` > diff --git a/configs/common/FSConfig.py > b/configs/common/FSConfig.py > index 6fd39a5aa..837d95982 100644 > --- a/configs/common/FSConfig.py > +++ b/configs/common/FSConfig.py > @@ -226,6 +226,8 @@ def makeArmSystem(mem_mode, > machine_type, num_cpus=1, mdesc=None, > else: > self.pci_ide = IdeController(disks=disks) > pci_devices.append(self.pci_ide) > +self.realview.ethernet = IGbE_e1000() > +pci_devices.append(self.realview.ethernet) > > self.mem_ranges = [] > size_remain = long(Addr(mdesc.mem())) > ``` > > Specifically, the error that results during boot is regarding an address > range collision once the device loads: > ``` > fatal: system.iobus has two ports responding within range > [0x8000:0x8002]: > system.realview.ethernet.pio > system.iobridge.cpu_side_port > ``` > > That `0x8000:0x8002` range corresponds exactly to the 128kB > BAR0 of the `IGbE` device. > > The recent change [1] makes it clear that this is an issue because the > `IGbE_e1000` device uses a `PciMemBar`, which overlaps with the Realview's > DRAM address range, while the pre-existing `IdeController` device only uses > `PciIoBar` BARs, which don't. It's possible to avoid the range collision by > changing the Ethernet device's `PciMemBar` t
[gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs
Out of curiosity, which DTB are you using? IMO the kernel shouldn't write 800 to the PCI Bar, knowing there is DRAM there and this is notified via the DTB Kind Regards Giacomo > -Original Message- > From: Bohren, Jonathan via gem5-users > Sent: 22 January 2021 12:51 > To: gem5 users mailing list > Cc: Gabe Black ; Bohren, Jonathan > > Subject: [gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs > > Gabe, > > > > Yeah, this is a very recent version of `develop`. This is after rebasing a > recent > proposed change `38796` [1] (which disables the HDLcd) onto `develop` HEAD > from 2020-01-20: > > > > ``` > > c6e3fd79f (develop-rebase-38796) dev-arm, system-arm: Remove HDLcd from > VExpress_GEM5_VX platforms > > b9ce8848e system-arm: Move display node into a shared DTS file > > 71932a360 (upstream/develop, develop) dev: Fixing EtherDevice stats > initialization order > > ``` > > > > The two changesets in CL `38796` [1] were based on `3059c6d` which was the > HEAD of `develop` on 2020-01-05, and it didn’t appear that CL `38796` reverted > any changes made in the interim. > > > > I came across a similar discussion in the archives [2], which did seem > similar, > but didn’t address the problem described below. Is the bug you’re referring to > the one addressed by CL `35516` [3]? > > > > Thanks for taking a look at this! > > > > [1] https://gem5-review.googlesource.com/c/public/gem5/+/38796 > <https://gem5-review.googlesource.com/c/public/gem5/+/38796> > > [2] https://www.mail-archive.com/gem5-users@gem5.org/msg18647.html > <https://www.mail-archive.com/gem5-users@gem5.org/msg18647.html> > > [3] https://gem5-review.googlesource.com/c/public/gem5/+/35516 > <https://gem5-review.googlesource.com/c/public/gem5/+/35516> > > > > -jrb > > > > From: Gabe Black via gem5-users > Sent: Thursday, January 21, 2021 11:59 PM > To: gem5 users mailing list > Cc: Gabe Black > Subject: [gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs > > > > Are you using up to date develop? There was a bug like this a while ago, but > it's > been fixed on develop for a while as well. > > > > Gabe > > > > On Thu, Jan 21, 2021 at 6:35 PM Bohren, Jonathan via gem5-users us...@gem5.org <mailto:gem5-users@gem5.org> > wrote: > > We've been using the `VExpress_GEM5_V1` platform but it was failing > on both `stable` and `develop` when adding an `IGbE_e1000` PCI device to the > `makeArmSystem()` function in `FSConfig.py`. We've worked around it, but > looking for the "right way" to add such a device. > > For reference, the change is shown here: > ``` > diff --git a/configs/common/FSConfig.py > b/configs/common/FSConfig.py > index 6fd39a5aa..837d95982 100644 > --- a/configs/common/FSConfig.py > +++ b/configs/common/FSConfig.py > @@ -226,6 +226,8 @@ def makeArmSystem(mem_mode, > machine_type, num_cpus=1, mdesc=None, > else: > self.pci_ide = IdeController(disks=disks) > pci_devices.append(self.pci_ide) > +self.realview.ethernet = IGbE_e1000() > +pci_devices.append(self.realview.ethernet) > > self.mem_ranges = [] > size_remain = long(Addr(mdesc.mem())) > ``` > > Specifically, the error that results during boot is regarding an address > range collision once the device loads: > ``` > fatal: system.iobus has two ports responding within range > [0x8000:0x8002]: > system.realview.ethernet.pio > system.iobridge.cpu_side_port > ``` > > That `0x8000:0x8002` range corresponds exactly to the 128kB > BAR0 of the `IGbE` device. > > The recent change [1] makes it clear that this is an issue because the > `IGbE_e1000` device uses a `PciMemBar`, which overlaps with the Realview's > DRAM address range, while the pre-existing `IdeController` device only uses > `PciIoBar` BARs, which don't. It's possible to avoid the range collision by > changing the Ethernet device's `PciMemBar` to a `PciIoBar` to keep it out of > the > DRAM range, but it's not yet clear if that workaround will cause other issues. > > What's the proper way to avoid this collision? > > [1] https://gem5-review.googlesource.com/c/public/gem5/+/35516 > <https://gem5-review.googlesource.com/c/public/gem5/+/35516> > > -jrb > > > > DISCLAIMER: > This e-mail may contain ITAR controlled technical data as defined by 22 > CFR 120.10 and may not be forwarded or disclosed to any Foreign Person, as > defined at 22 CFR 120.16, without the prior written consent of Honeybee > Robotics and the United States D
[gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs
Gabe, Yeah, this is a very recent version of `develop`. This is after rebasing a recent proposed change `38796` [1] (which disables the HDLcd) onto `develop` HEAD from 2020-01-20: ``` c6e3fd79f (develop-rebase-38796) dev-arm, system-arm: Remove HDLcd from VExpress_GEM5_VX platforms b9ce8848e system-arm: Move display node into a shared DTS file 71932a360 (upstream/develop, develop) dev: Fixing EtherDevice stats initialization order ``` The two changesets in CL `38796` [1] were based on `3059c6d` which was the HEAD of `develop` on 2020-01-05, and it didn’t appear that CL `38796` reverted any changes made in the interim. I came across a similar discussion in the archives [2], which did seem similar, but didn’t address the problem described below. Is the bug you’re referring to the one addressed by CL `35516` [3]? Thanks for taking a look at this! [1] https://gem5-review.googlesource.com/c/public/gem5/+/38796 [2] https://www.mail-archive.com/gem5-users@gem5.org/msg18647.html [3] https://gem5-review.googlesource.com/c/public/gem5/+/35516 -jrb From: Gabe Black via gem5-users Sent: Thursday, January 21, 2021 11:59 PM To: gem5 users mailing list Cc: Gabe Black Subject: [gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs Are you using up to date develop? There was a bug like this a while ago, but it's been fixed on develop for a while as well. Gabe On Thu, Jan 21, 2021 at 6:35 PM Bohren, Jonathan via gem5-users mailto:gem5-users@gem5.org>> wrote: We've been using the `VExpress_GEM5_V1` platform but it was failing on both `stable` and `develop` when adding an `IGbE_e1000` PCI device to the `makeArmSystem()` function in `FSConfig.py`. We've worked around it, but looking for the "right way" to add such a device. For reference, the change is shown here: ``` diff --git a/configs/common/FSConfig.py b/configs/common/FSConfig.py index 6fd39a5aa..837d95982 100644 --- a/configs/common/FSConfig.py +++ b/configs/common/FSConfig.py @@ -226,6 +226,8 @@ def makeArmSystem(mem_mode, machine_type, num_cpus=1, mdesc=None, else: self.pci_ide = IdeController(disks=disks) pci_devices.append(self.pci_ide) +self.realview.ethernet = IGbE_e1000() +pci_devices.append(self.realview.ethernet) self.mem_ranges = [] size_remain = long(Addr(mdesc.mem())) ``` Specifically, the error that results during boot is regarding an address range collision once the device loads: ``` fatal: system.iobus has two ports responding within range [0x8000:0x8002]: system.realview.ethernet.pio system.iobridge.cpu_side_port ``` That `0x8000:0x8002` range corresponds exactly to the 128kB BAR0 of the `IGbE` device. The recent change [1] makes it clear that this is an issue because the `IGbE_e1000` device uses a `PciMemBar`, which overlaps with the Realview's DRAM address range, while the pre-existing `IdeController` device only uses `PciIoBar` BARs, which don't. It's possible to avoid the range collision by changing the Ethernet device's `PciMemBar` to a `PciIoBar` to keep it out of the DRAM range, but it's not yet clear if that workaround will cause other issues. What's the proper way to avoid this collision? [1] https://gem5-review.googlesource.com/c/public/gem5/+/35516 -jrb DISCLAIMER: This e-mail may contain ITAR controlled technical data as defined by 22 CFR 120.10 and may not be forwarded or disclosed to any Foreign Person, as defined at 22 CFR 120.16, without the prior written consent of Honeybee Robotics and the United States Department of State. ___ gem5-users mailing list -- gem5-users@gem5.org<mailto:gem5-users@gem5.org> To unsubscribe send an email to gem5-users-le...@gem5.org<mailto:gem5-users-le...@gem5.org> %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s DISCLAIMER: This e-mail may contain ITAR controlled technical data as defined by 22 CFR 120.10 and may not be forwarded or disclosed to any Foreign Person, as defined at 22 CFR 120.16, without the prior written consent of Honeybee Robotics and the United States Department of State. ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
[gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs
Are you using up to date develop? There was a bug like this a while ago, but it's been fixed on develop for a while as well. Gabe On Thu, Jan 21, 2021 at 6:35 PM Bohren, Jonathan via gem5-users < gem5-users@gem5.org> wrote: > We've been using the `VExpress_GEM5_V1` platform but it was failing on > both `stable` and `develop` when adding an `IGbE_e1000` PCI device to the > `makeArmSystem()` function in `FSConfig.py`. We've worked around it, but > looking for the "right way" to add such a device. > > For reference, the change is shown here: > ``` > diff --git a/configs/common/FSConfig.py b/configs/common/FSConfig.py > index 6fd39a5aa..837d95982 100644 > --- a/configs/common/FSConfig.py > +++ b/configs/common/FSConfig.py > @@ -226,6 +226,8 @@ def makeArmSystem(mem_mode, machine_type, num_cpus=1, > mdesc=None, > else: > self.pci_ide = IdeController(disks=disks) > pci_devices.append(self.pci_ide) > +self.realview.ethernet = IGbE_e1000() > +pci_devices.append(self.realview.ethernet) > > self.mem_ranges = [] > size_remain = long(Addr(mdesc.mem())) > ``` > > Specifically, the error that results during boot is regarding an address > range collision once the device loads: > ``` > fatal: system.iobus has two ports responding within range > [0x8000:0x8002]: > system.realview.ethernet.pio > system.iobridge.cpu_side_port > ``` > > That `0x8000:0x8002` range corresponds exactly to the 128kB BAR0 > of the `IGbE` device. > > The recent change [1] makes it clear that this is an issue because the > `IGbE_e1000` device uses a `PciMemBar`, which overlaps with the Realview's > DRAM address range, while the pre-existing `IdeController` device only uses > `PciIoBar` BARs, which don't. It's possible to avoid the range collision > by changing the Ethernet device's `PciMemBar` to a `PciIoBar` to keep it > out of the DRAM range, but it's not yet clear if that workaround will cause > other issues. > > What's the proper way to avoid this collision? > > [1] https://gem5-review.googlesource.com/c/public/gem5/+/35516 > > -jrb > > > > DISCLAIMER: > This e-mail may contain ITAR controlled technical data as defined by 22 > CFR 120.10 and may not be forwarded or disclosed to any Foreign Person, as > defined at 22 CFR 120.16, without the prior written consent of Honeybee > Robotics and the United States Department of State. > ___ > gem5-users mailing list -- gem5-users@gem5.org > To unsubscribe send an email to gem5-users-le...@gem5.org > %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s > ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
