Giacomo,
Replacing the third word on line 70 of the dtsi [1] with `0x0` resolved the collision! Aside, running without the `--dtb` argument caused numerous failures (including an inability to enumerate PCI devices) leading up to a panic. Thanks to you both for your input resolving this, I really appreciate it. [1] https://github.com/gem5/gem5/blob/0d703041fcd5d119012b62287695723a2955b408/system/arm/dt/platforms/vexpress_gem5_v1_base.dtsi#L70 -jrb ________________________________ From: Giacomo Travaglini <giacomo.travagl...@arm.com> Sent: Friday, January 22, 2021 9:10 PM To: Bohren, Jonathan; Gabe Black Cc: gem5 users mailing list Subject: RE: [gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs I think the problem lies in the DTB and more specifically in the range mapping https://github.com/gem5/gem5/blob/stable/system/arm/dt/platforms/vexpress_gem5_v1_base.dtsi#L70 Could you replace the third word with 0x0? Otherwise you could rely on DTB autogeneration (simply omit the --dtb argument from cmdline) Please let me know if it works Giacomo > -----Original Message----- > From: Bohren, Jonathan <jrboh...@honeybeerobotics.com> > Sent: 23 January 2021 01:59 > To: Gabe Black <gabe.bl...@gmail.com> > Cc: Giacomo Travaglini <giacomo.travagl...@arm.com>; gem5 users mailing list > <gem5-users@gem5.org> > Subject: Re: [gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs > > Yeah, the first time it gets offset in the PciDevice constructor, and the > second > time is in a call to PciDevice::writeConfig() by the PCI host. Does it look > like the > `GenericArmPciHost` used by VExpress_GEM5_V1 (but not VExpress_EMM) > might be the source of the double-offset? > > > > > -jrb > > > ________________________________ > > From: Gabe Black <gabe.bl...@gmail.com> > Sent: Friday, January 22, 2021 8:21 PM > To: Bohren, Jonathan > Cc: Giacomo Travaglini; gem5 users mailing list > Subject: Re: [gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs > > I haven't looked at this closely, but it's normal for the values in the BARs > to be > displaced based on the platform object's idea of where the PCI ranges are in > the address space, ie there is an offset for config space, etc. It sounds to > me > like this offset is being added in twice, one by the kernel, and then once > again > by the PCI controller in the platform. That would not be a bug in the PCI BAR > system, it would be a configuration bug where only one of those should be > happening, probably the controller but I imagine it would depend somewhat. > > Gabe > > On Fri, Jan 22, 2021 at 1:25 PM Bohren, Jonathan > <jrboh...@honeybeerobotics.com > <mailto:jrboh...@honeybeerobotics.com> > wrote: > > > Gabe, > > It looks like after being initialized properly to 0x4000'000 range, the > BAR0 start for the Ethernet device is getting changed from 0x4000'0000 to > 0x8000'0000 in a call to `PciMemBar::write()` [1] once the driver loads. > [1] > https://github.com/gem5/gem5/blob/develop/src/dev/pci/device.hh#L195- > L213 > <https://github.com/gem5/gem5/blob/develop/src/dev/pci/device.hh#L195- > L213> > > Let me know if this conversation should migrate to the gem5-dev list. > > The stack trace below shows from where the call is being made: > ``` > #0 PciMemBar::write (this=<optimized out>, host=..., val=<optimized > out>) at build/ARM/dev/pci/device.hh:218 > #1 0x0000555557659221 in PciDevice::writeConfig > (this=0x55555d8a6000, pkt=0x7fffffffb540) at build/ARM/dev/pci/device.cc:381 > #2 0x00005555575a500c in IGbE::writeConfig (this=0x55555d8a6000, > pkt=<optimized out>) at build/ARM/dev/net/i8254xGBe.cc:152 > #3 0x000055555765f0b8 in GenericPciHost::write > (this=0x55555a048100, pkt=0x7fffffffb540) at build/ARM/dev/pci/host.cc:175 > #4 0x00005555574870b6 in non-virtual thunk to > PioPort<PioDevice>::recvAtomic(Packet*) () > #5 0x0000555557282617 in RequestPort::sendAtomic > (pkt=0x7fffffffb540, this=0x55555d859030) at build/ARM/mem/port.hh:464 > #6 NoncoherentXBar::recvAtomicBackdoor (this=0x55555a047e40, > pkt=0x7fffffffb540, cpu_side_port_id=<optimized out>, backdoor=0x0) at > build/ARM/mem/noncoherent_xbar.cc:264 > #7 0x000055555725626a in RequestPort::sendAtomic (pkt=<optimized > out>, this=0x555559f67e50) at build/ARM/mem/port.hh:464 > #8 Bridge::BridgeResponsePort::recvAtomic (this=<optimized out>, > pkt=<optimized out>) at build/ARM/mem/bridge.cc:344 > #9 0x000055555725f897 in RequestPort::sendAtomic > (pkt=0x7fffffffb540, this=0x55555d85aa00) at build/ARM/mem/port.hh:464 > #10 CoherentXBar::recvAtomicBackdoor (this=0x55555a016d00, > pkt=<optimized out>, cpu_side_port_id=<optimized out>, > backdoor=<optimized out>) at build/ARM/mem/coherent_xbar.cc:813 > #11 0x0000555557679db4 in RequestPort::sendAtomic > (pkt=0x7fffffffb540, this=0x555559649a50) at build/ARM/mem/port.hh:464 > #12 AtomicSimpleCPU::sendPacket (pkt=@0x7fffffffb530: > 0x7fffffffb540, port=..., this=0x555559649600) at > build/ARM/cpu/simple/atomic.cc:276 > #13 AtomicSimpleCPU::writeMem (this=<optimized out>, > data=0x7fffffffb674 "", size=<optimized out>, addr=<optimized out>, flags=..., > res=<optimized out>, byte_enable=...) at build/ARM/cpu/simple/atomic.cc:505 > #14 0x000055555768bb02 in SimpleExecContext::writeMem > (this=<optimized out>, data=<optimized out>, size=<optimized out>, > addr=<optimized out>, flags=..., res=0x0, byte_enable=std::vector<bool> of > length 4, capacity 64 = {...}) > at build/ARM/cpu/simple/exec_context.hh:553 > #15 0x00005555569d137d in writeMemAtomic<ExecContext> > (byte_enable=std::vector<bool> of length 4, capacity 64 = {...}, res=0x0, > flags=..., size=4, addr=2693333008, mem=0x7fffffffb674 "", xc=0x555559d856c0) > at build/ARM/arch/generic/memhelpers.hh:190 > #16 writeMemAtomic<(ByteOrder)1, ExecContext, unsigned int> > (xc=xc@entry=0x555559d856c0, traceData=traceData@entry=0x0, > mem=@0x7fffffffb6ec: 1073741824, addr=addr@entry=2693333008, flags=..., > flags@entry=..., res=0x0) > at build/ARM/arch/generic/memhelpers.hh:203 > #17 0x0000555556477426 in writeMemAtomicLE<ExecContext, > unsigned int> (res=0x0, flags=..., addr=2693333008, mem=@0x7fffffffb6ec: > 1073741824, traceData=0x0, xc=0x555559d856c0) at > build/ARM/arch/generic/memhelpers.hh:220 > #18 ArmISAInst::STORE_IMM_PN_AY_WN_SN_UN_SZ4::execute > (this=<optimized out>, xc=0x555559d856c0, traceData=0x0) at > build/ARM/arch/arm/generated/exec-ns.cc.inc:21605 > #19 0x00005555576770dc in AtomicSimpleCPU::tick > (this=0x555559649600) at build/ARM/cpu/simple/atomic.cc:685 > #20 0x000055555715ffcd in std::function<void ()>::operator()() const > (this=0x5555596499a0) at /usr/include/c++/7/bits/std_function.h:706 > #21 EventFunctionWrapper::process (this=0x555559649968) at > build/ARM/sim/eventq.hh:1138 > #22 EventQueue::serviceOne (this=this@entry=0x55555900b200) at > build/ARM/sim/eventq.cc:223 > #23 0x0000555557181cd0 in doSimLoop (eventq=0x55555900b200) at > build/ARM/sim/simulate.cc:216 > #24 0x0000555557182eda in simulate (num_cycles=<optimized out>) > at build/ARM/sim/simulate.cc:129 > ``` > > -jrb > > > > > > > > From: Bohren, Jonathan via gem5-users <gem5-users@gem5.org > <mailto:gem5-users@gem5.org> > > Sent: Friday, January 22, 2021 10:07 AM > To: gem5 users mailing list; Giacomo Travaglini > Cc: Gabe Black; Bohren, Jonathan > Subject: [gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs > > > Giacomo, > > > That's good to know. > > > We're running `fs.py` with `system/arm/dt/armv7_gem5_v1_1cpu.dtb` > which we've copied into our working directory. > > > Full invocation: > > > ``` > > ./gem5/build/ARM/gem5.opt \ > > gem5/configs/example/fs.py \ > > --bootloader ./boot.arm \ > > --kernel=./vmlinux \ > > --machine-type=VExpress_GEM5_V1 \ > > --dtb-file=./armv7_gem5_v1_1cpu.dtb \ > > --disk-image=../image/ubuntu-base-18.04.5-base-armhf.img \ > > --root-device=/dev/sda > > ``` > > > -jrb > > From: Giacomo Travaglini <giacomo.travagl...@arm.com > <mailto:giacomo.travagl...@arm.com> > > Sent: Friday, January 22, 2021 8:16:48 AM > To: gem5 users mailing list <gem5-users@gem5.org <mailto:gem5- > us...@gem5.org> > > Cc: Gabe Black <gabe.bl...@gmail.com > <mailto:gabe.bl...@gmail.com> >; Bohren, Jonathan > <jrboh...@honeybeerobotics.com > <mailto:jrboh...@honeybeerobotics.com> > > Subject: RE: [gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs > > > Out of curiosity, which DTB are you using? > IMO the kernel shouldn't write 8000000 to the PCI Bar, knowing there > is DRAM there and this is notified via the DTB > > Kind Regards > > Giacomo > > > -----Original Message----- > > From: Bohren, Jonathan via gem5-users <gem5-users@gem5.org > <mailto:gem5-users@gem5.org> > > > Sent: 22 January 2021 12:51 > > To: gem5 users mailing list <gem5-users@gem5.org <mailto:gem5- > us...@gem5.org> > > > Cc: Gabe Black <gabe.bl...@gmail.com > <mailto:gabe.bl...@gmail.com> >; Bohren, Jonathan > > <jrboh...@honeybeerobotics.com > <mailto:jrboh...@honeybeerobotics.com> > > > Subject: [gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs > > > > Gabe, > > > > > > > > Yeah, this is a very recent version of `develop`. This is after rebasing a > recent > > proposed change `38796` [1] (which disables the HDLcd) onto > `develop` HEAD > > from 2020-01-20: > > > > > > > > ``` > > > > c6e3fd79f (develop-rebase-38796) dev-arm, system-arm: Remove > HDLcd from > > VExpress_GEM5_VX platforms > > > > b9ce8848e system-arm: Move display node into a shared DTS file > > > > 71932a360 (upstream/develop, develop) dev: Fixing EtherDevice > stats > > initialization order > > > > ``` > > > > > > > > The two changesets in CL `38796` [1] were based on `3059c6d` which > was the > > HEAD of `develop` on 2020-01-05, and it didn't appear that CL > `38796` reverted > > any changes made in the interim. > > > > > > > > I came across a similar discussion in the archives [2], which did seem > similar, > > but didn't address the problem described below. Is the bug you're > referring to > > the one addressed by CL `35516` [3]? > > > > > > > > Thanks for taking a look at this! > > > > > > > > [1] https://gem5-review.googlesource.com/c/public/gem5/+/38796 > <https://gem5-review.googlesource.com/c/public/gem5/+/38796> > > <https://gem5-review.googlesource.com/c/public/gem5/+/38796> > > > > [2] https://www.mail-archive.com/gem5- > us...@gem5.org/msg18647.html <https://www.mail-archive.com/gem5- > us...@gem5.org/msg18647.html> > > <https://www.mail-archive.com/gem5- > us...@gem5.org/msg18647.html> > > > > [3] https://gem5-review.googlesource.com/c/public/gem5/+/35516 > <https://gem5-review.googlesource.com/c/public/gem5/+/35516> > > <https://gem5-review.googlesource.com/c/public/gem5/+/35516> > > > > > > > > -jrb > > > > > > > > From: Gabe Black via gem5-users <gem5-users@gem5.org > <mailto:gem5-users@gem5.org> > > > Sent: Thursday, January 21, 2021 11:59 PM > > To: gem5 users mailing list <gem5-users@gem5.org <mailto:gem5- > us...@gem5.org> > > > Cc: Gabe Black <gabe.bl...@gmail.com > <mailto:gabe.bl...@gmail.com> > > > Subject: [gem5-users] Re: VExpress_GEM5_V1, Ethernet, and BARs > > > > > > > > Are you using up to date develop? There was a bug like this a while > ago, but it's > > been fixed on develop for a while as well. > > > > > > > > Gabe > > > > > > > > On Thu, Jan 21, 2021 at 6:35 PM Bohren, Jonathan via gem5-users > <gem5- > > us...@gem5.org <mailto:us...@gem5.org> <mailto:gem5- > us...@gem5.org <mailto:gem5-users@gem5.org> > > wrote: > > > > We've been using the `VExpress_GEM5_V1` platform but it was > failing > > on both `stable` and `develop` when adding an `IGbE_e1000` PCI > device to the > > `makeArmSystem()` function in `FSConfig.py`. We've worked around > it, but > > looking for the "right way" to add such a device. > > > > For reference, the change is shown here: > > ``` > > diff --git a/configs/common/FSConfig.py > > b/configs/common/FSConfig.py > > index 6fd39a5aa..837d95982 100644 > > --- a/configs/common/FSConfig.py > > +++ b/configs/common/FSConfig.py > > @@ -226,6 +226,8 @@ def makeArmSystem(mem_mode, > > machine_type, num_cpus=1, mdesc=None, > > else: > > self.pci_ide = IdeController(disks=disks) > > pci_devices.append(self.pci_ide) > > + self.realview.ethernet = IGbE_e1000() > > + pci_devices.append(self.realview.ethernet) > > > > self.mem_ranges = [] > > size_remain = long(Addr(mdesc.mem())) > > ``` > > > > Specifically, the error that results during boot is regarding an address > > range collision once the device loads: > > ``` > > fatal: system.iobus has two ports responding within range > > [0x80000000:0x80020000]: > > system.realview.ethernet.pio > > system.iobridge.cpu_side_port > > ``` > > > > That `0x80000000:0x80020000` range corresponds exactly to the > 128kB > > BAR0 of the `IGbE` device. > > > > The recent change [1] makes it clear that this is an issue because the > > `IGbE_e1000` device uses a `PciMemBar`, which overlaps with the > Realview's > > DRAM address range, while the pre-existing `IdeController` device > only uses > > `PciIoBar` BARs, which don't. It's possible to avoid the range collision > by > > changing the Ethernet device's `PciMemBar` to a `PciIoBar` to keep it > out of the > > DRAM range, but it's not yet clear if that workaround will cause other > issues. > > > > What's the proper way to avoid this collision? > > > > [1] https://gem5-review.googlesource.com/c/public/gem5/+/35516 > <https://gem5-review.googlesource.com/c/public/gem5/+/35516> > > <https://gem5-review.googlesource.com/c/public/gem5/+/35516> > > > > -jrb > > > > ________________________________ > > > > DISCLAIMER: > > This e-mail may contain ITAR controlled technical data as defined by > 22 > > CFR 120.10 and may not be forwarded or disclosed to any Foreign > Person, as > > defined at 22 CFR 120.16, without the prior written consent of > Honeybee > > Robotics and the United States Department of State. > > _______________________________________________ > > gem5-users mailing list -- gem5-users@gem5.org <mailto:gem5- > us...@gem5.org> <mailto:gem5- <mailto:gem5-> > > us...@gem5.org <mailto:us...@gem5.org> > > > To unsubscribe send an email to gem5-users-le...@gem5.org > <mailto:gem5-users-le...@gem5.org> > > <mailto:gem5-users-le...@gem5.org <mailto:gem5-users- > le...@gem5.org> > > > %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s > > > > > > ________________________________ > > > > > > DISCLAIMER: > > This e-mail may contain ITAR controlled technical data as defined by > 22 CFR > > 120.10 and may not be forwarded or disclosed to any Foreign Person, > as > > defined at 22 CFR 120.16, without the prior written consent of > Honeybee > > Robotics and the United States Department of State. > > IMPORTANT NOTICE: The contents of this email and any attachments > are confidential and may also be privileged. If you are not the intended > recipient, please notify the sender immediately and do not disclose the > contents to any other person, use it for any purpose, or store or copy the > information in any medium. Thank you. > > > DISCLAIMER: > This e-mail may contain ITAR controlled technical data as defined by 22 > CFR 120.10 and may not be forwarded or disclosed to any Foreign Person, as > defined at 22 CFR 120.16, without the prior written consent of Honeybee > Robotics and the United States Department of State. > > > ________________________________ > > DISCLAIMER: > This e-mail may contain ITAR controlled technical data as defined by 22 > CFR 120.10 and may not be forwarded or disclosed to any Foreign Person, as > defined at 22 CFR 120.16, without the prior written consent of Honeybee > Robotics and the United States Department of State. > > > > ________________________________ > > > DISCLAIMER: > This e-mail may contain ITAR controlled technical data as defined by 22 CFR > 120.10 and may not be forwarded or disclosed to any Foreign Person, as > defined at 22 CFR 120.16, without the prior written consent of Honeybee > Robotics and the United States Department of State. IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. ________________________________ DISCLAIMER: This e-mail may contain ITAR controlled technical data as defined by 22 CFR 120.10 and may not be forwarded or disclosed to any Foreign Person, as defined at 22 CFR 120.16, without the prior written consent of Honeybee Robotics and the United States Department of State.
_______________________________________________ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
