Re: [gem5-dev] Ethernet device doesn't work with O3 cpu model in X86 ISA
Sorry for the late reply. Nilay, I have already answered this this question in
a follow up email:
>All I did is to add the following line to "finalizePhysical" function in
>"src/arch/x86/tlb.cc" :
> if (FullSystem) {
>+ if(paddr >= 3221225472 && paddr <= 4294901760) //if
>0xC000+ req->setFlags(Request::UNCACHEABLE);
> LocalApicBase localApicBase =
> tc->readMiscRegNoEffect(MISCREG_APIC_BASE);
>
>Maybe there are some wiser solutions for it, but it works ...
And I think the cleaner way is what Alex proposed.
Thanks,Mohammad
On Wednesday, October 1, 2014 1:56 PM, "Dutu, Alexandru via gem5-dev"
wrote:
From what I see the MTRRs are not used by gem5. On a real system I would
imagine the hardware would check MTRRs and PATs to determine if an access is
uncacheable or not.
It seems that memory accesses to the range of physical addresses
[0xC000-0x] are not marked as uncacheable by the TLB with either
atomic or O3 model. I am curios to why the same problem does not manifest with
the atomic model.
In addition, to get rid of all the hardcoded physical address range checks that
the memory subsystem has to do, it would be a bit cleaner to set the memory
request flags based on the flags in page table entries. In this way the flags
for cacheable/uncacheable, read-only/read-write etc. will be set once in the
page table, the TLB will be updated according to the page table and
TLB::finalizePhysical can actually set memory access request flags in the same
way the TlbEntry has them set.
I am preparing to post a patch that adds these flags in the TlbEntry and
changes the PageTable mapping interface to account for the extra flags.
Best regards,
Alex
From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Nilay Vaish via
gem5-dev [gem5-dev@gem5.org]
Sent: Wednesday, October 01, 2014 9:41 AM
To: Mohammad Alian via gem5-dev
Subject: Re: [gem5-dev] Ethernet device doesn't work with O3 cpu model in X86
ISA
How did you do this? My current understanding is that we need to support
MTRRs to mark the PCI address ranges as uncacheable.
--
Nilay
On Mon, 8 Sep 2014, Mohammad Alian via gem5-dev wrote:
> Hi, For some reasons gem5 doesn't recognize that PCI device address
> ranges are uncacheable. I set UNCACHEABLE flag for all accesses in the
> PCI physical address range and it fixed the problem.
>
> Thanks,
> Mohammad
>
>
>
> On Friday, September 5, 2014 10:48 AM, Steve Reinhardt via gem5-dev
> wrote:
>
>
>
> I don't know exactly what the problem is, but it's very suspicious that the
> device would receive a cache-block-sized access when it's in uncacheable
> memory space. My guess is that O3 is for some reason issuing a
> misspeculated cacheable access to the physical address where the device
> lives. Normally these would be suppressed when the TLB lookup indicates
> that the address is uncacheable, but if you're in kernel mode, you could
> end up with speculative accesses to the translation bypass range.
>
> To be honest, I don't know off the top of my head how this is dealt with in
> actual x86 systems... it would certainly be safest to suppress all
> speculative accesses to bypassed addresses, but I suspect that we can be
> smarter than that using the MTRRs. I could dig into it but don't have time
> at the moment, so if someone else knows these details, please speak up.
>
> Steve
>
>
> On Thu, Sep 4, 2014 at 2:30 PM, Mohammad Alian via gem5-dev <
> gem5-dev@gem5.org> wrote:
>
>> Hi all,
>>
>> Ethernet device work perfectly with atomic cpu in X86 ISA but I get an
>> error when I switch cpu to detailed O3 cpu. The error is:
>>
>> gem5.opt: build/X86/dev/i8254xGBe.cc:182: virtual Tick
>> IGbE::read(PacketPtr): Assertion `pkt->getSize() == 4' failed.
>> Program aborted at tick 131516086224500
>> Aborted
>>
>>
>> I noticed that the size of packet that Ethenet device receive with O3 cpu
>> is same as cache line size(64Bytes) however the Ethernet device just
>> support 32bit accesses.
>> Any Idea how to fix this problem?
>>
>> Thank you,
>> Mohammad Alian
>> ___
>> gem5-dev mailing list
>> gem5-dev@gem5.org
>> http://m5sim.org/mailman/listinfo/gem5-dev
>
>>
> ___
> gem5-dev mailing list
> gem5-dev@gem5.org
> http://m5sim.org/mailman/listinfo/gem5-dev
> ___
> gem5-dev mailing list
> gem5-dev@gem5.org
> http://m5sim.org/mailman/listinfo/gem5-dev
>
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev
Re: [gem5-dev] Ethernet device doesn't work with O3 cpu model in X86 ISA
>From what I see the MTRRs are not used by gem5. On a real system I would >imagine the hardware would check MTRRs and PATs to determine if an access is >uncacheable or not. It seems that memory accesses to the range of physical addresses [0xC000-0x] are not marked as uncacheable by the TLB with either atomic or O3 model. I am curios to why the same problem does not manifest with the atomic model. In addition, to get rid of all the hardcoded physical address range checks that the memory subsystem has to do, it would be a bit cleaner to set the memory request flags based on the flags in page table entries. In this way the flags for cacheable/uncacheable, read-only/read-write etc. will be set once in the page table, the TLB will be updated according to the page table and TLB::finalizePhysical can actually set memory access request flags in the same way the TlbEntry has them set. I am preparing to post a patch that adds these flags in the TlbEntry and changes the PageTable mapping interface to account for the extra flags. Best regards, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Nilay Vaish via gem5-dev [gem5-dev@gem5.org] Sent: Wednesday, October 01, 2014 9:41 AM To: Mohammad Alian via gem5-dev Subject: Re: [gem5-dev] Ethernet device doesn't work with O3 cpu model in X86 ISA How did you do this? My current understanding is that we need to support MTRRs to mark the PCI address ranges as uncacheable. -- Nilay On Mon, 8 Sep 2014, Mohammad Alian via gem5-dev wrote: > Hi, For some reasons gem5 doesn't recognize that PCI device address > ranges are uncacheable. I set UNCACHEABLE flag for all accesses in the > PCI physical address range and it fixed the problem. > > Thanks, > Mohammad > > > > On Friday, September 5, 2014 10:48 AM, Steve Reinhardt via gem5-dev > wrote: > > > > I don't know exactly what the problem is, but it's very suspicious that the > device would receive a cache-block-sized access when it's in uncacheable > memory space. My guess is that O3 is for some reason issuing a > misspeculated cacheable access to the physical address where the device > lives. Normally these would be suppressed when the TLB lookup indicates > that the address is uncacheable, but if you're in kernel mode, you could > end up with speculative accesses to the translation bypass range. > > To be honest, I don't know off the top of my head how this is dealt with in > actual x86 systems... it would certainly be safest to suppress all > speculative accesses to bypassed addresses, but I suspect that we can be > smarter than that using the MTRRs. I could dig into it but don't have time > at the moment, so if someone else knows these details, please speak up. > > Steve > > > On Thu, Sep 4, 2014 at 2:30 PM, Mohammad Alian via gem5-dev < > gem5-dev@gem5.org> wrote: > >> Hi all, >> >> Ethernet device work perfectly with atomic cpu in X86 ISA but I get an >> error when I switch cpu to detailed O3 cpu. The error is: >> >> gem5.opt: build/X86/dev/i8254xGBe.cc:182: virtual Tick >> IGbE::read(PacketPtr): Assertion `pkt->getSize() == 4' failed. >> Program aborted at tick 131516086224500 >> Aborted >> >> >> I noticed that the size of packet that Ethenet device receive with O3 cpu >> is same as cache line size(64Bytes) however the Ethernet device just >> support 32bit accesses. >> Any Idea how to fix this problem? >> >> Thank you, >> Mohammad Alian >> ___ >> gem5-dev mailing list >> gem5-dev@gem5.org >> http://m5sim.org/mailman/listinfo/gem5-dev > >> > ___ > gem5-dev mailing list > gem5-dev@gem5.org > http://m5sim.org/mailman/listinfo/gem5-dev > ___ > gem5-dev mailing list > gem5-dev@gem5.org > http://m5sim.org/mailman/listinfo/gem5-dev > ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev
Re: [gem5-dev] Ethernet device doesn't work with O3 cpu model in X86 ISA
How did you do this? My current understanding is that we need to support MTRRs to mark the PCI address ranges as uncacheable. -- Nilay On Mon, 8 Sep 2014, Mohammad Alian via gem5-dev wrote: Hi, For some reasons gem5 doesn't recognize that PCI device address ranges are uncacheable. I set UNCACHEABLE flag for all accesses in the PCI physical address range and it fixed the problem. Thanks, Mohammad On Friday, September 5, 2014 10:48 AM, Steve Reinhardt via gem5-dev wrote: I don't know exactly what the problem is, but it's very suspicious that the device would receive a cache-block-sized access when it's in uncacheable memory space. My guess is that O3 is for some reason issuing a misspeculated cacheable access to the physical address where the device lives. Normally these would be suppressed when the TLB lookup indicates that the address is uncacheable, but if you're in kernel mode, you could end up with speculative accesses to the translation bypass range. To be honest, I don't know off the top of my head how this is dealt with in actual x86 systems... it would certainly be safest to suppress all speculative accesses to bypassed addresses, but I suspect that we can be smarter than that using the MTRRs. I could dig into it but don't have time at the moment, so if someone else knows these details, please speak up. Steve On Thu, Sep 4, 2014 at 2:30 PM, Mohammad Alian via gem5-dev < gem5-dev@gem5.org> wrote: Hi all, Ethernet device work perfectly with atomic cpu in X86 ISA but I get an error when I switch cpu to detailed O3 cpu. The error is: gem5.opt: build/X86/dev/i8254xGBe.cc:182: virtual Tick IGbE::read(PacketPtr): Assertion `pkt->getSize() == 4' failed. Program aborted at tick 131516086224500 Aborted I noticed that the size of packet that Ethenet device receive with O3 cpu is same as cache line size(64Bytes) however the Ethernet device just support 32bit accesses. Any Idea how to fix this problem? Thank you, Mohammad Alian ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev
Re: [gem5-dev] Ethernet device doesn't work with O3 cpu model in X86 ISA
All I did is to add the following line to "finalizePhysical" function in
"src/arch/x86/tlb.cc" :
if (FullSystem) {
+if(paddr >= 3221225472 && paddr <= 4294901760) //if
0xC000setFlags(Request::UNCACHEABLE);
LocalApicBase localApicBase =
tc->readMiscRegNoEffect(MISCREG_APIC_BASE);
Maybe there are some wiser solutions for it, but it works ...
--Mohammad
On Wednesday, September 10, 2014 12:03 PM, Steve Reinhardt via gem5-dev
wrote:
Thanks for the update. Can you provide some more details on your fix?
Thanks,
Steve
On Mon, Sep 8, 2014 at 2:55 PM, Mohammad Alian via gem5-dev <
gem5-dev@gem5.org> wrote:
> Hi,
> For some reasons gem5 doesn't recognize that PCI device address ranges are
> uncacheable. I set UNCACHEABLE flag for all accesses in the PCI physical
> address range and it fixed the problem.
>
> Thanks,
> Mohammad
>
>
>
> On Friday, September 5, 2014 10:48 AM, Steve Reinhardt via gem5-dev <
> gem5-dev@gem5.org> wrote:
>
>
>
> I don't know exactly what the problem is, but it's very suspicious that the
> device would receive a cache-block-sized access when it's in uncacheable
> memory space. My guess is that O3 is for some reason issuing a
> misspeculated cacheable access to the physical address where the device
> lives. Normally these would be suppressed when the TLB lookup indicates
> that the address is uncacheable, but if you're in kernel mode, you could
> end up with speculative accesses to the translation bypass range.
>
> To be honest, I don't know off the top of my head how this is dealt with in
> actual x86 systems... it would certainly be safest to suppress all
> speculative accesses to bypassed addresses, but I suspect that we can be
> smarter than that using the MTRRs. I could dig into it but don't have time
> at the moment, so if someone else knows these details, please speak up.
>
> Steve
>
>
> On Thu, Sep 4, 2014 at 2:30 PM, Mohammad Alian via gem5-dev <
> gem5-dev@gem5.org> wrote:
>
> > Hi all,
> >
> > Ethernet device work perfectly with atomic cpu in X86 ISA but I get an
> > error when I switch cpu to detailed O3 cpu. The error is:
> >
> > gem5.opt: build/X86/dev/i8254xGBe.cc:182: virtual Tick
> > IGbE::read(PacketPtr): Assertion `pkt->getSize() == 4' failed.
> > Program aborted at tick 131516086224500
> > Aborted
> >
> >
> > I noticed that the size of packet that Ethenet device receive with O3 cpu
> > is same as cache line size(64Bytes) however the Ethernet device just
> > support 32bit accesses.
> > Any Idea how to fix this problem?
> >
> > Thank you,
> > Mohammad Alian
> > ___
> > gem5-dev mailing list
> > gem5-dev@gem5.org
> > http://m5sim.org/mailman/listinfo/gem5-dev
>
> >
> ___
> gem5-dev mailing list
> gem5-dev@gem5.org
> http://m5sim.org/mailman/listinfo/gem5-dev
> ___
> gem5-dev mailing list
> gem5-dev@gem5.org
> http://m5sim.org/mailman/listinfo/gem5-dev
>
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev
Re: [gem5-dev] Ethernet device doesn't work with O3 cpu model in X86 ISA
Thanks for the update. Can you provide some more details on your fix? Thanks, Steve On Mon, Sep 8, 2014 at 2:55 PM, Mohammad Alian via gem5-dev < gem5-dev@gem5.org> wrote: > Hi, > For some reasons gem5 doesn't recognize that PCI device address ranges are > uncacheable. I set UNCACHEABLE flag for all accesses in the PCI physical > address range and it fixed the problem. > > Thanks, > Mohammad > > > > On Friday, September 5, 2014 10:48 AM, Steve Reinhardt via gem5-dev < > gem5-dev@gem5.org> wrote: > > > > I don't know exactly what the problem is, but it's very suspicious that the > device would receive a cache-block-sized access when it's in uncacheable > memory space. My guess is that O3 is for some reason issuing a > misspeculated cacheable access to the physical address where the device > lives. Normally these would be suppressed when the TLB lookup indicates > that the address is uncacheable, but if you're in kernel mode, you could > end up with speculative accesses to the translation bypass range. > > To be honest, I don't know off the top of my head how this is dealt with in > actual x86 systems... it would certainly be safest to suppress all > speculative accesses to bypassed addresses, but I suspect that we can be > smarter than that using the MTRRs. I could dig into it but don't have time > at the moment, so if someone else knows these details, please speak up. > > Steve > > > On Thu, Sep 4, 2014 at 2:30 PM, Mohammad Alian via gem5-dev < > gem5-dev@gem5.org> wrote: > > > Hi all, > > > > Ethernet device work perfectly with atomic cpu in X86 ISA but I get an > > error when I switch cpu to detailed O3 cpu. The error is: > > > > gem5.opt: build/X86/dev/i8254xGBe.cc:182: virtual Tick > > IGbE::read(PacketPtr): Assertion `pkt->getSize() == 4' failed. > > Program aborted at tick 131516086224500 > > Aborted > > > > > > I noticed that the size of packet that Ethenet device receive with O3 cpu > > is same as cache line size(64Bytes) however the Ethernet device just > > support 32bit accesses. > > Any Idea how to fix this problem? > > > > Thank you, > > Mohammad Alian > > ___ > > gem5-dev mailing list > > gem5-dev@gem5.org > > http://m5sim.org/mailman/listinfo/gem5-dev > > > > ___ > gem5-dev mailing list > gem5-dev@gem5.org > http://m5sim.org/mailman/listinfo/gem5-dev > ___ > gem5-dev mailing list > gem5-dev@gem5.org > http://m5sim.org/mailman/listinfo/gem5-dev > ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev
Re: [gem5-dev] Ethernet device doesn't work with O3 cpu model in X86 ISA
Hi, For some reasons gem5 doesn't recognize that PCI device address ranges are uncacheable. I set UNCACHEABLE flag for all accesses in the PCI physical address range and it fixed the problem. Thanks, Mohammad On Friday, September 5, 2014 10:48 AM, Steve Reinhardt via gem5-dev wrote: I don't know exactly what the problem is, but it's very suspicious that the device would receive a cache-block-sized access when it's in uncacheable memory space. My guess is that O3 is for some reason issuing a misspeculated cacheable access to the physical address where the device lives. Normally these would be suppressed when the TLB lookup indicates that the address is uncacheable, but if you're in kernel mode, you could end up with speculative accesses to the translation bypass range. To be honest, I don't know off the top of my head how this is dealt with in actual x86 systems... it would certainly be safest to suppress all speculative accesses to bypassed addresses, but I suspect that we can be smarter than that using the MTRRs. I could dig into it but don't have time at the moment, so if someone else knows these details, please speak up. Steve On Thu, Sep 4, 2014 at 2:30 PM, Mohammad Alian via gem5-dev < gem5-dev@gem5.org> wrote: > Hi all, > > Ethernet device work perfectly with atomic cpu in X86 ISA but I get an > error when I switch cpu to detailed O3 cpu. The error is: > > gem5.opt: build/X86/dev/i8254xGBe.cc:182: virtual Tick > IGbE::read(PacketPtr): Assertion `pkt->getSize() == 4' failed. > Program aborted at tick 131516086224500 > Aborted > > > I noticed that the size of packet that Ethenet device receive with O3 cpu > is same as cache line size(64Bytes) however the Ethernet device just > support 32bit accesses. > Any Idea how to fix this problem? > > Thank you, > Mohammad Alian > ___ > gem5-dev mailing list > gem5-dev@gem5.org > http://m5sim.org/mailman/listinfo/gem5-dev > ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev
Re: [gem5-dev] Ethernet device doesn't work with O3 cpu model in X86 ISA
I don't know exactly what the problem is, but it's very suspicious that the device would receive a cache-block-sized access when it's in uncacheable memory space. My guess is that O3 is for some reason issuing a misspeculated cacheable access to the physical address where the device lives. Normally these would be suppressed when the TLB lookup indicates that the address is uncacheable, but if you're in kernel mode, you could end up with speculative accesses to the translation bypass range. To be honest, I don't know off the top of my head how this is dealt with in actual x86 systems... it would certainly be safest to suppress all speculative accesses to bypassed addresses, but I suspect that we can be smarter than that using the MTRRs. I could dig into it but don't have time at the moment, so if someone else knows these details, please speak up. Steve On Thu, Sep 4, 2014 at 2:30 PM, Mohammad Alian via gem5-dev < gem5-dev@gem5.org> wrote: > Hi all, > > Ethernet device work perfectly with atomic cpu in X86 ISA but I get an > error when I switch cpu to detailed O3 cpu. The error is: > > gem5.opt: build/X86/dev/i8254xGBe.cc:182: virtual Tick > IGbE::read(PacketPtr): Assertion `pkt->getSize() == 4' failed. > Program aborted at tick 131516086224500 > Aborted > > > I noticed that the size of packet that Ethenet device receive with O3 cpu > is same as cache line size(64Bytes) however the Ethernet device just > support 32bit accesses. > Any Idea how to fix this problem? > > Thank you, > Mohammad Alian > ___ > gem5-dev mailing list > gem5-dev@gem5.org > http://m5sim.org/mailman/listinfo/gem5-dev > ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev
[gem5-dev] Ethernet device doesn't work with O3 cpu model in X86 ISA
Hi all, Ethernet device work perfectly with atomic cpu in X86 ISA but I get an error when I switch cpu to detailed O3 cpu. The error is: gem5.opt: build/X86/dev/i8254xGBe.cc:182: virtual Tick IGbE::read(PacketPtr): Assertion `pkt->getSize() == 4' failed. Program aborted at tick 131516086224500 Aborted I noticed that the size of packet that Ethenet device receive with O3 cpu is same as cache line size(64Bytes) however the Ethernet device just support 32bit accesses. Any Idea how to fix this problem? Thank you, Mohammad Alian ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev
