The packet problems seem to come from this bit of code in
mem/cache/cache_impl.hh
waitingOnRetry = !masterPort.sendTimingReq(pkt);
if (waitingOnRetry) {
DPRINTF(CachePort, "now waiting on a retry\n");
if (!mshr->isForwardNoResponse()) {
// we are awaiting a retry, but we
// delete the packet and will be creating a new packet
// when we get the opportunity
delete pkt;
}
// note that we have now masked any requestBus and
// schedSendEvent (we will wait for a retry before
// doing anything), and this is so even if we do not
// care about this packet and might override it before
// it gets retried
} else {
cache.markInService(mshr, pkt);
}
The first line ends up calling timingAccess, and in that function the
packet may be deleted. timingAccess is a long function and I didn't
bother to try to figure out under what conditions the packet is deleted.
Then on the last line of code above, cache.markInService is called with
the packet as a parameter. That ends up reading some attributes from the
packet which has already been deleted.
I'm not familiar enough with how the caches are supposed to work to
propose a fix, but it looks like the faulty assumption is that packets
aren't deleted by sendTimingReq.
Gabe
On 06/03/12 02:09, Gabe Black wrote:
> I found the source of the messages about the decoder, and it was fairly
> harmless although worth fixing.
>
> http://reviews.gem5.org/r/1248/
>
> The only other messages that have popped up for me after that (below)
> are related to the memory system and caches, so maybe something's not
> right with how packets are being handled. It looks like a use after
> free, and when packets all came from the same pool (they were
> FastAlloc-ed, right?) they may have ended up over something inert and
> relatively valid, where as now they can end up pointing at anything.
>
> Gabe
>
> ==29119== Invalid read of size 4
> ==29119== at 0x617387: MSHR::markInService(Packet*) (bitset:406)
> ==29119== by 0x61B643: MSHRQueue::markInService(MSHR*, Packet*)
> (mshr_queue.cc:202)
> ==29119== by 0x5E5CA0: Cache<LRU>::markInService(MSHR*, Packet*)
> (base.hh:242)
> ==29119== by 0x5FA221:
> Cache<LRU>::MemSidePacketQueue::sendDeferredPacket() (cache_impl.hh:1704)
> ==29119== by 0x45E491: PacketQueue::retry() (packet_queue.cc:64)
> ==29119== by 0x433A39: EventWrapper<BaseBus,
> &(BaseBus::releaseBus())>::process() (port.hh:137)
> ==29119== by 0x10234E3: EventQueue::serviceOne() (eventq.cc:204)
> ==29119== by 0x10644E1: simulate(long) (simulate.cc:73)
> ==29119== by 0x10BE378: _wrap_simulate (event_wrap.cc:4489)
> ==29119== by 0x552955E: PyEval_EvalFrameEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x552B788: PyEval_EvalCodeEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x5529D52: PyEval_EvalFrameEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x552B788: PyEval_EvalCodeEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x552B901: PyEval_EvalCode (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x5529983: PyEval_EvalFrameEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x552B788: PyEval_EvalCodeEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x5529D52: PyEval_EvalFrameEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x552B788: PyEval_EvalCodeEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x552B901: PyEval_EvalCode (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x5545F30: run_mod (in /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x554612B: PyRun_StringFlags (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x102CA0E: m5Main(int, char**) (init.cc:256)
> ==29119== by 0x40A380: main (main.cc:57)
> ==29119== Address 0x904edbc is 12 bytes inside a block of size 88 free'd
> ==29119== at 0x4C2568F: operator delete(void*) (vg_replace_malloc.c:387)
> ==29119== by 0x5FD869: Cache<LRU>::timingAccess(Packet*)
> (cache_impl.hh:493)
> ==29119== by 0x5FE312:
> Cache<LRU>::CpuSidePort::recvTimingReq(Packet*) (cache_impl.hh:1594)
> ==29119== by 0x43FF15: CoherentBus::recvTimingReq(Packet*, short)
> (coherent_bus.cc:154)
> ==29119== by 0x5F9EB3:
> Cache<LRU>::MemSidePacketQueue::sendDeferredPacket() (cache_impl.hh:1688)
> ==29119== by 0x45E491: PacketQueue::retry() (packet_queue.cc:64)
> ==29119== by 0x433A39: EventWrapper<BaseBus,
> &(BaseBus::releaseBus())>::process() (port.hh:137)
> ==29119== by 0x10234E3: EventQueue::serviceOne() (eventq.cc:204)
> ==29119== by 0x10644E1: simulate(long) (simulate.cc:73)
> ==29119== by 0x10BE378: _wrap_simulate (event_wrap.cc:4489)
> ==29119== by 0x552955E: PyEval_EvalFrameEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x552B788: PyEval_EvalCodeEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x5529D52: PyEval_EvalFrameEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x552B788: PyEval_EvalCodeEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x552B901: PyEval_EvalCode (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x5529983: PyEval_EvalFrameEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x552B788: PyEval_EvalCodeEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x5529D52: PyEval_EvalFrameEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x552B788: PyEval_EvalCodeEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x552B901: PyEval_EvalCode (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x5545F30: run_mod (in /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x554612B: PyRun_StringFlags (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x102CA0E: m5Main(int, char**) (init.cc:256)
> ==29119== by 0x40A380: main (main.cc:57)
> ==29119==
> ==29119== Invalid read of size 4
> ==29119== at 0x617358: MSHR::markInService(Packet*) (mshr.cc:463)
> ==29119== by 0x61B643: MSHRQueue::markInService(MSHR*, Packet*)
> (mshr_queue.cc:202)
> ==29119== by 0x5E5CA0: Cache<LRU>::markInService(MSHR*, Packet*)
> (base.hh:242)
> ==29119== by 0x5FA221:
> Cache<LRU>::MemSidePacketQueue::sendDeferredPacket() (cache_impl.hh:1704)
> ==29119== by 0x10234E3: EventQueue::serviceOne() (eventq.cc:204)
> ==29119== by 0x10644E1: simulate(long) (simulate.cc:73)
> ==29119== by 0x10BE378: _wrap_simulate (event_wrap.cc:4489)
> ==29119== by 0x552955E: PyEval_EvalFrameEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x552B788: PyEval_EvalCodeEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x5529D52: PyEval_EvalFrameEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x552B788: PyEval_EvalCodeEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x552B901: PyEval_EvalCode (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x5529983: PyEval_EvalFrameEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x552B788: PyEval_EvalCodeEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x5529D52: PyEval_EvalFrameEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x552B788: PyEval_EvalCodeEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x552B901: PyEval_EvalCode (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x5545F30: run_mod (in /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x554612B: PyRun_StringFlags (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x102CA0E: m5Main(int, char**) (init.cc:256)
> ==29119== by 0x40A380: main (main.cc:57)
> ==29119== Address 0x90605f8 is 8 bytes inside a block of size 88 free'd
> ==29119== at 0x4C2568F: operator delete(void*) (vg_replace_malloc.c:387)
> ==29119== by 0x5FD288: Cache<LRU>::timingAccess(Packet*)
> (cache_impl.hh:425)
> ==29119== by 0x5FE312:
> Cache<LRU>::CpuSidePort::recvTimingReq(Packet*) (cache_impl.hh:1594)
> ==29119== by 0x43FF15: CoherentBus::recvTimingReq(Packet*, short)
> (coherent_bus.cc:154)
> ==29119== by 0x5F9EB3:
> Cache<LRU>::MemSidePacketQueue::sendDeferredPacket() (cache_impl.hh:1688)
> ==29119== by 0x10234E3: EventQueue::serviceOne() (eventq.cc:204)
> ==29119== by 0x10644E1: simulate(long) (simulate.cc:73)
> ==29119== by 0x10BE378: _wrap_simulate (event_wrap.cc:4489)
> ==29119== by 0x552955E: PyEval_EvalFrameEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x552B788: PyEval_EvalCodeEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x5529D52: PyEval_EvalFrameEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x552B788: PyEval_EvalCodeEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x552B901: PyEval_EvalCode (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x5529983: PyEval_EvalFrameEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x552B788: PyEval_EvalCodeEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x5529D52: PyEval_EvalFrameEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x552B788: PyEval_EvalCodeEx (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x552B901: PyEval_EvalCode (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x5545F30: run_mod (in /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x554612B: PyRun_StringFlags (in
> /usr/lib64/libpython2.6.so.1.0)
> ==29119== by 0x102CA0E: m5Main(int, char**) (init.cc:256)
> ==29119== by 0x40A380: main (main.cc:57)
>
>
> On 06/02/12 09:22, Ali Saidi wrote:
>> I was running regressions after removing FastAlloc and getting some very odd
>> behavior. It seems as though FastAlloc was either masking some use of
>> uninitialized values and the x86 decoder seems to have some issues. I don't
>> know if this is related to Gabe's recent restructuring, but I'm not seeing
>> the same issue on other architectures.
>>
>> Gabe, could you take a look?
>>
>> Thanks,
>> Ali
>>
>>
>> ==24715== Use of uninitialised value of size 8
>> ==24715== at 0x600738: X86ISA::Decoder::decode(X86ISA::ExtMachInst,
>> unsigned long) (hashtable.h:695)
>> ==24715== by 0x462555: BaseSimpleCPU::preExecute() (decoder.hh:240)
>> ==24715== by 0x456C0C: TimingSimpleCPU::completeIfetch(Packet*)
>> (timing.cc:661)
>> ==24715== by 0x45A49B:
>> TimingSimpleCPU::IcachePort::recvTimingResp(Packet*) (timing.cc:726)
>> ==24715== by 0xF307DC: PacketQueue::trySendTiming() (packet_queue.cc:147)
>> ==24715== by 0xF308CA: PacketQueue::sendDeferredPacket()
>> (packet_queue.cc:183)
>> ==24715== by 0xC24CE3: EventQueue::serviceOne() (eventq.cc:204)
>> ==24715== by 0xC66301: simulate(long) (simulate.cc:73)
>> ==24715== by 0xD35C88: _wrap_simulate (event_wrap.cc:4491)
>> ==24715== by 0x55540B0: PyEval_EvalFrameEx (in
>> /usr/lib/libpython2.7.so.1.0)
>> ==24715== by 0x555627C: PyEval_EvalCodeEx (in
>> /usr/lib/libpython2.7.so.1.0)
>> ==24715== by 0x555428C: PyEval_EvalFrameEx (in
>> /usr/lib/libpython2.7.so.1.0)
>> ==24715==
>> ==24715== Use of uninitialised value of size 8
>> ==24715== at 0x601B4C: std::tr1::__detail::_Map_base<X86ISA::ExtMachInst,
>> std::pair<X86ISA::ExtMachInst const, RefCountingPtr<StaticInst> >,
>> std::_Select1st<std::pair<X86ISA::ExtMachInst const,
>> RefCountingPtr<StaticInst> > >, true,
>> std::tr1::_Hashtable<X86ISA::ExtMachInst, std::pair<X86ISA::ExtMachInst
>> const, RefCountingPtr<StaticInst> >,
>> std::allocator<std::pair<X86ISA::ExtMachInst const,
>> RefCountingPtr<StaticInst> > >,
>> std::_Select1st<std::pair<X86ISA::ExtMachInst const,
>> RefCountingPtr<StaticInst> > >, std::equal_to<X86ISA::ExtMachInst>,
>> std::tr1::hash<X86ISA::ExtMachInst>, std::tr1::__detail::_Mod_range_hashing,
>> std::tr1::__detail::_Default_ranged_hash,
>> std::tr1::__detail::_Prime_rehash_policy, false, false, true>
>> >::operator[](X86ISA::ExtMachInst const&) (hashtable_policy.h:525)
>> ==24715== by 0x600AA0: X86ISA::Decoder::decode(X86ISA::ExtMachInst,
>> unsigned long) (decoder.cc:428)
>> ==24715== by 0x462555: BaseSimpleCPU::preExecute() (decoder.hh:240)
>> ==24715== by 0x456C0C: TimingSimpleCPU::completeIfetch(Packet*)
>> (timing.cc:661)
>> ==24715== by 0x45A49B:
>> TimingSimpleCPU::IcachePort::recvTimingResp(Packet*) (timing.cc:726)
>> ==24715== by 0xF307DC: PacketQueue::trySendTiming() (packet_queue.cc:147)
>> ==24715== by 0xF308CA: PacketQueue::sendDeferredPacket()
>> (packet_queue.cc:183)
>> ==24715== by 0xC24CE3: EventQueue::serviceOne() (eventq.cc:204)
>> ==24715== by 0xC66301: simulate(long) (simulate.cc:73)
>> ==24715== by 0xD35C88: _wrap_simulate (event_wrap.cc:4491)
>> ==24715== by 0x55540B0: PyEval_EvalFrameEx (in
>> /usr/lib/libpython2.7.so.1.0)
>> ==24715== by 0x555627C: PyEval_EvalCodeEx (in
>> /usr/lib/libpython2.7.so.1.0)
>> ==24715==
>> ==24715== Use of uninitialised value of size 8
>> ==24715== at 0x60191E: std::tr1::_Hashtable<X86ISA::ExtMachInst,
>> std::pair<X86ISA::ExtMachInst const, RefCountingPtr<StaticInst> >,
>> std::allocator<std::pair<X86ISA::ExtMachInst const,
>> RefCountingPtr<StaticInst> > >,
>> std::_Select1st<std::pair<X86ISA::ExtMachInst const,
>> RefCountingPtr<StaticInst> > >, std::equal_to<X86ISA::ExtMachInst>,
>> std::tr1::hash<X86ISA::ExtMachInst>, std::tr1::__detail::_Mod_range_hashing,
>> std::tr1::__detail::_Default_ranged_hash,
>> std::tr1::__detail::_Prime_rehash_policy, false, false,
>> true>::_M_insert_bucket(std::pair<X86ISA::ExtMachInst const,
>> RefCountingPtr<StaticInst> > const&, unsigned long, unsigned long)
>> (hashtable.h:862)
>> ==24715== by 0x601C26: std::tr1::__detail::_Map_base<X86ISA::ExtMachInst,
>> std::pair<X86ISA::ExtMachInst const, RefCountingPtr<StaticInst> >,
>> std::_Select1st<std::pair<X86ISA::ExtMachInst const,
>> RefCountingPtr<StaticInst> > >, true,
>> std::tr1::_Hashtable<X86ISA::ExtMachInst, std::pair<X86ISA::ExtMachInst
>> const, RefCountingPtr<StaticInst> >,
>> std::allocator<std::pair<X86ISA::ExtMachInst const,
>> RefCountingPtr<StaticInst> > >,
>> std::_Select1st<std::pair<X86ISA::ExtMachInst const,
>> RefCountingPtr<StaticInst> > >, std::equal_to<X86ISA::ExtMachInst>,
>> std::tr1::hash<X86ISA::ExtMachInst>, std::tr1::__detail::_Mod_range_hashing,
>> std::tr1::__detail::_Default_ranged_hash,
>> std::tr1::__detail::_Prime_rehash_policy, false, false, true>
>> >::operator[](X86ISA::ExtMachInst const&) (hashtable_policy.h:528)
>> ==24715== by 0x600AA0: X86ISA::Decoder::decode(X86ISA::ExtMachInst,
>> unsigned long) (decoder.cc:428)
>> ==24715== by 0x462555: BaseSimpleCPU::preExecute() (decoder.hh:240)
>> ==24715== by 0x456C0C: TimingSimpleCPU::completeIfetch(Packet*)
>> (timing.cc:661)
>> ==24715== by 0x45A49B:
>> TimingSimpleCPU::IcachePort::recvTimingResp(Packet*) (timing.cc:726)
>> ==24715== by 0xF307DC: PacketQueue::trySendTiming() (packet_queue.cc:147)
>> ==24715== by 0xF308CA: PacketQueue::sendDeferredPacket()
>> (packet_queue.cc:183)
>> ==24715== by 0xC24CE3: EventQueue::serviceOne() (eventq.cc:204)
>> ==24715== by 0xC66301: simulate(long) (simulate.cc:73)
>> ==24715== by 0xD35C88: _wrap_simulate (event_wrap.cc:4491)
>> ==24715== by 0x55540B0: PyEval_EvalFrameEx (in
>> /usr/lib/libpython2.7.so.1.0)
>> ==24715==
>> ==24715== Use of uninitialised value of size 8
>> ==24715== at 0x60192A: std::tr1::_Hashtable<X86ISA::ExtMachInst,
>> std::pair<X86ISA::ExtMachInst const, RefCountingPtr<StaticInst> >,
>> std::allocator<std::pair<X86ISA::ExtMachInst const,
>> RefCountingPtr<StaticInst> > >,
>> std::_Select1st<std::pair<X86ISA::ExtMachInst const,
>> RefCountingPtr<StaticInst> > >, std::equal_to<X86ISA::ExtMachInst>,
>> std::tr1::hash<X86ISA::ExtMachInst>, std::tr1::__detail::_Mod_range_hashing,
>> std::tr1::__detail::_Default_ranged_hash,
>> std::tr1::__detail::_Prime_rehash_policy, false, false,
>> true>::_M_insert_bucket(std::pair<X86ISA::ExtMachInst const,
>> RefCountingPtr<StaticInst> > const&, unsigned long, unsigned long)
>> (hashtable.h:864)
>> ==24715== by 0x601C26: std::tr1::__detail::_Map_base<X86ISA::ExtMachInst,
>> std::pair<X86ISA::ExtMachInst const, RefCountingPtr<StaticInst> >,
>> std::_Select1st<std::pair<X86ISA::ExtMachInst const,
>> RefCountingPtr<StaticInst> > >, true,
>> std::tr1::_Hashtable<X86ISA::ExtMachInst, std::pair<X86ISA::ExtMachInst
>> const, RefCountingPtr<StaticInst> >,
>> std::allocator<std::pair<X86ISA::ExtMachInst const,
>> RefCountingPtr<StaticInst> > >,
>> std::_Select1st<std::pair<X86ISA::ExtMachInst const,
>> RefCountingPtr<StaticInst> > >, std::equal_to<X86ISA::ExtMachInst>,
>> std::tr1::hash<X86ISA::ExtMachInst>, std::tr1::__detail::_Mod_range_hashing,
>> std::tr1::__detail::_Default_ranged_hash,
>> std::tr1::__detail::_Prime_rehash_policy, false, false, true>
>> >::operator[](X86ISA::ExtMachInst const&) (hashtable_policy.h:528)
>> ==24715== by 0x600AA0: X86ISA::Decoder::decode(X86ISA::ExtMachInst,
>> unsigned long) (decoder.cc:428)
>> ==24715== by 0x462555: BaseSimpleCPU::preExecute() (decoder.hh:240)
>> ==24715== by 0x456C0C: TimingSimpleCPU::completeIfetch(Packet*)
>> (timing.cc:661)
>> ==24715== by 0x45A49B:
>> TimingSimpleCPU::IcachePort::recvTimingResp(Packet*) (timing.cc:726)
>> ==24715== by 0xF307DC: PacketQueue::trySendTiming() (packet_queue.cc:147)
>> ==24715== by 0xF308CA: PacketQueue::sendDeferredPacket()
>> (packet_queue.cc:183)
>> ==24715== by 0xC24CE3: EventQueue::serviceOne() (eventq.cc:204)
>> ==24715== by 0xC66301: simulate(long) (simulate.cc:73)
>> ==24715== by 0xD35C88: _wrap_simulate (event_wrap.cc:4491)
>> ==24715== by 0x55540B0: PyEval_EvalFrameEx (in
>> /usr/lib/libpython2.7.so.1.0)
>> ==24715==
>>
>>
>>
>> _______________________________________________
>> gem5-dev mailing list
>> [email protected]
>> http://m5sim.org/mailman/listinfo/gem5-dev
> _______________________________________________
> gem5-dev mailing list
> [email protected]
> http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
[email protected]
http://m5sim.org/mailman/listinfo/gem5-dev
