I am sorry for the late update. I tried running with MESI_Two_Level but the simulation ends with this error.
warn: instruction 'prefetch_nta' unimplemented gem5.opt: build/X86_MESI_Two_Level/cpu/o3/lsq_unit.hh:621: Fault LSQUnit<Impl>::read(LSQUnit<Impl>::LSQRequest*, int) [with Impl = O3CPUImpl; Fault = std::shared_ptr<FaultBase>; LSQUnit<Impl>::LSQRequest = L SQ<O3CPUImpl>::LSQRequest]: Assertion `!load_inst->isExecuted()' failed. Which I believe has something to do with a recent update since I don't remember seeing it before. And this error happens even for just 2 cores and 2 threads. On Fri, Sep 6, 2019 at 3:16 PM Pouya Fotouhi <pfoto...@ucdavis.edu> wrote: > Hi Shehab, > > As Jason pointed out, I won’t be surprised if you are having issues with > classic caches running workloads that rely on locking mechanisms. Your > pthread implementation is possibly using some synchronization variables > which requires cache coherence to maintain its correctness, and classic > caches (at least for now) doesn’t support that. > > Switch to ruby caches (I suggest MESI Two Level to begin with), and given > your kernel version you should be getting stable behavior from gem5. > > Best, > > On Fri, Sep 6, 2019 at 11:47 AM Jason Lowe-Power <ja...@lowepower.com> > wrote: > >> Hi Shehab, >> >> IIRC, there are some issues when using classic caches + x86 + multiple >> cores on full system mode. I suggest using Ruby (MESI_two_level or >> MOESI_hammer) for FS simulations. >> >> Jason >> >> On Fri, Sep 6, 2019 at 11:24 AM Shehab Elsayed <shehaby...@gmail.com> >> wrote: >> >>> My latest experiments are with the classical memory system, but I >>> remember trying Ruby and it was not different. I am using kernel 4.8.13 and >>> ubuntu-16.04.1-server-amd64 disk image. I am using Pthreads for my Hello >>> World program. >>> >>> >>> On Fri, Sep 6, 2019 at 1:13 PM Pouya Fotouhi <pfoto...@ucdavis.edu> >>> wrote: >>> >>>> Hi Shehab, >>>> >>>> Can you confirm a few details about the configuration you are using? >>>> Are you using classic caches or Ruby? What is the kernel version and disk >>>> image you are using? What is the implementation of your "multithreaded >>>> hello world" (are you using OMP)? >>>> >>>> Best, >>>> >>>> On Fri, Sep 6, 2019 at 8:58 AM Shehab Elsayed <shehaby...@gmail.com> >>>> wrote: >>>> >>>>> First of all, thanks for your replies, Ryan and Jason. >>>>> >>>>> I have already pulled the latest changes by Pouya and the problem >>>>> still persists. >>>>> >>>>> As for checkpointing, I was originally doing exactly what Jason >>>>> mentioned and ran into the same problem. I then switched to not >>>>> checkpointing just to avoid any problems that might be caused by >>>>> checkpointing (if any). My plan was to go back to checkpointing after >>>>> proving that it works without it. >>>>> >>>>> However, the problem doesn't seem to be related to KVM as linux boots >>>>> reliable every time. The problem happens after the benchmarks starts >>>>> execution and it seems to be happening only when running multiple cores >>>>> (>=4). My latest experiments with a single core and 8 threads for the >>>>> benchmark seem to be working fine. But once I increase the number of >>>>> simulated cores problems happen. >>>>> >>>>> Also, I have posted a link to the repo I am using to run my tests in >>>>> a previous message. I have also added 2 debug traces with the Exec flag >>>>> for >>>>> a working and non-working examples. >>>>> >>>>> >>>>> On Fri, Sep 6, 2019 at 11:28 AM Jason Lowe-Power <ja...@lowepower.com> >>>>> wrote: >>>>> >>>>>> Hi Shehab, >>>>>> >>>>>> One quick note: There is *no way* to have deterministic behavior when >>>>>> running with KVM. Since you are using the hardware, the underlying host >>>>>> OS >>>>>> will influence the execution path of the workload. >>>>>> >>>>>> To try to narrow down the bug you're seeing, you can try to take a >>>>>> checkpoint after booting with KVM. Then, the execution from the >>>>>> checkpoint >>>>>> should be deterministic since it is 100% in gem5. >>>>>> >>>>>> BTW, I doubt you can run the KVM CPU in a VM since this would require >>>>>> your hardware and the VM to support nested virtualization. There *is* >>>>>> support for this in the Linux kernel, but I don't think it's been widely >>>>>> deployed outside of specific cloud environments. >>>>>> >>>>>> One other note: Pouya has pushed some changes which implement some >>>>>> x86 instructions that were causing issues for him. You can try with the >>>>>> current gem5 mainline to see if that helps. >>>>>> >>>>>> Cheers, >>>>>> Jason >>>>>> >>>>>> On Fri, Sep 6, 2019 at 8:22 AM Shehab Elsayed <shehaby...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> That's interesting. Are you using Full System as well? I don't think >>>>>>> FS behavior is supposed to be so dependent on the host environment! >>>>>>> >>>>>>> On Fri, Sep 6, 2019 at 11:16 AM Gambord, Ryan < >>>>>>> gambo...@oregonstate.edu> wrote: >>>>>>> >>>>>>>> I have found that gem5 behavior is sensitive to the execution >>>>>>>> environment. I now run gem5 inside an ubuntu vm on qemu and have had >>>>>>>> much >>>>>>>> more consistent results. I haven't tried running kvm gem5 inside a kvm >>>>>>>> qemu >>>>>>>> vm, so not sure how that works, but might be worth trying. >>>>>>>> >>>>>>>> Ryan >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Sep 6, 2019, 08:07 Shehab Elsayed <shehaby...@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> I was wondering if anyone is running into the same problem or if >>>>>>>>> anyone has any suggestions on how to proceed with debugging this >>>>>>>>> problem. >>>>>>>>> >>>>>>>>> On Mon, Jul 29, 2019 at 4:57 PM Shehab Elsayed < >>>>>>>>> shehaby...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Sorry for the spam. I just forgot to mention that the system >>>>>>>>>> configuration I am using is mainly from >>>>>>>>>> https://github.com/darchr/gem5/tree/jason/kvm-testing/configs/myconfigs. >>>>>>>>>> <https://github.com/darchr/gem5/tree/jason/kvm-testing/configs/myconfigs> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Shehab Y. Elsayed, MSc. >>>>>>>>>> PhD Student >>>>>>>>>> The Edwards S. Rogers Sr. Dept. of Electrical and Computer >>>>>>>>>> Engineering >>>>>>>>>> University of Toronto >>>>>>>>>> E-mail: shehaby...@gmail.com >>>>>>>>>> <https://webmail.rice.edu/imp/message.php?mailbox=INBOX&index=11#> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, Jul 29, 2019 at 4:08 PM Shehab Elsayed < >>>>>>>>>> shehaby...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> I have set up a repo with gem5 that demonstrates the problem. >>>>>>>>>>> The repo includes the latest version of gem5 from gem5's github >>>>>>>>>>> repo with a >>>>>>>>>>> few patches applied to get KVM working together with the kernel >>>>>>>>>>> binary and >>>>>>>>>>> disk image I am using. You can get the repo at >>>>>>>>>>> https://github.com/ShehabElsayed/gem5_debug.git. >>>>>>>>>>> <https://github.com/ShehabElsayed/gem5_debug.git> >>>>>>>>>>> >>>>>>>>>>> These steps should reproduce the problem: >>>>>>>>>>> 1- scons build/X86/gem5.opt >>>>>>>>>>> 2- ./scripts/get_fs_stuff.sh >>>>>>>>>>> 3- ./scripts/run_fs.sh 8 >>>>>>>>>>> >>>>>>>>>>> I have also included sample m5term outputs for both a 2 thread >>>>>>>>>>> run (m5out_2t) and an 8 thread run (m5out_8t) >>>>>>>>>>> >>>>>>>>>>> Any help is really appreciated. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Tue, Jul 23, 2019 at 11:01 AM Shehab Elsayed < >>>>>>>>>>> shehaby...@gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> When I enable the Exec debug flag I can see that it seems to be >>>>>>>>>>>> stuck in a spin lock (queued_spin_lock_slowpath) >>>>>>>>>>>> >>>>>>>>>>>> On Fri, Jul 19, 2019 at 5:28 PM Shehab Elsayed < >>>>>>>>>>>> shehaby...@gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hello All, >>>>>>>>>>>>> >>>>>>>>>>>>> I have a gem5 X86 full system set up that starts with KVM >>>>>>>>>>>>> cores and then switches to O3 cores once the benchmark reaches >>>>>>>>>>>>> the region >>>>>>>>>>>>> of interest. Right now I am testing with a simple >>>>>>>>>>>>> multithreaded hello world benchmark. Sometimes the benchmark >>>>>>>>>>>>> completes successfully while others gem5 just seems to hang after >>>>>>>>>>>>> starting >>>>>>>>>>>>> the benchmark. I believe it is still executing some instructions >>>>>>>>>>>>> but >>>>>>>>>>>>> without making any progress. The chance of this behavior ( >>>>>>>>>>>>> indeterminism) happening increases as the number of simulated >>>>>>>>>>>>> cores or the number of threads created by the benchmark increases. >>>>>>>>>>>>> >>>>>>>>>>>>> Any ideas what might be the reason for this or how I can start >>>>>>>>>>>>> debugging this problem? >>>>>>>>>>>>> >>>>>>>>>>>>> Note: I have tried the patch in https://gem5-review. >>>>>>>>>>>>> googlesource.com/c/public/gem5/+/19568 but the problem >>>>>>>>>>>>> persists. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks! >>>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>> gem5-users mailing list >>>>>>>>> gem5-users@gem5.org >>>>>>>>> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> gem5-users mailing list >>>>>>>> gem5-users@gem5.org >>>>>>>> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users >>>>>>> >>>>>>> _______________________________________________ >>>>>>> gem5-users mailing list >>>>>>> gem5-users@gem5.org >>>>>>> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users >>>>>> >>>>>> _______________________________________________ >>>>>> gem5-users mailing list >>>>>> gem5-users@gem5.org >>>>>> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users >>>>> >>>>> _______________________________________________ >>>>> gem5-users mailing list >>>>> gem5-users@gem5.org >>>>> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users >>>> >>>> >>>> >>>> -- >>>> Pouya Fotouhi >>>> PhD Candidate >>>> Department of Electrical and Computer Engineering >>>> University of California, Davis >>>> _______________________________________________ >>>> gem5-users mailing list >>>> gem5-users@gem5.org >>>> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users >>> >>> _______________________________________________ >>> gem5-users mailing list >>> gem5-users@gem5.org >>> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users >> >> _______________________________________________ >> gem5-users mailing list >> gem5-users@gem5.org >> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users > > -- > Pouya Fotouhi > PhD Candidate > Department of Electrical and Computer Engineering > University of California, Davis > _______________________________________________ > gem5-users mailing list > gem5-users@gem5.org > http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
_______________________________________________ gem5-users mailing list gem5-users@gem5.org http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
