Karthik, I've never used the -enable-kvm switch (nor have I heard anyone talk about it). Can you script your runs on our CS nodes, or any other private cluster, to help counteract the slow boots?
Tyler > Hi Tyler, > > Thanks for trying to simulate the problem. Actually I go to the > qemu-monitor to start the simulation only after the OS is properly > booted and after I log into it. But I would like to check whether you > have used the '-enable-kvm' switch in the qemu command as below, as > inclusion of this switch causes the problem. > > qemu/qemu-system-x86_64 -m [memory-size] -enable-kvm -hda > [path-to-qemu-disk-image] > > I used memory size of 1024. I tried a fresh build of MARSS and with > the above command I brought up the VM. Then when I tried to simulate > "shared_l2" or "private_L2" it gave the same reported error and dumped > core. Without the '-enable-kvm' everything works perfect. > > But I am trying to use '-enable-kvm' switch so that the QEMU emulation > will be faster at higher core counts. This will make life easier when > we use check points. Without '-enable-kvm' switch, QEMU emulation > itself is very slow at higher core counts as it emulates each CPU in > sequential order. Your comments are more welcome. > > Thanks a lot for pointing out the 'rootdelay' tip. Many times I > wondered why the VM did not boot up when core count is more(8 or > higher I think). Using the rootdelay parameter solved the issue. > > Regards, > karthik > > On Wed, Jul 23, 2014 at 2:58 PM, <[email protected]> wrote: >> Karthik, >> >> Wait for the VM to boot up. MARSS is prone to crash if you try to >> simulate >> before init finishes booting services. >> >> I just checked out a clean copy of MARSS. After passing rootdelay=200 >> (as >> documented: http://marss86.org/~marss86/index.php/QEMU_Tips), I passed >> "simconfig -machine private_L2" as the VM was booting. When the VM got >> to >> the shell, I did >> >> ./start_sim; ls; ./stop_sim >> >> And it worked fine... >> 1037eeaPTLCALL type PTLCALL_ENQUEUE >> MARSSx86::Command received : -stop >> >> Stopped after 5854991 cycles, 7428498 instructions and 63 seconds of sim >> time (cycle/sec: 92936 Hz, insns/sec: 117912, insns/cyc: >> 1.2687462713435427) >> >> Tyler >> >>> Hi Tyler, >>> >>> Actually this error occurs very soon after I start the simulation from >>> the QEMU monitor. The error occurs even before I get to the guest OS >>> window. For both the machines (shared_l2 &private_L2) I get this >>> error. More specifically I get the error message when I give the >>> command "simconfig -run -machine MACHINE_NAME" in the QEMU monitor and >>> when I press Ctrl+Alt+Shift-1 to switch to the guest OS. Hence I could >>> not start any benchmarks too. >>> >>> Also please find the error message I while simulating both the >>> "shared_l2" & "private_L2" machines provided in the 'default.conf' >>> configuration file. I am using the disk images provided in the >>> marss86.org website. Both the machines provide the same error message: >>> >>> shared_l2: >>> ERROR: guest physical address 0xfffffff0 is out of bounds >>> ERROR: guest physical address 0xfffffff0 is out of bounds >>> ERROR: guest physical address 0xfffffff0 is out of bounds >>> ERROR: guest physical address 0xfffffff0 is out of bounds >>> ERROR: guest physical address 0xfffffff0 is out of bounds >>> ERROR: guest physical address 0xfffffff0 is out of bounds >>> ERROR: guest physical address 0xfffffff0 is out of bounds >>> qemu-system-x86_64: ptlsim/build/core/ooo-core/ooo-pipe.cpp:2054: int >>> ooo::ReorderBufferEntry::commit(): Assertion `ctx.get_cs_eip() == >>> uop.rip' failed. >>> >>> private_L2: >>> ERROR: guest physical address 0xfffffff0 is out of bounds >>> ERROR: guest physical address 0xfffffff0 is out of bounds >>> ERROR: guest physical address 0xfffffff0 is out of bounds >>> ERROR: guest physical address 0xfffffff0 is out of bounds >>> ERROR: guest physical address 0xfffffff0 is out of bounds >>> ERROR: guest physical address 0xfffffff0 is out of bounds >>> ERROR: guest physical address 0xfffffff0 is out of bounds >>> qemu-system-x86_64: ptlsim/build/core/ooo-core/ooo-pipe.cpp:2054: int >>> ooo::ReorderBufferEntry::commit(): Assertion `ctx.get_cs_eip() == >>> uop.rip' failed. >>> >>> Your comments are more welcome. >>> >>> Thanks, >>> karthik >>> >>> Note: In my previous mail there were differences in error messages for >>> these same machines as I was using my own disk image instead of the >>> disk image available for download from www.marss86.org website. >>> >>> On Wed, Jul 23, 2014 at 8:24 AM, <[email protected]> wrote: >>>> Karthik, >>>> >>>> Which benchmark(s) are you running? Kernel version? Are you using one >>>> of >>>> the stock images? This shouldn't happen in a stock configuration like >>>> this... >>>> >>>> Tyler >>>> >>>>> Hi, >>>>> >>>>> Thanks a lot for taking your time to help me out amidst your busy >>>>> schedule. Do you need any more data from my side? Kindly let me know. >>>>> >>>>> Regards, >>>>> karthik >>>>> >>>>> On Mon, Jul 14, 2014 at 5:38 PM, karthik vm <[email protected]> wrote: >>>>>> Hi Brendan &Tyler, >>>>>> >>>>>> I have compiled MARSS with C=32 and ran both shared_l2 and >>>>>> private_L2. >>>>>> Unfortunately both of them crashed. Please find below the assertions >>>>>> I >>>>>> got with the simulation of each of them: >>>>>> >>>>>> shared_l2: >>>>>> ERROR: guest physical address 0xfffffff0 is out of bounds >>>>>> qemu-system-x86_64: ptlsim/build/core/ooo-core/ooo-pipe.cpp:2054: >>>>>> int >>>>>> ooo::ReorderBufferEntry::commit(): Assertion `ctx.get_cs_eip() == >>>>>> uop.rip' failed. >>>>>> >>>>>> private_L2: >>>>>> qemu-system-x86_64: ptlsim/build/core/ooo-core/ooo-pipe.cpp:2054: >>>>>> int >>>>>> ooo::ReorderBufferEntry::commit(): Assertion `ctx.get_cs_eip() == >>>>>> uop.rip' failed. >>>>>> >>>>>> I got the above assertions for the simulated machines consistently. >>>>>> I >>>>>> tried to simulate each one thrice. >>>>>> Kindly let me know if you need any other data. >>>>>> >>>>>> Thanks, >>>>>> karthik >>>>>> >>>>>> On Sun, Jul 13, 2014 at 10:51 PM, Brendan Fitzgerald >>>>>> <[email protected]> wrote: >>>>>>> Compile with c=32 then run either shared_l2 or private_L2 since >>>>>>> they >>>>>>> are >>>>>>> configured with OoO cores. >>>>>>> >>>>>>> By sample he means it's the one we provide. The core doesn't >>>>>>> reflect >>>>>>> an >>>>>>> actual machine, but it's close. >>>>>>> >>>>>>> >>>>>>> On Sun, Jul 13, 2014 at 9:25 PM, karthik vm <[email protected]> >>>>>>> wrote: >>>>>>>> >>>>>>>> Hi Tyler, >>>>>>>> >>>>>>>> I am not quite clear what you mean by sample OoO configurations. >>>>>>>> Currently the 'shared_l2' or 'private_L2' machine configurations >>>>>>>> are >>>>>>>> configured as OoO cores. So you want me to compile MARRS for 32 >>>>>>>> cores >>>>>>>> and simulate one of these machines? Can you clarify? Thanks. >>>>>>>> >>>>>>>> Regards, >>>>>>>> karthik >>>>>>>> >>>>>>>> On Sat, Jul 12, 2014 at 2:12 PM, <[email protected]> >>>>>>>> wrote: >>>>>>>> > Karthik, >>>>>>>> > >>>>>>>> > Can you try one of the sample OoO configurations? The >>>>>>>> configuration >>>>>>>> > files >>>>>>>> > are sensitive and currently not fully checked for validity. >>>>>>>> > >>>>>>>> > Compile with: >>>>>>>> > scons c=32 >>>>>>>> > >>>>>>>> > Then use the 'shared_l2" or 'private_L2' machine configuration >>>>>>>> (be >>>>>>>> aware >>>>>>>> > of the capitalization differences...) >>>>>>>> > >>>>>>>> > That should give you a 32-core non-SMT configuration. >>>>>>>> > >>>>>>>> > Tyler >>>>>>>> > >>>>>>>> >> Hi Tyler, >>>>>>>> >> >>>>>>>> >> Thanks for your immediate reply. Great to know that MARSS is >>>>>>>> suited >>>>>>>> >> for analyzing lock contention. >>>>>>>> >> >>>>>>>> >> Basically I was trying to simulate the 'private_L2' machine >>>>>>>> from >>>>>>>> the >>>>>>>> >> 'default' configuration file with enable-kvm switch in qemu. >>>>>>>> The >>>>>>>> >> configuration of 'cores' in private_L2 is below: >>>>>>>> >> >>>>>>>> >> cores: >>>>>>>> >> - type: ooo >>>>>>>> >> name_prefix: ooo_ >>>>>>>> >> >>>>>>>> >> Since there are no option for # of threads. I guess the cores >>>>>>>> are >>>>>>>> >> single threaded or non SMT. Anyway to enforce this I added the >>>>>>>> threads >>>>>>>> >> option as below: >>>>>>>> >> >>>>>>>> >> cores: >>>>>>>> >> - type: ooo >>>>>>>> >> name_prefix: ooo_ >>>>>>>> >> option: >>>>>>>> >> threads: 1 >>>>>>>> >> >>>>>>>> >> When I ran the simulation I get the same error again: >>>>>>>> >> qemu-system-x86_64: >>>>>>>> ptlsim/build/core/ooo-core/ooo-pipe.cpp:2054: >>>>>>>> int >>>>>>>> >> ooo::ReorderBufferEntry::commit(): Assertion `ctx.get_cs_eip() >>>>>>>> == >>>>>>>> >> uop.rip' failed. >>>>>>>> >> >>>>>>>> >> Then I tried to change the core type from 'ooo' to 'atom' which >>>>>>>> is >>>>>>>> non >>>>>>>> >> SMT in the machine. Please find the configuration below: >>>>>>>> >> >>>>>>>> >> cores: >>>>>>>> >> - type: atom >>>>>>>> >> name_prefix: atom_ >>>>>>>> >> option: >>>>>>>> >> threads: 1 >>>>>>>> >> >>>>>>>> >> Now also I see that the simulation stops with the following >>>>>>>> error >>>>>>>> >> shortly after it started: >>>>>>>> >> qemu-system-x86_64: >>>>>>>> ptlsim/build/core/atom-core/atomcore.cpp:2128: >>>>>>>> >> void atom::AtomThread::dtlb_walk(): Assertion `dtlb_miss_addr' >>>>>>>> failed. >>>>>>>> >> >>>>>>>> >> Kindly let me know your comments. >>>>>>>> >> >>>>>>>> >> Thanks, >>>>>>>> >> karthik >>>>>>>> >> >>>>>>>> >> On Thu, Jul 10, 2014 at 11:50 AM, <[email protected]> >>>>>>>> wrote: >>>>>>>> >>> Karthik, >>>>>>>> >>> >>>>>>>> >>> This assertion: >>>>>>>> >>> ooo::ReorderBufferEntry::commit(): Assertion `ctx.get_cs_eip() >>>>>>>> == >>>>>>>> >>> uop.rip' >>>>>>>> >>> failed. >>>>>>>> >>> >>>>>>>> >>> is a long-living bug in the current MARSS code. I believe it >>>>>>>> may >>>>>>>> been >>>>>>>> >>> linked to SMT-based configurations, but I have not had the >>>>>>>> time >>>>>>>> to >>>>>>>> >>> look >>>>>>>> >>> into it. If your simulated machine uses SMT, can you try >>>>>>>> disabling >>>>>>>> SMT >>>>>>>> >>> and >>>>>>>> >>> please regenerate checkpoints and try again? >>>>>>>> >>> >>>>>>>> >>> As for your second question, MARSS is definitely suited to >>>>>>>> giving >>>>>>>> your >>>>>>>> >>> more statistics than other options (i.e., instrumenting the >>>>>>>> kernel, >>>>>>>> >>> using >>>>>>>> >>> hardware/machine counters, etc.). If you need detailed, >>>>>>>> hardware-level >>>>>>>> >>> statistics relating to lock contention, you can easily >>>>>>>> instrument >>>>>>>> the >>>>>>>> >>> MARSS code base with some heuristics/counters that suit your >>>>>>>> research. >>>>>>>> >>> >>>>>>>> >>> Tyler >>>>>>>> >>> >>>>>>>> >>>> Hi, >>>>>>>> >>>> >>>>>>>> >>>> I am trying to use MARlSS for analyzing lock contention in >>>>>>>> >>>> multi-threaded apps and in linux OS code. I have successfully >>>>>>>> built >>>>>>>> >>>> the simulator and able to simulate the cores. Some of the >>>>>>>> questions I >>>>>>>> >>>> have are: >>>>>>>> >>>> >>>>>>>> >>>> 1) When I try to run MARSS configured with large number of >>>>>>>> cores(e.g >>>>>>>> >>>> 32), the simulation is very slow. From the discussion here >>>>>>>> >>>> >>>>>>>> >>>> "http://article.gmane.org/gmane.comp.emulators.marss86/112/match=enable+kvm"; >>>>>>>> >>>> I get that atleast for creating checkpoints we can use KVM >>>>>>>> which >>>>>>>> will >>>>>>>> >>>> be much faster. But I do not understand where should we >>>>>>>> enable >>>>>>>> the >>>>>>>> >>>> KVM? I tried the following: >>>>>>>> >>>> >>>>>>>> >>>> $ qemu/qemu-system-x86_64 -enable-kvm -m [memory_size] -hda >>>>>>>> >>>> [path-to-qemu-disk-image] >>>>>>>> >>>> >>>>>>>> >>>> Then in qemu monitor mode I configured the simulator as >>>>>>>> follows: >>>>>>>> >>>> simconfig -run -machine private_L2 >>>>>>>> >>>> >>>>>>>> >>>> But shortly after the simulator starts I get the following >>>>>>>> error: >>>>>>>> >>>> qemu-system-x86_64: >>>>>>>> ptlsim/build/core/ooo-core/ooo-pipe.cpp:2054: >>>>>>>> int >>>>>>>> >>>> ooo::ReorderBufferEntry::commit(): Assertion >>>>>>>> `ctx.get_cs_eip() >>>>>>>> == >>>>>>>> >>>> uop.rip' failed. >>>>>>>> >>>> >>>>>>>> >>>> Can someone point me what I miss here and what Avadh was >>>>>>>> talking >>>>>>>> >>>> about >>>>>>>> >>>> enabling KVM? >>>>>>>> >>>> >>>>>>>> >>>> 2) From the mailing list discussion >>>>>>>> >>>> >>>>>>>> >>>> (http://www.mail-archive.com/marss86-devel%40cs.binghamton.edu/msg01882.html) >>>>>>>> >>>> I see that MARSS runs the cores concurrently. Hence I believe >>>>>>>> MARSS >>>>>>>> >>>> can simulate the lock contentions in both application & OS >>>>>>>> code >>>>>>>> >>>> needed >>>>>>>> >>>> for my research. Kindly let me know your comments for >>>>>>>> confirmation. >>>>>>>> >>>> >>>>>>>> >>>> Thanks for your time, >>>>>>>> >>>> karthik >>>>>>>> >>>> >>>>>>>> >>>> _______________________________________________ >>>>>>>> >>>> http://www.marss86.org >>>>>>>> >>>> Marss86-Devel mailing list >>>>>>>> >>>> [email protected] >>>>>>>> >>>> https://www.cs.binghamton.edu/mailman/listinfo/marss86-devel >>>>>>>> >>>> >>>>>>>> >>> >>>>>>>> >>> >>>>>>>> >> >>>>>>>> > >>>>>>>> > >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> http://www.marss86.org >>>>>>>> Marss86-Devel mailing list >>>>>>>> [email protected] >>>>>>>> https://www.cs.binghamton.edu/mailman/listinfo/marss86-devel >>>>>>> >>>>>>> >>>>> >>>> >>>> >>> >> > _______________________________________________ http://www.marss86.org Marss86-Devel mailing list [email protected] https://www.cs.binghamton.edu/mailman/listinfo/marss86-devel
