Thank you very much for the responses ! I had no idea of SST , but it definitely looks like a great project for me to start looking for ideas. Reg. the TCP timeouts, I had forgotten that gem5 skips over idle cycles, so in effect I either need to make the time slice for syncing much lower than the TCP timeout or I need to disable this 'cycle-skipping'.
While on the topic of SST, do you have any calibration studies wrt other potentially more-accurate simulators (or) real hardware ? I would be very interested in looking at any such studies. Anirudh On Thu, Mar 29, 2012 at 1:43 PM, nathan binkert <[email protected]> wrote: > Doh! I totally forgot about SST. Any patches that you can get back into > gem5? > > Nate > > On Thu, Mar 29, 2012 at 10:28 AM, Rodrigues, Arun F <[email protected]> > wrote: >> Anirudh, >> >> You might want to take a look at the SST project here at Sandia >> (http://code.google.com/p/sst-simulator/). We've incorporated GeM5 into a >> parallel discrete event framework and run it to a few hundred nodes. We >> use a latency based conservative optimization similar to what you describe >> (i.e. fixed lookahead), but the lookahead is bounded by the minimum >> latency between nodes, so it is deterministic. This is all layered over >> MPI, and the performance seems reasonable (~80% scaling efficiency). >> (Note, we went over MPI instead of threading for scalability reasons. We >> eventually want to look at several thousands of nodes, and we don't have >> easy access to a large enough multithreaded machine. Our early tests seem >> to indicate that there is not a huge performance difference for our use >> case.). Currently, we have a NIC based on the Portals API connected to a >> model of the router used in the Cray XT3 series or routers from the >> Georgia Tech IRIS simulator. >> >> The current version of SST hangs a network interface off the memory bus >> (see attached picture), but we are working on a more generic version which >> would allow you to run each CPU on a different MPI rank to improve socket >> simulation speeds. >> >> >> >> Thanks, >> >> arun >> >> On 3/29/12 10:47 AM, "Anirudh Sivaraman" <[email protected]> wrote: >> >>>I have a design for a parallel version of GEM5. I wanted to run it by >>>the dev list before jumping in. The idea is to simulate a networked >>>system of multiple machines. The networking simulation will be handled >>>by ns3, a standard networking simulator. Each GEM5 instance will >>>connect into ns3 using a tap device (I hope to use ethertap.cc for >>>this) and ns3 will act as a "router" forwarding packets between GEM5 >>>instances. Each machine will be simulated by it's own GEM5 instance in >>>a separate thread and will hook into ns-3 using a tap device (ns-3 has >>>some support for this). ns3 is pretty flexible and can simulate >>>wired/wireless networks, but that should hopefully not matter to GEM5. >>> >>>The natural question is handling synchronization between the simulated >>>times in the various GEM5 instances. My idea is to use barrier >>>synchronization between the various GEM5 instances at periodic time >>>intervals. Let's assume this time interval is 10 ms. Then each GEM5 >>>instance runs from 0 through 10 ms of simulated time, and then waits >>>until all other GEM5 instances have finished their 10 ms slice as >>>well. The process then repeats itself from simulated time 10 to 20 ms. >>>Consequently, instances don't get out of sync by more than 10 ms at >>>any point. This interval is tunable, a lower interval gives you more >>>accuracy but more run time as well. >>> >>>I realize that determinism is impossible in this framework, but that's >>>a hit I am willing to take for my work. I wanted to know if there were >>>any code examples on using ethertap.cc just like etherlink.cc >>>(twosys-tsunami-simple-atomic.py) >>> >>>Anirudh >>>_______________________________________________ >>>gem5-dev mailing list >>>[email protected] >>>http://m5sim.org/mailman/listinfo/gem5-dev >>> >> >> >> _______________________________________________ >> gem5-dev mailing list >> [email protected] >> http://m5sim.org/mailman/listinfo/gem5-dev >> > _______________________________________________ > gem5-dev mailing list > [email protected] > http://m5sim.org/mailman/listinfo/gem5-dev _______________________________________________ gem5-dev mailing list [email protected] http://m5sim.org/mailman/listinfo/gem5-dev
