[gem5-users] Re: Setting up cluster for gem5
Thank you so much for your reply, Daniel! It is really helpful. On Fri, Nov 6, 2020 at 10:21 AM Daniel Gerzhoy wrote: > Hey Shehab, > > I've been working with gem5 on my group's research cluster for a while > now. > 1) Gem5 isn't very memory hungry in my experience, sometimes long > simulations (I'm talking 3 weeks+) will start bloating to GB of RAM but > its usually not paging so it doesn't slow things down (depends on the > program you are running) > I exclusively use Syscall Emulation mode, so that may not apply in Full > System. > > *It is however single-threaded. So if your entire group is running many > experiments at the same time make sure you have a ton of cores.* > > 2) As for job management, I created my own system for > configuring/running/parsing etc.that I've built with python. > > gem5 as of recently has been shipped with dockerfiles. I use the gcn3 > dockerfile for instance. I'd recommend using them. > Again I use a custom solution here, but I'm pretty sure container job > management is a solved problem. I think one of them is "kubernets" (see > https://kubernetes.io/) > I don't have experience with anything like that, but I'm sure that would > be useful. > > Also, if you plan on editing gem5 and your sourcecode is going to be > located on the cluster, I'd recommend using code-server ( > https://github.com/cdr/code-server) > It broadcasts an instance of vscode to a web page that you can access from > anywhere. I used to use gvim and bash scripts and it was hell. Code server > was a life-saver. > > If you (or anyone else) already have a solution for editing code on the > cluster I would be interested in what it is. > > Good luck! > > Dan Gerzhoy > PhD Candidate, Computer Engineering > University of Maryland College Park > > On Fri, Nov 6, 2020 at 8:38 AM Shehab Elsayed via gem5-users < > gem5-users@gem5.org> wrote: > >> Hello All, >> >> My group is in the process of upgrading our cluster and since many of us >> are using gem5 I was wondering if anyone has experience or recommendation >> they would like to share about the process for a smooth gem5 operation. >> Mainly I am concerned about 2 issues: >> >> 1) Required hard disk and memory on the nodes for a smooth gem5 >> operation. >> 2) OS and job management systems or any software related recommendations. >> >> Thank you very much in advance. >> >> Best Regards, >> Shehab >> ___ >> gem5-users mailing list -- gem5-users@gem5.org >> To unsubscribe send an email to gem5-users-le...@gem5.org >> %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s > > ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
[gem5-users] Setting up cluster for gem5
Hello All, My group is in the process of upgrading our cluster and since many of us are using gem5 I was wondering if anyone has experience or recommendation they would like to share about the process for a smooth gem5 operation. Mainly I am concerned about 2 issues: 1) Required hard disk and memory on the nodes for a smooth gem5 operation. 2) OS and job management systems or any software related recommendations. Thank you very much in advance. Best Regards, Shehab ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
[gem5-users] Re: 2 level TLB in ARM Full System with Ruby
Hi Ciro, Thanks for your reply! I don't remember seeing this patch before. I will check it out. The reason I specified RUBY is that one solution I found posted used a cache as a second level TLB and modified the port connections accordingly. However, that was a cache from the classical system and therefore wouldn't work with RUBY. Thanks again! Best Regards, Shehab On Wed, Jul 8, 2020 at 8:30 AM Ciro Santilli wrote: > Shehab, sorry for the delay, I had to check a few things about this, > > First, are you aware that there is a not-yet-merged patch that implements > a two level TLB at: > https://github.com/giactra/gem5/commit/3022ecc8a06a9182b2cf1936941901a785c1b21d > ? > > It hasn't been merged because we noticed that it broke Linux boot I think. > But we would like to merge it in the following months. > > I'm not sure why Ruby vs classic would matter since the TLB sits behind > caches anyways? I believe that model will work for either classic or Ruby. > -------------- > *From:* Shehab Elsayed via gem5-users > *Sent:* Tuesday, June 23, 2020 12:20 AM > *To:* gem5 users mailing list > *Cc:* Shehab Elsayed > *Subject:* [gem5-users] 2 level TLB in ARM Full System with Ruby > > Hello All, > > I was wondering if there is a way to simulate a system with 2 levels of > TLBs in full system simulation with ruby for ARM? > > I have seen other examples that use the classical memory model and use a > cache as the second level TLB. Is there something similar that can be > done in Ruby memory system. Can I use a standalone RubyCache as the > second level TLB? > > Thank you very much in advance. > > Best Regards, > Shehab > IMPORTANT NOTICE: The contents of this email and any attachments are > confidential and may also be privileged. If you are not the intended > recipient, please notify the sender immediately and do not disclose the > contents to any other person, use it for any purpose, or store or copy the > information in any medium. Thank you. > ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
[gem5-users] Modelling a deeper pipeline (O3-ARM)
Hello All, I am trying to model a deeper O3 pipeline as suggested in https://gem5-users.gem5.narkive.com/LNMJQ1M5/model-deeper-pipeline-in-x86 but I keep running into some assertion failures related to the time buffers and skid buffers even though that patch mentioned in the previous link is already added in my gem5 version. Is there any relation between the different pipeline delay values, widths, forward and backward communication sizes and any others parameter of the O3 cores that has to be maintained to avoid running into problems? For reference, these are the assertion failures I a facing depending on the values I choose: /cpu/timebuf.hh:54: void TimeBuffer::valid(int) const [with T = DefaultRenameDefaultIEW]: Assertion `idx >= -past && idx <= future' failed. cpu/o3/decode_impl.hh:425: void DefaultDecode::skidInsert(ThreadID) [with Impl = O3CPUImpl; ThreadID = short int]: Assertion `skidBuffer[tid].size() <= skidBufferMax' failed. Best Regards, Shehab ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
[gem5-users] 2 level TLB in ARM Full System with Ruby
Hello All, I was wondering if there is a way to simulate a system with 2 levels of TLBs in full system simulation with ruby for ARM? I have seen other examples that use the classical memory model and use a cache as the second level TLB. Is there something similar that can be done in Ruby memory system. Can I use a standalone RubyCache as the second level TLB? Thank you very much in advance. Best Regards, Shehab ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
[gem5-users] Re: GEM5/Ruby and MESI_Three_Level protocol
Which files do you think are missing? There are some shared files between MESI_Three_Level and MESI_Two-Level such as the L2 controller. You can find a list of all files used by the MESI_Three_Level protocol in src/mem/ruby/protocol/MESI_Three_Level.slicc. I hope this helps. On Thu, May 28, 2020 at 11:37 AM Javed Osmany via gem5-users < gem5-users@gem5.org> wrote: > Hello > > > > 1. I am able to successfully generate the executable gem5 simulator > for [ARM ISA, MESI_Three_Level protocol]. The command I used being: > > a. scons -j4 build/ARM_MESI_3_level/gem5.opt --default=ARM > PROTOCOL=MESI_Three_Level SLICC_HTML=True > > > > 2. Also, I am able successfully generate the executable gem5 > simulator for [X86 ISA, MESI_Three_Level protocol]. The command I used > being: > > a. scons -j4 build/X86_MESI_3_level/gem5.opt --default=X86 > PROTOCOL=MESI_Three_Level SLICC_HTML=True > > > > However, if I look in src/mem/ruby/protocol, the code for MESI_Three_Level > is as follows: > > > > [j00533938@lhrplinux1 protocol]$ ll MESI_Three_Level* > > -rw-rw-r-- 1 j00533938 j00533938 40031 May 28 09:17 > MESI_Three_Level-L0cache.sm > > -rw-rw-r-- 1 j00533938 j00533938 36841 May 28 09:17 > MESI_Three_Level-L1cache.sm > > -rw-rw-r-- 1 j00533938 j00533938 4270 May 28 09:17 MESI_Three_Level-msg.sm > > -rw-rw-r-- 1 j00533938 j00533938 316 Mar 20 17:42 MESI_Three_Level.slicc > > > > > > Therefore, it looks to me that the code for MESI_Three_Level is not > complete. Thus it is not clear to me how the executable gem5 simulator for > MESI_Three_Level is being generated. > > > > > > Any thoughts on this please? > > > > Thanks in advance. > > JO > > > ___ > gem5-users mailing list -- gem5-users@gem5.org > To unsubscribe send an email to gem5-users-le...@gem5.org > %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
[gem5-users] Re: Question about Ruby cache latencies
I see, Thanks for the explanation! On Thu, May 14, 2020 at 3:36 PM Tiago Muck wrote: > Right now it's possible the redefine the mandatoryQueueLatency function to > return the cache latency, but this only works for L1 hit latency. It's > currently not possible to have a fully generic model since each protocol > can have different assumptions regarding how a cache lookup/update latency > would affect each transaction. > > Best, > Tiago > -- > *From:* Shehab Elsayed > *Sent:* Thursday, May 14, 2020 11:50 AM > *To:* gem5 users mailing list > *Cc:* Tiago Muck > *Subject:* Re: [gem5-users] Question about Ruby cache latencies > > Thank you very much for your reply and explanation, Tiago! > > Wouldn't it be more generic to add the latencies at the time of performing > the access in the cache itself instead of having it in the controllers > since any cache access should incur access latency? I am not sure how easy > that would be though given the way ruby works right now. I don't know the > exact details of ruby operation but I took a quick look and noticed that > getEntry(...) can be called multiple times for the same request which, I > guess, makes my suggestion more difficult to add. > > On Tue, May 12, 2020 at 12:11 PM Tiago Muck via gem5-users < > gem5-users@gem5.org> wrote: > > Hi Shehab, > > Your understanding is correct, there are some cases that are not being > handled. This https://gem5-review.googlesource.com/c/public/gem5/+/18414 > patched > MOESI_CMP_directory to some extent (there was no cache latency being > considered before) but was not a complete solution. Other then the case > you mentioned, MOESI_CMP_directory is also currently missing the > transaction annotations so it can generate stalls on cache/directory bank > access conflicts. > > Best, > Tiago > IMPORTANT NOTICE: The contents of this email and any attachments are > confidential and may also be privileged. If you are not the intended > recipient, please notify the sender immediately and do not disclose the > contents to any other person, use it for any purpose, or store or copy the > information in any medium. Thank you. > ___ > gem5-users mailing list -- gem5-users@gem5.org > To unsubscribe send an email to gem5-users-le...@gem5.org > %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s > > IMPORTANT NOTICE: The contents of this email and any attachments are > confidential and may also be privileged. If you are not the intended > recipient, please notify the sender immediately and do not disclose the > contents to any other person, use it for any purpose, or store or copy the > information in any medium. Thank you. > ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
[gem5-users] Re: Question about Ruby cache latencies
Thank you very much for your reply and explanation, Tiago! Wouldn't it be more generic to add the latencies at the time of performing the access in the cache itself instead of having it in the controllers since any cache access should incur access latency? I am not sure how easy that would be though given the way ruby works right now. I don't know the exact details of ruby operation but I took a quick look and noticed that getEntry(...) can be called multiple times for the same request which, I guess, makes my suggestion more difficult to add. On Tue, May 12, 2020 at 12:11 PM Tiago Muck via gem5-users < gem5-users@gem5.org> wrote: > Hi Shehab, > > Your understanding is correct, there are some cases that are not being > handled. This https://gem5-review.googlesource.com/c/public/gem5/+/18414 > patched > MOESI_CMP_directory to some extent (there was no cache latency being > considered before) but was not a complete solution. Other then the case > you mentioned, MOESI_CMP_directory is also currently missing the > transaction annotations so it can generate stalls on cache/directory bank > access conflicts. > > Best, > Tiago > IMPORTANT NOTICE: The contents of this email and any attachments are > confidential and may also be privileged. If you are not the intended > recipient, please notify the sender immediately and do not disclose the > contents to any other person, use it for any purpose, or store or copy the > information in any medium. Thank you. > ___ > gem5-users mailing list -- gem5-users@gem5.org > To unsubscribe send an email to gem5-users-le...@gem5.org > %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
[gem5-users] Question about Ruby cache latencies
Hello All, In MOESI_CMP_directory, the ruby cache latencies (tagAccessLatency and dataAccessLatency) are included in the SLICC cache controllers through cacheReponsLatency() function. However, the function is only included in messages that include a data response while all other messages use the controllers latency as defined in corresponding SLICC file. My question is the following: Doesn't the cache still need to at least access the tag array for other actions as well that might not include sending data. For example, when receiving a request to invalidate a certain block, the cache would need to access the tag array first to invalidate the block before sending the Ack. If that is the case, shouldn't the sendAck get its enqueue latency using cacheResponseLatecy() as well?! Right now it is hardcoded to response_latency regardless of tagAccessLatency. Is my understanding correct? Or am I missing something? ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
