Hi Andreas,
> Thanks. I really do not understand the return queue argument. Why on earth > would you need such a large return queue? Surely the agent making the > requests (the GPU in this case) should have allocated space for the > response, no? > Good question. I believe that buffer is just to get the appropriate overall minimum latency for memory accesses (i.e. GPGPU-Sim does some level of functional controller modeling, and the buffer is just for timing). If there a way to add arbitrary latency to the DRAMCtrl, it would be nice to aim for no-load latency of roughly 100ns (from when the controller receives the access to when a response is returned to the Ruby network). Also, if I have a chance to test out Nilay's patch, I could tune on this setting. Concerning the configuration, what is the assumed clock speed (tCK), and > is it operated as a x16 or x32 part? Is the burst length implicit in your > configuration (or is 8 the default)? > The memory modeled in GPGPU-Sim is x32 based on the addressing scheme. That's where I'd start the mode. In GDDR5, the core frequency is 1/4 of command frequency (and 1/2 in GDDR3?). A common channel frequency is 4GHz, resulting in effective channel bandwidth of 32GT/s. So, a starting baseline of tCK = 1ns should be sufficient. Does that sound right? I believe burst length is typically 8 and can be much longer. In practice, this depend highly on address hashing/interleaving across many controllers in various different GPUs, so 8 should be a sufficient baseline. Joel On 10/13/14, 10:28 PM, "Joel Hestness via gem5-dev" <gem5-dev@gem5.org> > wrote: > > >Hi Andreas, > > Sure thing. We try to closely replicate the parameters used in GPGPU-Sim > >v3.2.2, which are specified in the Hynix datasheet here: > >http://www.hynix.com/datasheet/pdf/graphics/H5GQ1H24AFR(Rev1.0).pdf. > > > > Here are relevant excerpts from the GPGPU-Sim GTX480 config file > >(attached): > > > ># The DRAM return queue and the scheduler queue together should provide > >buffer > ># to sustain the memory level parallelism to tolerate DRAM latency > ># To allow 100% DRAM utility, there should at least be enough buffer to > >sustain > ># the minimum DRAM latency (100 core cycles). I.e. > ># Total buffer space required = 100 x 924MHz / 700MHz = 132 > >-gpgpu_frfcfs_dram_sched_queue_size 16 > >-gpgpu_dram_return_queue_size 116 > > > >-dram_data_command_freq_ratio 4 # GDDR5 is QDR > >-gpgpu_dram_timing_opt "nbk=16:CCD=2:RRD=6:RCD=12:RAS=28:RP=12:RC=40: > > CL=12:WL=4:CDLR=5:WR=12:nbkgrp=4:CCDL=3:RTPL=2" > > > > If anything needs clarification, I'm happy to help sort it out. Just let > >me know. > > > > Thanks! > > Joel > > > > > > > >On Mon, Oct 13, 2014 at 4:11 PM, Andreas Hansson via gem5-dev < > >gem5-dev@gem5.org> wrote: > > > >> Hi Joel, > >> > >> I am happy to spend the 5 minutes creating a GDDR5 configuration. Do you > >> have any specific data sheet you would like to capture? > >> > >> Andreas > >> > >> On 10/13/14, 10:09 PM, "Joel Hestness via gem5-dev" <gem5-dev@gem5.org> > >> wrote: > >> > >> >Hi guys, > >> > > >> > > >> >> Thanks for the clarification. I believe the RubyMemoryController is > >> >> completely Pareto dominated by the vanilla DRAMCtrl module, but if > >>there > >> >> is any specific feature/setting missing I would be keen to know. > >> >> > >> >> If possible I would like to make sure we use the same controller as a > >> >> default for all timing simulations (even if the other one would be > >> >> maintained as a fallback). > >> > > >> > > >> >I'd like to second the desire to have a simple replacement baseline > >>that > >> >performs at least as well as the RubyMemoryController in most/all > >>cases: > >> >gem5-gpu now has more than 100 users, and as far as I know, we are all > >> >using Ruby and thus the RubyMemoryController. The RubyMemoryController > >>is > >> >pretty simple to configure similarly to DDR3 or GDDR5 and to interpret > >> >results. It performs surprisingly close to some GPU hardware. If this > >> >controller goes away, I (and I'm sure other gem5-gpu users) would > >>prefer > >> >to > >> >have something that is known to perform as well and is also easy to > >> >configure. > >> > > >> >I think we (the gem5-gpu crew) are fine with the RubyMemoryController > >> >going > >> >away eventually. However, given that there isn't currently a GDDR-like > >> >DRAMCtrl configuration in gem5, I'd like to second Nilay and Brad that > >>we > >> >offer users sufficient time to prepare for RubyMemoryController > >>removal. > >> >We > >> >will need to adapt our heterogeneous Ruby coherence protocols, and > >>other > >> >users have their own protocols they'd need to adapt as well. > >> > > >> > > >> > Joel > >> > > >> > > >> > > >> > > >> > > >> >> On 10/13/14, 9:01 PM, "Nilay Vaish via gem5-dev" <gem5-dev@gem5.org> > >> >> wrote: > >> >> > >> >> >On Mon, 13 Oct 2014, Andreas Hansson via gem5-dev wrote: > >> >> > > >> >> >> Hi all, > >> >> >> > >> >> >> With Nilay?s recent improvements to Ruby I would like to > >>understand > >> >>if > >> >> >> there is any point in still having the RubyMemoryControl, or if we > >> >> >> should just clean things up a bit and remove it. I would think the > >> >>best > >> >> >> way forward is to clean up the integration of Ruby and classic and > >> >> >> ensure that there is no duplicated functionality beyond what is > >> >> >>strictly > >> >> >> necessary. > >> >> >> > >> >> >> Nilay, do you think this would make sense? Is there anyone else > >>with > >> >> >>any > >> >> >> opinions in this matter? > >> >> >> > >> >> > > >> >> > > >> >> >I was in favor of dropping RubyMemoryControl. But I had some > >> >>discussion > >> >> >with Brad Beckmann from AMD. Since AMD has some infrastructure in > >> >>place > >> >> >already, they would like to retain RubyMemoryControl for the time > >> >>being. > >> >> > > >> >> >I suggest that we retain the memory controller code in ruby for > >>another > >> >> >six months or so, and then we will drop it. In the mean time, we > >> >> >will update the interface so that ruby protocols can use classic > >>memory > >> >> >controller. The code for this is already on the reviewboard. Over > >> >>this > >> >> >six month period, I hope, most users would have switched to using > >> >>classic > >> >> >controller. > >> >> > > >> >> >Thanks > >> >> >Nilay > >> >> >_______________________________________________ > >> >> >gem5-dev mailing list > >> >> >gem5-dev@gem5.org > >> >> >http://m5sim.org/mailman/listinfo/gem5-dev > >> >> > >> >> > >> >> -- IMPORTANT NOTICE: The contents of this email and any attachments > >>are > >> >> confidential and may also be privileged. If you are not the intended > >> >> recipient, please notify the sender immediately and do not disclose > >>the > >> >> contents to any other person, use it for any purpose, or store or > >>copy > >> >>the > >> >> information in any medium. Thank you. > >> >> > >> >> ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, > >> >> Registered in England & Wales, Company No: 2557590 > >> >> ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 > >> >>9NJ, > >> >> Registered in England & Wales, Company No: 2548782 > >> >> _______________________________________________ > >> >> gem5-dev mailing list > >> >> gem5-dev@gem5.org > >> >> http://m5sim.org/mailman/listinfo/gem5-dev > >> >> > >> > > >> > > >> > > >> >-- > >> > Joel Hestness > >> > PhD Student, Computer Architecture > >> > Dept. of Computer Science, University of Wisconsin - Madison > >> > http://pages.cs.wisc.edu/~hestness/ > >> >_______________________________________________ > >> >gem5-dev mailing list > >> >gem5-dev@gem5.org > >> >http://m5sim.org/mailman/listinfo/gem5-dev > >> > > >> > >> > >> -- IMPORTANT NOTICE: The contents of this email and any attachments are > >> confidential and may also be privileged. If you are not the intended > >> recipient, please notify the sender immediately and do not disclose the > >> contents to any other person, use it for any purpose, or store or copy > >>the > >> information in any medium. Thank you. > >> > >> ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, > >> Registered in England & Wales, Company No: 2557590 > >> ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 > >>9NJ, > >> Registered in England & Wales, Company No: 2548782 > >> > >> _______________________________________________ > >> gem5-dev mailing list > >> gem5-dev@gem5.org > >> http://m5sim.org/mailman/listinfo/gem5-dev > >> > > > > > > > >-- > > Joel Hestness > > PhD Student, Computer Architecture > > Dept. of Computer Science, University of Wisconsin - Madison > > http://pages.cs.wisc.edu/~hestness/ > >_______________________________________________ > >gem5-dev mailing list > >gem5-dev@gem5.org > >http://m5sim.org/mailman/listinfo/gem5-dev > > > > > -- IMPORTANT NOTICE: The contents of this email and any attachments are > confidential and may also be privileged. If you are not the intended > recipient, please notify the sender immediately and do not disclose the > contents to any other person, use it for any purpose, or store or copy the > information in any medium. Thank you. > > ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, > Registered in England & Wales, Company No: 2557590 > ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, > Registered in England & Wales, Company No: 2548782 > > _______________________________________________ > gem5-dev mailing list > gem5-dev@gem5.org > http://m5sim.org/mailman/listinfo/gem5-dev > -- Joel Hestness PhD Student, Computer Architecture Dept. of Computer Science, University of Wisconsin - Madison http://pages.cs.wisc.edu/~hestness/ _______________________________________________ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev
