Matthew Poremba has submitted this change. (
https://gem5-review.googlesource.com/c/public/gem5/+/57710 )
(
16 is the latest approved patch-set.
No files were changed between the latest approved patch-set and the
submitted one.
)Change subject: gpu-compute: Command processor read path from device
......................................................................
gpu-compute: Command processor read path from device
In full system mode, the AMDKernelCode object can reside in either the
system memory or in the dGPU device memory. Currently only reading from
the host/system memory is supported. This adds the necessary code to
read from the dGPU device memory.
Change-Id: I887fc706b3f9834db14e40f36fd29dd3d4602925
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/57710
Reviewed-by: Matt Sinclair <mattdsincl...@gmail.com>
Maintainer: Matt Sinclair <mattdsincl...@gmail.com>
Tested-by: kokoro <noreply+kok...@google.com>
---
M src/gpu-compute/gpu_command_processor.cc
1 file changed, 76 insertions(+), 7 deletions(-)
Approvals:
Matt Sinclair: Looks good to me, approved; Looks good to me, approved
kokoro: Regressions pass
diff --git a/src/gpu-compute/gpu_command_processor.cc
b/src/gpu-compute/gpu_command_processor.cc
index 08bce77..d46ace6 100644
--- a/src/gpu-compute/gpu_command_processor.cc
+++ b/src/gpu-compute/gpu_command_processor.cc
@@ -39,6 +39,8 @@
#include "debug/GPUKernelInfo.hh"
#include "dev/amdgpu/amdgpu_device.hh"
#include "gpu-compute/dispatcher.hh"
+#include "mem/abstract_mem.hh"
+#include "mem/packet_access.hh"
#include "mem/se_translating_port_proxy.hh"
#include "mem/translating_port_proxy.hh"
#include "params/GPUCommandProcessor.hh"
@@ -77,12 +79,20 @@
TranslationGenPtr
GPUCommandProcessor::translate(Addr vaddr, Addr size)
{
- // Grab the process and try to translate the virtual address with it;
with
- // new extensions, it will likely be wrong to just arbitrarily grab
context
- // zero.
- auto process = sys->threads[0]->getProcessPtr();
+ if (!FullSystem) {
+ // Grab the process and try to translate the virtual address with
it;
+ // with new extensions, it will likely be wrong to just arbitrarily
+ // grab context zero.
+ auto process = sys->threads[0]->getProcessPtr();
- return process->pTable->translateRange(vaddr, size);
+ return process->pTable->translateRange(vaddr, size);
+ }
+
+ // In full system use the page tables setup by the kernel driver rather
+ // than the CPU page tables.
+ return TranslationGenPtr(
+ new AMDGPUVM::UserTranslationGen(&gpuDevice->getVM(), walker,
+ 1 /* vmid */, vaddr, size));
}
/**
@@ -120,6 +130,27 @@
PortProxy &virt_proxy = FullSystem ? fs_proxy : se_proxy;
/**
+ * In full system mode, the page table entry may point to a system page
+ * or a device page. System pages use the proxy as normal, but a device
+ * page needs to be read from device memory. Check what type it is
here.
+ */
+ bool is_system_page = true;
+ Addr phys_addr = disp_pkt->kernel_object;
+ if (FullSystem) {
+ /**
+ * Full system currently only supports running on single VMID (one
+ * virtual memory space), i.e., one application running on GPU at a
+ * time. Because of this, for now we know the VMID is always 1.
Later
+ * the VMID would have to be passed on to the command processor.
+ */
+ int vmid = 1;
+ unsigned tmp_bytes;
+ walker->startFunctional(gpuDevice->getVM().getPageTableBase(vmid),
+ phys_addr, tmp_bytes, BaseMMU::Mode::Read,
+ is_system_page);
+ }
+
+ /**
* The kernel_object is a pointer to the machine code, whose entry
* point is an 'amd_kernel_code_t' type, which is included in the
* kernel binary, and describes various aspects of the kernel. The
@@ -129,8 +160,28 @@
* instructions.
*/
AMDKernelCode akc;
- virt_proxy.readBlob(disp_pkt->kernel_object, (uint8_t*)&akc,
- sizeof(AMDKernelCode));
+ if (is_system_page) {
+ DPRINTF(GPUCommandProc, "kernel_object in system, using proxy\n");
+ virt_proxy.readBlob(disp_pkt->kernel_object, (uint8_t*)&akc,
+ sizeof(AMDKernelCode));
+ } else {
+ assert(FullSystem);
+ DPRINTF(GPUCommandProc, "kernel_object in device, using device
mem\n");
+ // Read from GPU memory manager
+ uint8_t raw_akc[sizeof(AMDKernelCode)];
+ for (int i = 0; i < sizeof(AMDKernelCode) / sizeof(uint8_t); ++i) {
+ Addr mmhubAddr = phys_addr + i*sizeof(uint8_t);
+ Request::Flags flags = Request::PHYSICAL;
+ RequestPtr request = std::make_shared<Request>(
+ mmhubAddr, sizeof(uint8_t), flags,
walker->getDevRequestor());
+ Packet *readPkt = new Packet(request, MemCmd::ReadReq);
+ readPkt->allocate();
+ system()->getDeviceMemory(readPkt)->access(readPkt);
+ raw_akc[i] = readPkt->getLE<uint8_t>();
+ delete readPkt;
+ }
+ memcpy(&akc, &raw_akc, sizeof(AMDKernelCode));
+ }
DPRINTF(GPUCommandProc, "GPU machine code is %lli bytes from start of
the "
"kernel object\n", akc.kernel_code_entry_byte_offset);
--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/57710
To unsubscribe, or for help writing mail filters, visit
https://gem5-review.googlesource.com/settings
Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: I887fc706b3f9834db14e40f36fd29dd3d4602925
Gerrit-Change-Number: 57710
Gerrit-PatchSet: 22
Gerrit-Owner: Matthew Poremba <matthew.pore...@amd.com>
Gerrit-Reviewer: Matt Sinclair <mattdsincl...@gmail.com>
Gerrit-Reviewer: Matthew Poremba <matthew.pore...@amd.com>
Gerrit-Reviewer: kokoro <noreply+kok...@google.com>
Gerrit-MessageType: merged
_______________________________________________
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
