Re: [Mesa-dev] [Libre-soc-dev] Loading Vulkan Driver
On 23/8/20 8:11, Luke Kenneth Casson Leighton wrote: On Sun, Aug 23, 2020 at 1:56 AM apinheiro wrote: On 22/8/20 23:59, Luke Kenneth Casson Leighton wrote: constructive feedback on this approach greatly appreciated: https://bugs.libre-soc.org/show_bug.cgi?id=251#c36 In any case, those steps look good enough, although I lack any context of the final target. (if i can fill in this bit) the context is: to augment the PowerISA Instruction set to add 3D opcodes *directly* to PowerISA, with the guidance, full knowledge and under the supervision of the OpenPOWER Foundation. this will include: adding sin, cos, atan2... *to PowerISA*. adding texture interpolation instructions... *to PowerISA*. adding YUV2RGB and FP32-to-RGB conversion instructions... *to PowerISA*. therefore, whilst the initial target is "general purpose scalar non-accelerated non-vectorised soft-3D", this is simply because the next step is to add 3D opcodes *directly* to the POWER9 core that we are developing. Then, in addition to the drivers you already looking as reference, it would make sense if you take a look to Vallium, that have just landed on Mesa master https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6082 i mention this because normally, GPUs are considered completely separate entities - completely separate processors. LibreSOC will be the first ever libre-licensed *hybrid* 3D-capable CPU, i.e. where the *main processor* ISA is augmented to no longer need a completely separate GPU. i appreciate that i have said the same thing in several different ways. this is because hybrid CPU/VPU/GPUs are so unusual and so "not normal industry practice" (because they're easier to sell if they're separate) that it takes a while to sink in. i know of only one commercial company that has developed a hybrid CPU/VPU/GPU (they called it a GPGPU - general purpose GPU) - by ICubeCorp. back around 2012-2015 they received government funding sufficient to complete the IC3128 and associated VLIW compiler. an SGI Engineer who went on to work for both AMD and NVidia created the ISA and designed the architecture. l. ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [Libre-soc-dev] Loading Vulkan Driver
Hello all, I just cleanup a bit and commited code at : https://gitlab.freedesktop.org/vivekvpandya/mesa/-/commit/d1f29adf926eb44453db1255834575a6f7169d5f However it is not working as per my expectation (or my expectations are wrong) I want to see that driver fail at https://gitlab.freedesktop.org/vivekvpandya/mesa/-/commit/d1f29adf926eb44453db1255834575a6f7169d5f#54a82819ff146aa01755bf951d797d55a0332a32_0_43 however in debugger control never reach there. I see that in https://gitlab.freedesktop.org/apinheiro/mesa/-/commit/07d01ebf6aae2f9ae71a8bea13a5d8acccb6280e CreateShaderModule() is defined I am defining libresoc_CreateGraphicsPipelines() and I want control to reach here. I need to debug more to know what part of code is responsible to call this function. Just to note I have not done much change in entrypoint generation / extension generation python script so my doubt is that there something I am missing. Also when I commented to https://gitlab.freedesktop.org/vivekvpandya/mesa it seems to be triggered CI/CD, for now I don't want that. Any way to disable CI/CD triggers? Thanks, Vivek On Sun, Aug 23, 2020 at 6:26 AM apinheiro wrote: > > On 22/8/20 23:59, Luke Kenneth Casson Leighton wrote: > > On Sat, Aug 22, 2020 at 10:34 PM apinheiro > wrote: > > > As Jason mentioned, to get a Vulkan driver started, you would need more > than just one method. If you want a reference: > https://gitlab.freedesktop.org/apinheiro/mesa/-/commit/07d01ebf6aae2f9ae71a8bea13a5d8acccb6280e > > This commit added the basic skeleton for v3dv (broadcom mesa vulkan > driver). > > fantastic this is extremely helpful and much appreciated, thank you. > > some background: vivek has kindly agreed to start the NLNet-funded > LibreSOC MESA Vulkan driver. given that the LibreSOC CPU is a hybrid > CPU/VPU/GPU with an augmented POWER9 ISA (rather than "a totally > separate custom GPU with a foreign ISA completely incompatible with > POWER9"), the first step is actually a general-purpose non-accelerated > *software* Vulkan Driver, very similar to SwiftShader in concept > except: > > * utilising NIR in order to take advantage of the 3D shader passes and > the information it can hold > * retaining vector, predication and texturisation intrinsics right up > to the very last second when handing over to LLVM-IR > * relying on "general" LLVM-IR to perform translation for *native* > (host) execution - not a "totally separate custom GPU..." > * initially entirely targetting *host* (native) scalar instructions > [or, if general-purpose LLVM-IR happens to use NEON, SSE etc. that's > great but not our concern] > > in other words it should be possible for the LibreSOC Vulkan driver to > run on native non-accelerated CPUs - x86, POWER9, ARM, MIPS and so on. > > the second step will be to add custom 3D instructions *to POWER9*, at > which point whilst we hope to retain the ability to still run on > unaccelerated hardware, it is a secondary priority, albeit a very > important one. > > as there may be questions "why not start from a different point, why > not use SwiftShader or gallium" and so on, that evaluation took place > here:https://bugs.libre-soc.org/show_bug.cgi?id=251 > > constructive feedback on this approach greatly > appreciated:https://bugs.libre-soc.org/show_bug.cgi?id=251#c36 > > > Quoting from there: > > > it should just return some error code as VkResult. > > Not sure why it should return an error code. Even if initially is a no-op > pipeline, if the method is able to create the object I think it is ok to > return success, and let errors to be for real errors (out of memory, etc). > But I guess that's ok if you prefer that until it starts to do something > real. > > > Test this setup with by forcing application to use this driver. Need to > figure out way how to force it. May be through VK_ICD_FILENAMES. > > Yes you can do that with the VK_ICD_FILENAMES envvar. > > > > Step3: Once we have above broken driver ready we can start enabling code > required to process "COMPUTE" shaders. > > Any specific reason to start with compute shaders? (mostly curiosity). > I think compute shaders are the simplest things to get running with much less setup compared to other shader types. > In any case, those steps look good enough, although I lack any context of > the final target. The only thing that I miss is something like "writing the > most basic vulkan program we can, and get it working with the driver". > FWIW, when we did that for v3dv, we didn't even rendered anything, we just > used clear/stores and confirmed that the output was correct. This allowed > us starting with a basic driver that didn't even need a working backend > compiler. > > BR > > > with thanks and gratitude, > > l. > ___ > mesa-dev mailing > listmesa-dev@lists.freedesktop.orghttps://lists.freedesktop.org/mailman/listinfo/mesa-dev > > ___ mesa-dev mailing list mesa-dev@lists.freede
Re: [Mesa-dev] [Libre-soc-dev] Loading Vulkan Driver
On Sun, Aug 23, 2020 at 10:19 AM apinheiro wrote: > On 23/8/20 8:11, Luke Kenneth Casson Leighton wrote: > > next step is to add 3D opcodes *directly* to the POWER9 core that we > > are developing. > > Then, in addition to the drivers you already looking as reference, it > would make sense if you take a look to Vallium, that have just landed on > Mesa master > > https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6082 appreciated - we missed this one in the evaluation process (i think), so thank you. a primary objective is a native Vulkan driver. correct me if i am wrong, vallium is a "gallium-to-vulkan adapter"? i saw something about gallium which explained that gallium is not (yet) entirely suited to this task? also that there are threading limitations in gallium and/or llvmpipe. one of the reasons for picking a route that involves NIR is to leverage the significant high-performance work done there, because whilst libre-soc is a software-renderer, it's a software-renderer on hardware that has (will have) full GPU capabilities. if we used gallium (or started from SwiftShader) the superb work done by NIR passes would need to be duplicated. l. ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [Libre-soc-dev] Loading Vulkan Driver
--- crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68 On Sun, Aug 23, 2020 at 10:36 AM vivek pandya wrote: > > Hello all, > > I just cleanup a bit and commited code at : > https://gitlab.freedesktop.org/vivekvpandya/mesa/-/commit/d1f29adf926eb44453db1255834575a6f7169d5f > > However it is not working as per my expectation (or my expectations are > wrong) > > I want to see that driver fail at > https://gitlab.freedesktop.org/vivekvpandya/mesa/-/commit/d1f29adf926eb44453db1255834575a6f7169d5f#54a82819ff146aa01755bf951d797d55a0332a32_0_43 > however in debugger control never reach there. hm does it reach here? https://gitlab.freedesktop.org/vivekvpandya/mesa/-/blob/d1f29adf926eb44453db1255834575a6f7169d5f/src/libre-soc/vulkan/libresoc_device.c#L105 (can i recommend a #ifdef VERBOSE_DEBUG and some printfs: it's crude but effective and when restarting or when others try to replicate this they will not need to set large numbers of breakpoints. plus, the debug print statements become a discussion anchor / reference-point, and a form of documentation in themselves) about that: is there an "official" MESA macro for use to do verbose development-level debug output? i see the radv code checks an environment variable: https://gitlab.freedesktop.org/vivekvpandya/mesa/-/blob/d1f29adf926eb44453db1255834575a6f7169d5f/src/amd/vulkan/radv_device.c#L2569 > I see that in > https://gitlab.freedesktop.org/apinheiro/mesa/-/commit/07d01ebf6aae2f9ae71a8bea13a5d8acccb6280e > CreateShaderModule() is defined hmm i don't see an equivalent createShaderModule in here: https://gitlab.freedesktop.org/vivekvpandya/mesa/-/blob/d1f29adf926eb44453db1255834575a6f7169d5f/src/libre-soc/vulkan/libresoc_pipeline.c also, enumerateInstanceDeviceProperties is empty: https://gitlab.freedesktop.org/vivekvpandya/mesa/-/blob/d1f29adf926eb44453db1255834575a6f7169d5f/src/libre-soc/vulkan/libresoc_device.c#L259 whereas it is not, here: https://gitlab.freedesktop.org/apinheiro/mesa/-/commit/07d01ebf6aae2f9ae71a8bea13a5d8acccb6280e#0b80c3a6757284417a6c75db460ee183cd5e80dd_0_35 i would recommend putting debug printfs at the top of libresoc_GetInstanceProcAddr, to get a handle (ha ha) on what functions are being looked for. ah: it *might* also be worthwhile actually checking out that very early version of the broadcom driver, liberally sprinkling it with debug printfs at least at the start of every function, and see what is called, and in which order. by doing the same thing in the libresoc driver that would give you an idea of what is missing. the other thought that occurred to me: it could just be that expecting createGraphicsPipelines to be called is too early, and that to trigger it, a little more of the infrastructure has to be in place, such as responding to vkinfo: https://gitlab.axiodl.com/AxioDL/mesa/-/blob/abd629eb3d4027b89c13158e90c6732b412e550e/src/intel/vulkan/anv_device.c#L782 just a thought. l. ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [Libre-soc-dev] Loading Vulkan Driver
> crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68 > > On Sun, Aug 23, 2020 at 10:36 AM vivek pandya > wrote: > > > > Hello all, > > > > I just cleanup a bit and commited code at : > > > https://gitlab.freedesktop.org/vivekvpandya/mesa/-/commit/d1f29adf926eb44453db1255834575a6f7169d5f > > > > However it is not working as per my expectation (or my expectations are > > wrong) > > > > I want to see that driver fail at > > > https://gitlab.freedesktop.org/vivekvpandya/mesa/-/commit/d1f29adf926eb44453db1255834575a6f7169d5f#54a82819ff146aa01755bf951d797d55a0332a32_0_43 > > however in debugger control never reach there. > > hm does it reach here? > > https://gitlab.freedesktop.org/vivekvpandya/mesa/-/blob/d1f29adf926eb44453db1255834575a6f7169d5f/src/libre-soc/vulkan/libresoc_device.c#L105 > > (can i recommend a #ifdef VERBOSE_DEBUG and some printfs: it's crude > but effective and when restarting or when others try to replicate this > they will not need to set large numbers of breakpoints. plus, the > debug print statements become a discussion anchor / reference-point, > and a form of documentation in themselves) > > about that: is there an "official" MESA macro for use to do verbose > development-level debug output? i see the radv code checks an > environment variable: > > https://gitlab.freedesktop.org/vivekvpandya/mesa/-/blob/d1f29adf926eb44453db1255834575a6f7169d5f/src/amd/vulkan/radv_device.c#L2569 > > > > > I see that in > > > https://gitlab.freedesktop.org/apinheiro/mesa/-/commit/07d01ebf6aae2f9ae71a8bea13a5d8acccb6280e > > CreateShaderModule() is defined > > hmm i don't see an equivalent createShaderModule in here: > > https://gitlab.freedesktop.org/vivekvpandya/mesa/-/blob/d1f29adf926eb44453db1255834575a6f7169d5f/src/libre-soc/vulkan/libresoc_pipeline.c > > also, enumerateInstanceDeviceProperties is empty: > > https://gitlab.freedesktop.org/vivekvpandya/mesa/-/blob/d1f29adf926eb44453db1255834575a6f7169d5f/src/libre-soc/vulkan/libresoc_device.c#L259 > > whereas it is not, here: > > https://gitlab.freedesktop.org/apinheiro/mesa/-/commit/07d01ebf6aae2f9ae71a8bea13a5d8acccb6280e#0b80c3a6757284417a6c75db460ee183cd5e80dd_0_35 > > i would recommend putting debug printfs at the top of > libresoc_GetInstanceProcAddr, to get a handle (ha ha) on what > functions are being looked for. ah: it *might* also be worthwhile > actually checking out that very early version of the broadcom driver, > liberally sprinkling it with debug printfs at least at the start of > every function, and see what is called, and in which order. > > by doing the same thing in the libresoc driver that would give you an > idea of what is missing. > > the other thought that occurred to me: it could just be that expecting > createGraphicsPipelines to be called is too early, and that to trigger > it, a little more of the infrastructure has to be in place, such as > responding to vkinfo: > > https://gitlab.axiodl.com/AxioDL/mesa/-/blob/abd629eb3d4027b89c13158e90c6732b412e550e/src/intel/vulkan/anv_device.c#L782 > > I added few debug statements but still the same result: Here is my debug log: vivek@vivek-VirtualBox:~/mesa/dbg$ vulkaninfo GetProcAddr called for: vkCreateInstance GetProcAddr called for: vkEnumerateInstanceExtensionProperties libreSoc_EnumerateInstanceExtensionProperties called for layer: (null) libreSoc_EnumerateInstanceExtensionProperties called for layer: (null) GetProcAddr called for: vkCreateInstance GetProcAddr called for: vkEnumerateInstanceExtensionProperties libreSoc_EnumerateInstanceExtensionProperties called for layer: (null) libreSoc_EnumerateInstanceExtensionProperties called for layer: (null) GetProcAddr called for: vkCreateInstance GetProcAddr called for: vkEnumerateInstanceExtensionProperties libreSoc_EnumerateInstanceExtensionProperties called for layer: (null) libreSoc_EnumerateInstanceExtensionProperties called for layer: (null) libreSoc_EnumerateInstanceExtensionProperties called for layer: (null) libreSoc_EnumerateInstanceExtensionProperties called for layer: (null) GetProcAddr called for: vkEnumerateInstanceVersion GetProcAddr called for: vkDestroyInstance WARNING: WARNING: [Loader Message] Code 0 : terminator_CreateInstance: Failed to CreateInstance and find entrypoints with ICD. Skipping ICD. WARNING: terminator_CreateInstance: Failed to CreateInstance and find entrypoints with ICD. Skipping ICD. Cannot create Vulkan instance. This problem is often caused by a faulty installation of the Vulkan driver or attempting to use a GPU that does not support Vulkan. /build/vulkan-tools-KEbD_A/vulkan-tools-1.2.131.1+dfsg1/vulkaninfo/vulkaninfo.h:371: failed with ERROR_INCOMPATIBLE_DRIVER I hope this is not due to the VirtualBox environment. just a thought. > > l ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [Libre-soc-dev] Loading Vulkan Driver
More detailed log with VK_LOADER_DEBUG=1 INFO: Found ICD manifest file /home/vivek/install/share/vulkan/icd.d/libresoc_icd.x86_64.json, version "1.0.0" DEBUG: Searching for ICD drivers named /home/vivek/install/lib/x86_64-linux-gnu/libvulkan_libresoc.so GetProcAddr called for: vkCreateInstance GetProcAddr called for: vkEnumerateInstanceExtensionProperties DEBUG: Build ICD instance extension list libreSoc_EnumerateInstanceExtensionProperties called for layer: (null) libreSoc_EnumerateInstanceExtensionProperties called for layer: (null) DEBUG: Instance Extension: VK_KHR_surface (/home/vivek/install/lib/x86_64-linux-gnu/libvulkan_libresoc.so) version 0.0.25 DEBUG: Instance Extension: VK_KHR_display (/home/vivek/install/lib/x86_64-linux-gnu/libvulkan_libresoc.so) version 0.0.23 DEBUG: Instance Extension: VK_EXT_direct_mode_display (/home/vivek/install/lib/x86_64-linux-gnu/libvulkan_libresoc.so) version 0.0.1 DEBUG: Instance Extension: VK_EXT_display_surface_counter (/home/vivek/install/lib/x86_64-linux-gnu/libvulkan_libresoc.so) version 0.0.1 DEBUG: Instance Extension: VK_EXT_debug_report (/home/vivek/install/lib/x86_64-linux-gnu/libvulkan_libresoc.so) version 0.0.9 DEBUG: ReadDataFilesInSearchPaths: Searching the following paths for manifest files: /etc/xdg/xdg-ubuntu/vulkan/implicit_layer.d:/etc/xdg/vulkan/implicit_layer.d:/etc/vulkan/implicit_layer.d:/usr/share/ubuntu/vulkan/implicit_layer.d:/usr/local/share/vulkan/implicit_layer.d:/usr/share/vulkan/implicit_layer.d:/var/lib/snapd/desktop/vulkan/implicit_layer.d:/home/vivek/.local/share/vulkan/implicit_layer.d DEBUG: ReadDataFilesInSearchPaths: Searching the following paths for manifest files: /etc/xdg/xdg-ubuntu/vulkan/implicit_layer.d:/etc/xdg/vulkan/implicit_layer.d:/etc/vulkan/implicit_layer.d:/usr/share/ubuntu/vulkan/implicit_layer.d:/usr/local/share/vulkan/implicit_layer.d:/usr/share/vulkan/implicit_layer.d:/var/lib/snapd/desktop/vulkan/implicit_layer.d:/home/vivek/.local/share/vulkan/implicit_layer.d DEBUG: ReadDataFilesInSearchPaths: Searching the following paths for manifest files: /etc/xdg/xdg-ubuntu/vulkan/explicit_layer.d:/etc/xdg/vulkan/explicit_layer.d:/etc/vulkan/explicit_layer.d:/usr/share/ubuntu/vulkan/explicit_layer.d:/usr/local/share/vulkan/explicit_layer.d:/usr/share/vulkan/explicit_layer.d:/var/lib/snapd/desktop/vulkan/explicit_layer.d:/home/vivek/.local/share/vulkan/explicit_layer.d INFO: Found manifest file /usr/share/vulkan/explicit_layer.d/VkLayer_MESA_overlay.json, version "1.0.0" DEBUG: Adding VK_LAYER_LUNARG_standard_validation using the loader legacy path. This is not an error. INFO: Meta-layer VK_LAYER_LUNARG_standard_validation all 0 component layers appear to be valid. Is this version 0.0.25 and similar values look bad? On Sun, Aug 23, 2020 at 8:33 PM vivek pandya wrote: > > > > >> crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68 >> >> On Sun, Aug 23, 2020 at 10:36 AM vivek pandya >> wrote: >> > >> > Hello all, >> > >> > I just cleanup a bit and commited code at : >> > >> https://gitlab.freedesktop.org/vivekvpandya/mesa/-/commit/d1f29adf926eb44453db1255834575a6f7169d5f >> > >> > However it is not working as per my expectation (or my expectations are >> > wrong) >> > >> > I want to see that driver fail at >> > >> https://gitlab.freedesktop.org/vivekvpandya/mesa/-/commit/d1f29adf926eb44453db1255834575a6f7169d5f#54a82819ff146aa01755bf951d797d55a0332a32_0_43 >> > however in debugger control never reach there. >> >> hm does it reach here? >> >> https://gitlab.freedesktop.org/vivekvpandya/mesa/-/blob/d1f29adf926eb44453db1255834575a6f7169d5f/src/libre-soc/vulkan/libresoc_device.c#L105 >> >> (can i recommend a #ifdef VERBOSE_DEBUG and some printfs: it's crude >> but effective and when restarting or when others try to replicate this >> they will not need to set large numbers of breakpoints. plus, the >> debug print statements become a discussion anchor / reference-point, >> and a form of documentation in themselves) >> >> about that: is there an "official" MESA macro for use to do verbose >> development-level debug output? i see the radv code checks an >> environment variable: >> >> https://gitlab.freedesktop.org/vivekvpandya/mesa/-/blob/d1f29adf926eb44453db1255834575a6f7169d5f/src/amd/vulkan/radv_device.c#L2569 >> >> >> >> > I see that in >> > >> https://gitlab.freedesktop.org/apinheiro/mesa/-/commit/07d01ebf6aae2f9ae71a8bea13a5d8acccb6280e >> > CreateShaderModule() is defined >> >> hmm i don't see an equivalent createShaderModule in here: >> >> https://gitlab.freedesktop.org/vivekvpandya/mesa/-/blob/d1f29adf926eb44453db1255834575a6f7169d5f/src/libre-soc/vulkan/libresoc_pipeline.c >> >> also, enumerateInstanceDeviceProperties is empty: >> >> https://gitlab.freedesktop.org/vivekvpandya/mesa/-/blob/d1f29adf926eb44453db1255834575a6f7169d5f/src/libre-soc/vulkan/libresoc_device.c#L259 >> >> whereas it is not, here: >> >> https://gitlab.freedesktop.org/apinheiro/mesa/-/commit/07d01ebf6aae2f9ae71a8bea13a5d
Re: [Mesa-dev] [Libre-soc-dev] Loading Vulkan Driver
> (if i can fill in this bit) > > the context is: to augment the PowerISA Instruction set to add 3D > opcodes *directly* to PowerISA, with the guidance, full knowledge and > under the supervision of the OpenPOWER Foundation. > > this will include: adding sin, cos, atan2... *to PowerISA*. adding > texture interpolation instructions... *to PowerISA*. adding YUV2RGB > and FP32-to-RGB conversion instructions... *to PowerISA*. > > therefore, whilst the initial target is "general purpose scalar > non-accelerated non-vectorised soft-3D", this is simply because the > next step is to add 3D opcodes *directly* to the POWER9 core that we > are developing. > > > i mention this because normally, GPUs are considered completely > separate entities - completely separate processors. LibreSOC will be > the first ever libre-licensed *hybrid* 3D-capable CPU, i.e. where the > *main processor* ISA is augmented to no longer need a completely > separate GPU. > > i appreciate that i have said the same thing in several different > ways. this is because hybrid CPU/VPU/GPUs are so unusual and so "not > normal industry practice" (because they're easier to sell if they're > separate) that it takes a while to sink in. I don't want to get involved in half-thought out schemes here, but I'm going to indulge myself in one post. The programming model is what designs the driver: If your "GPU" is a scalar processor with 1000s of threads, then you want to operate it with a GPU driver. If your "GPU" is a wide-vector processor with only 10s of threads, you want to use llvmpipe style driver. llvmpipe takes NIR, vectorises it, converts it to LLVM IR, and uses LLVM to create an executable piece of code that for rasterization and compute is run multiple times across multiple threads on the CPU. The big thing doing what Jacob did before, and from memory where he went wrong despite advice to the contrary is he skipped the "vectorises it" stage, which is highly important for vector CPU execution, as opposed to scalar GPU. I'm guessing you want to add an LLVM backend for your "GPU" instructions (are these all vectorised?), and just work out how to use llvmpipe and vallium. Any perceived design issues in llvmpipe will be totally in the noise compared to designing a CPU rasterizer by yourself. But I'd also suggest not distracting people here with too much noise, if loading a skeleton vulkan driver is this hard, writing a rasterizer is infinitely harder. Dave. ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [Libre-soc-dev] Loading Vulkan Driver
On Sun, Aug 23, 2020, 12:49 Dave Airlie wrote: > The big thing doing what Jacob did before, and from memory where he > went wrong despite advice to the contrary is he skipped the > "vectorises it" stage, which is highly important for vector CPU > execution, as opposed to scalar GPU. > IIRC, my plan for Kazan all along was/is to vectorize and multi-thread it, I just didn't get that far during the GSoC portion due to running out of time, so, in order to get it to do something more impressive than parse SPIR-V I kludged a rasterizer onto it during the last few weeks of GSoC. Originally, my plan was to write a whole-function vectorization pass as a LLVM pass. The new plan is to have the vectorization pass be done in a new IR (shader-compiler-ir) before translating to LLVM IR since LLVM IR isn't very good at retaining the structured IR needed to do full-function vectorization. That also has the benefit of supporting non-LLVM backends, such as cranelift. For the rewritten version in Rust that I'm still working on (when not working on the hardware design side), I'm not going to skip ahead and actually get it to work properly. Jacob ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [Libre-soc-dev] Loading Vulkan Driver
On Sun, Aug 23, 2020 at 8:50 PM Dave Airlie wrote: > I don't want to get involved in half-thought out schemes here, [this has been in the planning stage for... almost 2 years now?] > but I'm > going to indulge myself in one post. appreciated. > The programming model is what designs the driver: > If your "GPU" is a scalar processor with 1000s of threads, then you > want to operate it with a GPU driver. > If your "GPU" is a wide-vector processor with only 10s of threads, you > want to use llvmpipe style driver. ironically due to it being a general-purpose processor with wide vector processing added, and custom vectorised 3D opcodes, and no SIMT, it strictly fits neither of those definitions, neither being SIMT, nor SIMD, and yet having the possibility (through a NoC such as OpenPITON) to run thousands of SMP cores (not SIMT threads: SMP cores). consequently we are in... "new ground". luckily, NLNet recognise that this is a Research Project. > The big thing doing what Jacob did before, and from memory where he > went wrong despite advice to the contrary is he skipped the > "vectorises it" stage, which is highly important for vector CPU > execution, as opposed to scalar GPU. i saw the cross-over message, he explained why that was as part of an early prototype (back in... 2018?) > I'm guessing you want to add an LLVM backend for your "GPU" > instructions (are these all vectorised?), yes they are. or, more to the point we're taking the *entire* scalar POWER9 ISA and vectorising that (and extending the regfiles to 128x 64-bit entries). vectors may be up to 64 elements (although doing so @ 32 bit takes 1/2 the entire 128-deep FP or INT regfile) > and just work out how to use llvmpipe and vallium. as i am focussing primarily on the hardware, and trusting jacob and vivek's knowledge, i am summarising. we decided to treat the CPU, from the perspective of this MESA driver, as more a GPU than a "SMT multi-processor". this will allow us the leg-room to *consider* adding SIMT at the hardware level in the future (2+ years). where from what i gather, if we go with llvmpipe right now that would not be possible. l. ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [Libre-soc-dev] Loading Vulkan Driver
On Mon, 24 Aug 2020 at 07:25, Luke Kenneth Casson Leighton wrote: > > On Sun, Aug 23, 2020 at 8:50 PM Dave Airlie wrote: > > > I don't want to get involved in half-thought out schemes here, > > [this has been in the planning stage for... almost 2 years now?] > > > but I'm > > going to indulge myself in one post. > > appreciated. > > > The programming model is what designs the driver: > > If your "GPU" is a scalar processor with 1000s of threads, then you > > want to operate it with a GPU driver. > > If your "GPU" is a wide-vector processor with only 10s of threads, you > > want to use llvmpipe style driver. > > ironically due to it being a general-purpose processor with wide > vector processing added, and custom vectorised 3D opcodes, and no > SIMT, it strictly fits neither of those definitions, neither being > SIMT, nor SIMD, and yet having the possibility (through a NoC such as > OpenPITON) to run thousands of SMP cores (not SIMT threads: SMP > cores). > > consequently we are in... "new ground". luckily, NLNet recognise that > this is a Research Project. > > > The big thing doing what Jacob did before, and from memory where he > > went wrong despite advice to the contrary is he skipped the > > "vectorises it" stage, which is highly important for vector CPU > > execution, as opposed to scalar GPU. > > i saw the cross-over message, he explained why that was as part of an > early prototype (back in... 2018?) > > > I'm guessing you want to add an LLVM backend for your "GPU" > > instructions (are these all vectorised?), > > yes they are. or, more to the point we're taking the *entire* scalar > POWER9 ISA and vectorising that (and extending the regfiles to 128x > 64-bit entries). vectors may be up to 64 elements (although doing so > @ 32 bit takes 1/2 the entire 128-deep FP or INT regfile) > > > and just work out how to use llvmpipe and vallium. > > as i am focussing primarily on the hardware, and trusting jacob and > vivek's knowledge, i am summarising. we decided to treat the CPU, > from the perspective of this MESA driver, as more a GPU than a "SMT > multi-processor". What is your work submission model then, Vulkan is designed around having work submitted to a secondary processor from a control processor. Do you intend for the device to be used only as a coprocessor, or as a CPU for normal tasks that can then use the features in their threads, because none of the API are designed around that model. They have a running thread of execution that queues stuff up to a separate execution domain. What represents a vulkan queue, what will be recorded into vulkan command buffers, what will execute those command buffers. These are the questions you need to ask yourself and answer before writing any code. Dave. ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [Libre-soc-dev] Loading Vulkan Driver
On Sun, Aug 23, 2020, 15:55 Dave Airlie wrote: > What is your work submission model then, Vulkan is designed around > having work submitted to a secondary processor from a control > processor. Do you intend for the device to be used only as a > coprocessor, or as a CPU for normal tasks that can then use the > features in their threads, because none of the API are designed around > that model. They have a running thread of execution that queues stuff > up to a separate execution domain. > It's intended to be a combination, where CPU threads schedule work and render threads dequeue and run the work, probably using something like Rayon: https://github.com/rayon-rs/rayon When a CPU thread waits in the Vulkan driver, it can possibly be used as a render thread instead of blocking on a futex, avoiding needing excessive numbers of Linux kernel-level threads. The CPU and render threads run on the same cores, as scheduled by Linux. > > What represents a vulkan queue, The rayon task queues. what will be recorded into vulkan > command buffers, a command sequence encoded as bytecode, a list of Rust enums, or something similar. what will execute those command buffers. the render threads will dequeue the command buffers, run through all the commands in them, and schedule the appropriate rendering tasks to rayon's task execution mechanism. These are > the questions you need to ask yourself and answer before writing any > code. Yup, did that 2 years ago, though I don't remember if I explicitly wrote them down before. Jacob ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [Libre-soc-dev] Loading Vulkan Driver
On Mon, 24 Aug 2020 at 09:10, Jacob Lifshay wrote: > > On Sun, Aug 23, 2020, 15:55 Dave Airlie wrote: >> >> What is your work submission model then, Vulkan is designed around >> having work submitted to a secondary processor from a control >> processor. Do you intend for the device to be used only as a >> coprocessor, or as a CPU for normal tasks that can then use the >> features in their threads, because none of the API are designed around >> that model. They have a running thread of execution that queues stuff >> up to a separate execution domain. > > > It's intended to be a combination, where CPU threads schedule work and render > threads dequeue and run the work, probably using something like Rayon: > https://github.com/rayon-rs/rayon > > When a CPU thread waits in the Vulkan driver, it can possibly be used as a > render thread instead of blocking on a futex, avoiding needing excessive > numbers of Linux kernel-level threads. > > The CPU and render threads run on the same cores, as scheduled by Linux. >> >> >> What represents a vulkan queue, > > > The rayon task queues. > >> what will be recorded into vulkan >> command buffers, > > > a command sequence encoded as bytecode, a list of Rust enums, or something > similar. > >> what will execute those command buffers. > > > the render threads will dequeue the command buffers, run through all the > commands in them, and schedule the appropriate rendering tasks to rayon's > task execution mechanism. > >> These are >> the questions you need to ask yourself and answer before writing any >> code. > > It's hard to know then what you can expect to leverage from Mesa for that sort of architecture, the SPIRV->NIR translation, and then you probably want some sort of driver specific NIR->LLVM translation, amdgpu is completely scalar, llvmpipe is manually vectorised, swr does something like you might want, afaik it does a scalar translation and then magic execution (but I haven't dug into it except to fix interfaces). I think you will hit problems with vectorisation, because it's always been a problem for every effort in this area, but if the CPU design is such that everything can be vectorised and you never hit a scalar path, and you workout how texture derivatives work early, it might be something prototype-able. But none of the current mesa drivers are even close to this architecture, llvmpipe or swr are probably a bit closer than anything, so it's likely you'll be doing most of this out-of-tree anyways. My other feeling is it sounds like over architecting, and reaching for the stars here, where it might be practical to bring vallium/llvmpipe up on the architecture first then work out how to do the big new thing, or bring up this architecture on an x86 chip and see it works at all. Dave. ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [Libre-soc-dev] Loading Vulkan Driver
On Sun, Aug 23, 2020, 16:31 Dave Airlie wrote: > It's hard to know then what you can expect to leverage from Mesa for > that sort of architecture, the SPIRV->NIR translation, and then you > probably want some sort of driver specific NIR->LLVM translation, > amdgpu is completely scalar, llvmpipe is manually vectorised, swr does > something like you might want, afaik it does a scalar translation and > then magic execution (but I haven't dug into it except to fix > interfaces). > Kazan isn't built on Mesa or NIR (though it was originally intended to integrate with Mesa). Luke decided that Libre-SOC should also have a similar Mesa-based driver and applied to NLNet for funding for it, Vivek is currently starting on implementing that Mesa driver. > > I think you will hit problems with vectorisation, because it's always > been a problem for every effort in this area, but if the CPU design is > such that everything can be vectorised and you never hit a scalar > path, and you workout how texture derivatives work early, it might be > something prototype-able. > My current plan for screen-space derivatives in Kazan is to have the fragment shaders vectorized in multiples of 4 pixels (IIRC there's a nvidia vulkan extension that basically requires that), so adjacent fragment shader can be subtracted as-needed to calculate derivatives. > > But none of the current mesa drivers are even close to this > architecture, llvmpipe or swr are probably a bit closer than anything, > so it's likely you'll be doing most of this out-of-tree anyways. My > other feeling is it sounds like over architecting, and reaching for > the stars here, where it might be practical to bring vallium/llvmpipe > I would expect it to be quite easy to get llvmpipe to run on Libre-SOC's processor, since it is PowerPC-compatible, it just won't have very good performance due to llvmpipe's architecture. up on the architecture first then work out how to do the big new > thing, or bring up this architecture on an x86 chip and see it works > at all. > The plan is to get Kazan working with vectorization on x86, then change the backend out to unmodified PowerPC (LLVM may scalarize the instructions here), then add the new vector ISA to the PowerPC backend, then add custom instructions as-needed to improve performance. Jacob ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [Libre-soc-dev] Loading Vulkan Driver
On Monday, August 24, 2020, Dave Airlie wrote: > > amdgpu is completely scalar, it is?? waah! that's new information to me. does it even squash vec2/3/4, predication and swizzle? what about the upstream amdgpu LLVM-IR? that still preserves vector intrinsics, right? i'm assuming that AMDVLK preserves vector intrinsics? AMDVLK's associated LLVM port was what ended up upstream, is that right? I think you will hit problems with vectorisation, because it's always > been a problem for every effort in this area, but if the CPU design is > such that everything can be vectorised and you never hit a scalar > path, that's the plan. every single scalar POWER9 opcode with very few exceptions (branch, trap) is vectorised. and you workout how texture derivatives work early, it might be > something prototype-able. good advice to plan on texture opcodes. thank you. > thing, or bring up this architecture on an x86 chip and see it works > at all. that's the plan. phase I. run on x86, IBM POWER9, etc. first [whilst still preserving vector intrinsics until as late as possible] by the time phase I is complete we hope that Simon Moll and Robin Kruppe's LLVM-IR Vector Intrinsics will have landed, at which point we can do an experimental LLVM port which supports LibreSOC POWER9 vector augmentation, swizzle, transcendentals and texturisation opcodes etc. l. -- --- crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68 ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [Libre-soc-dev] Loading Vulkan Driver
On Monday, August 24, 2020, Jacob Lifshay wrote: Kazan isn't built on Mesa or NIR (though it was originally intended to > integrate with Mesa). Luke decided that Libre-SOC should also have a > similar Mesa-based driver and applied to NLNet for funding for it, i'm amazed they said yes :) it means that we can do work on e.g LLVM under that budget rather than under the original (constrained) one covering Kazan. l. -- --- crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68 ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [Libre-soc-dev] Loading Vulkan Driver
On Sun, Aug 23, 2020, 17:08 Luke Kenneth Casson Leighton wrote: > > > On Monday, August 24, 2020, Dave Airlie wrote: > >> >> amdgpu is completely scalar, > > > it is?? waah! that's new information to me. does it even squash vec2/3/4, > predication and swizzle? > yes, iirc. > > what about the upstream amdgpu LLVM-IR? that still preserves vector > intrinsics, right? > > i'm assuming that AMDVLK preserves vector intrinsics? > > AMDVLK's associated LLVM port was what ended up upstream, is that right? > no, that is the existing LLVM backend from AMD's opengl/opencl drivers. amdvlk came later. Jacob ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [Libre-soc-dev] Loading Vulkan Driver
On Mon, 24 Aug 2020 at 10:12, Jacob Lifshay wrote: > > On Sun, Aug 23, 2020, 17:08 Luke Kenneth Casson Leighton > wrote: >> >> >> >> On Monday, August 24, 2020, Dave Airlie wrote: >>> >>> >>> amdgpu is completely scalar, >> >> >> it is?? waah! that's new information to me. does it even squash vec2/3/4, >> predication and swizzle? > > > yes, iirc. >> >> >> what about the upstream amdgpu LLVM-IR? that still preserves vector >> intrinsics, right? >> >> i'm assuming that AMDVLK preserves vector intrinsics? >> >> >> AMDVLK's associated LLVM port was what ended up upstream, is that right? > > > no, that is the existing LLVM backend from AMD's opengl/opencl drivers. > amdvlk came later. Those are the same codebase, amdvlk just uses a fork of llvm, but the differences are only minor changes for impedance mismatch and release timing, they never diverge more than necessary. Dave. ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [Libre-soc-dev] Loading Vulkan Driver
On Sun, Aug 23, 2020, 18:38 Dave Airlie wrote: > On Mon, 24 Aug 2020 at 10:12, Jacob Lifshay > wrote: > > no, that is the existing LLVM backend from AMD's opengl/opencl drivers. > amdvlk came later. > > Those are the same codebase, amdvlk just uses a fork of llvm, but the > differences are only minor changes for impedance mismatch and release > timing, they never diverge more than necessary. > yeah, what I had meant is that the llvm amdgpu backend was not originally created for amdvlk, since amdvlk didn't exist then. Jacob ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
