Re: Future direction of the Mesa Vulkan runtime (or "should we build a new gallium?")
On Wed, Jan 24, 2024 at 6:57 PM Marek Olšák wrote: > > Gallium looks like it was just a copy of DX10, and likely many things were > known from DX10 in advance before anything started. Vulkanium doesn't have > anything to draw inspiration from. It's a completely unexplored idea. I'm not sure if I follow this. GNU/Linux didn't have a unified driver interface to implement GL, but Windows did have a standardized interface to implement D3D10 which we drew inspiration from. The same is still true if you s/GL/Vulkan/ and s/D3D10/D3D12/. It's just that more features of modern API's are tied to kernel features (i.e. wddm versions) than in the past, but with gpuvm, drm scheduler and syncobj that's also going to be Vulkan's path. Now, you might say that this time we're not going to use any lessons from Windows and this interface will be completely unlike what Windows does for D3D12, which is fine but I still wouldn't call the idea of standardizing an interface for a low level graphics API a completely unexplored idea given that it works on Windows on an api that's a lot more like Vulkan, than D3D10 was like GL. z
Re: Future direction of the Mesa Vulkan runtime (or "should we build a new gallium?")
Gallium looks like it was just a copy of DX10, and likely many things were known from DX10 in advance before anything started. Vulkanium doesn't have anything to draw inspiration from. It's a completely unexplored idea. AMD's PAL is the same idea as Gallium. It's used to implement Vulkan, DX, Mantle, Metal, etc. Marek On Wed, Jan 24, 2024, 13:40 Faith Ekstrand wrote: > On Wed, Jan 24, 2024 at 12:26 PM Zack Rusin > wrote: > > > > On Wed, Jan 24, 2024 at 10:27 AM Faith Ekstrand > wrote: > > > > > > Jose, > > > > > > Thanks for your thoughts! > > > > > > On Wed, Jan 24, 2024 at 4:30 AM Jose Fonseca < > jose.fons...@broadcom.com> wrote: > > > > > > > > I don't know much about the current Vulkan driver internals to have > or provide an informed opinion on the path forward, but I'd like to share > my backwards looking perspective. > > > > > > > > Looking back, Gallium was two things effectively: > > > > (1) an abstraction layer, that's watertight (as in upper layers > shouldn't reach through to lower layers) > > > > (2) an ecosystem of reusable components (draw, util, tgsi, etc.) > > > > > > > > (1) was of course important -- and the discipline it imposed is what > enabled to great simplifications -- but it also became a straight-jacket, > as GPUs didn't stand still, and sooner or later the > see-every-hardware-as-the-same lenses stop reflecting reality. > > > > > > > > If I had to pick one, I'd say that (2) is far more useful and > practical.Take components like gallium's draw and other util modules. A > driver can choose to use them or not. One could fork them within Mesa > source tree, and only the drivers that opt-in into the fork would need to > be tested/adapted/etc > > > > > > > > On the flip side, Vulkan API is already a pretty low level HW > abstraction. It's also very flexible and extensible, so it's hard to > provide a watertight abstraction underneath it without either taking the > lowest common denominator, or having lots of optional bits of functionality > governed by a myriad of caps like you alluded to. > > > > > > There is a third thing that isn't really recognized in your > description: > > > > > > (3) A common "language" to talk about GPUs and data structures that > > > represent that language > > > > > > This is precisely what the Vulkan runtime today doesn't have. Classic > > > meta sucked because we were trying to implement GL in GL. u_blitter, > > > on the other hand, is pretty fantastic because Gallium provides a much > > > more sane interface to write those common components in terms of. > > > > > > So far, we've been trying to build those components in terms of the > > > Vulkan API itself with calls jumping back into the dispatch table to > > > try and get inside the driver. This is working but it's getting more > > > and more fragile the more tools we add to that box. A lot of what I > > > want to do with gallium2 or whatever we're calling it is to fix our > > > layering problems so that calls go in one direction and we can > > > untangle the jumble. I'm still not sure what I want that to look like > > > but I think I want it to look a lot like Vulkan, just with a handier > > > interface. > > > > Yes, that makes sense. When we were writing the initial components for > > gallium (draw and cso) I really liked the general concept and thought > > about trying to reuse them in the old, non-gallium Mesa drivers but > > the obstacle was that there was no common interface to lay them on. > > Using GL to implement GL was silly and using Vulkan to implement > > Vulkan is not much better. > > > > Having said that my general thoughts on GPU abstractions largely match > > what Jose has said. To me it's a question of whether a clean > > abstraction: > > - on top of which you can build an entire GPU driver toolkit (i.e. all > > the components and helpers) > > - that makes it trivial to figure up what needs to be done to write a > > new driver and makes bootstrapping a new driver a lot simpler > > - that makes it easier to reason about cross hardware concepts (it's a > > lot easier to understand the entirety of the ecosystem if every driver > > is not doing something unique to implement similar functionality) > > is worth more than almost exponentially increasing the difficulty of: > > - advancing the ecosystem (i.e. it might be easier to understand but > > it's way harder to create clean abstractions across such different > > hardware). > > - driver maintenance (i.e. there will be a constant stream of > > regressions hitting your driver as a result of other people working on > > their drivers) > > - general development (i.e. bug fixes/new features being held back > > because they break some other driver) > > > > Some of those can certainly be titled one way or the other, e.g. the > > driver maintenance con be somewhat eased by requiring that every > > driver working on top of the new abstraction has to have a stable > > Mesa-CI setup (be it lava or ci-tron, or whatever) but all of those
[ANNOUNCE] mesa 24.0.0-rc3
Hello everyone, I'm happy to announce the next release candidate, 24.0.0-rc3. As always, if you find any issues please report them here: https://gitlab.freedesktop.org/mesa/mesa/-/issues/new Any issue that should block the release of 24.0.0 final, thus adding more 24.0.0-rc* release candidates, must be added to this milestone: https://gitlab.freedesktop.org/mesa/mesa/-/milestones/45 The next release candidate is expected in one week, on January 31st. Cheers, Eric --- Boris Brezillon (1): panfrost: Clamp the render area to the damage region Caio Oliveira (1): intel/compiler: Fix rebuilding the CFG in fs_combine_constants Daniel Schürmann (1): aco: give spiller more room to assign spilled SGPRs to VGPRs Dave Airlie (2): radv/video: refactor sq start/end code to avoid decode hangs. radv: don't submit empty command buffers on encoder ring. David Heidelberg (1): ci/deqp: uprev deqp-runner for Linux too to 0.18.0 Eric Engestrom (6): ci/deqp: ensure that in `default` builds, wayland + x11 + xcb are all built .pick_status.json: Update to d2b08f9437f692f6ff4be2512967973f18796cb2 .pick_status.json: Update to d0a3bac163ca803eda03feb3afea80e516568caf .pick_status.json: Update to 90939e93f6657e1334a9c5edd05e80344b17ff66 .pick_status.json: Update to eca4f0f632b1e3e6e24bd12ee5f00522eb7d0fdb VERSION: bump for 24.0.0-rc3 Eric R. Smith (1): panfrost: fix panfrost drm-shim Faith Ekstrand (1): nvk: Unref shaders on pipeline free Friedrich Vock (3): radv: Fix shader replay allocation condition nir: Make is_trivial_deref_cast public nir: Handle casts in nir_opt_copy_prop_vars Georg Lehmann (1): aco: stop scheduling at p_logical_end Hans-Kristian Arntzen (1): wsi/x11: Add workaround for Detroit Become Human. Ian Romanick (3): intel/compiler: Disable DPAS instructions on MTL intel/compiler: Track lower_dpas flag in brw_get_compiler_config_value intel/compiler: Track mue_compaction and mue_header_packing flags in brw_get_compiler_config_value Karol Herbst (3): nir: rework and fix rotate lowering nak/opt_out: fix comparison in try_combine_outs rusticl/kernel: check that local size on dispatch doesn't exceed limits Konstantin Seurer (4): ac/llvm: Enable helper invocations for quad OPs lavapipe: Fix DGC vertex buffer handling lavapipe: Mark vertex elements dirty if the stride changed lavapipe: Report the correct preprocess buffer size Lionel Landwerlin (2): anv: fix pipeline executable properties with graphics libraries anv: implement undocumented tile cache flush requirements Rhys Perry (3): radv: do nir_shader_gather_info after radv_nir_lower_rt_abi nir/lower_non_uniform: set non_uniform=false when lowering is not needed nir/lower_shader_calls: remove CF before nir_opt_if Ryan Neph (1): venus: fix shmem leak on vn_ring_destroy Samuel Pitoiset (2): radv: fix indirect dispatches on the compute queue on GFX7 radv: fix indirect draws with NULL index buffer on GFX10 Sil Vilerino (1): d3d12: Implement cap for PIPE_VIDEO_CAP_ENC_INTRA_REFRESH Tapani Pälli (3): iris: expand pre-hiz data cache flush to gfx >= 125 anv: expand pre-hiz data cache flush to gfx >= 125 iris: replace constant cache invalidate with hdc flush Yiwei Zhang (3): venus: fix pipeline layout lifetime venus: fix pipeline derivatives venus: fix to respect the final pipeline layout git tag: mesa-24.0.0-rc3 https://mesa.freedesktop.org/archive/mesa-24.0.0-rc3.tar.xz SHA256: 6285a5f43fddafb4b3b726b148a4a222460e892bd1b69aaff4d380c98355 mesa-24.0.0-rc3.tar.xz SHA512: 0cd619cc79198981061f25e9464cdffa56b5cffbda69b339e7da275af92527c3cd3d3b6ec0d80f920fe0aae025874a2f15f37a737be6e051738fa26bb2907164 mesa-24.0.0-rc3.tar.xz PGP: https://mesa.freedesktop.org/archive/mesa-24.0.0-rc3.tar.xz.sig signature.asc Description: PGP signature
[ANNOUNCE] mesa 23.3.4
Hello everyone, The bugfix release 23.3.4 is now available. If you find any issues, please report them here: https://gitlab.freedesktop.org/mesa/mesa/-/issues/new The next bugfix release is due in two weeks, on February 7th. Cheers, Eric --- Alessandro Astone (1): zink: Fix resizable BAR detection logic Boris Brezillon (3): panvk: Fix tracing panvk: Fix access to unitialized panvk_pipeline_layout::num_sets field panfrost: Clamp the render area to the damage region Daniel Schürmann (1): aco: give spiller more room to assign spilled SGPRs to VGPRs Dave Airlie (2): radv/video: refactor sq start/end code to avoid decode hangs. radv: don't submit empty command buffers on encoder ring. David Rosca (1): radeonsi/vcn: Fix H264 slice header when encoding I frames Eric Engestrom (9): docs: add sha256sum for 23.3.3 .pick_status.json: Update to 68f5277887aae1cdc202f45ecd44df2c3c59ba7d .pick_status.json: Update to 4fe5f06d400a7310ffc280761c27b036aec86646 .pick_status.json: Update to ff84aef116f9d0d13440fd13edf2ac0b69a8c132 .pick_status.json: Update to 6e4bb8253ed36f911a0a45dfecf89c237a8cd362 .pick_status.json: Update to d0a3bac163ca803eda03feb3afea80e516568caf .pick_status.json: Update to eca4f0f632b1e3e6e24bd12ee5f00522eb7d0fdb docs: add release notes for 23.3.4 VERSION: bump for 23.3.4 Friedrich Vock (4): radv/rt: Add workaround to make leaves always active radv: Fix shader replay allocation condition nir: Make is_trivial_deref_cast public nir: Handle casts in nir_opt_copy_prop_vars Georg Lehmann (1): aco: stop scheduling at p_logical_end Hans-Kristian Arntzen (1): wsi/x11: Add workaround for Detroit Become Human. Ian Romanick (1): intel/compiler: Track mue_compaction and mue_header_packing flags in brw_get_compiler_config_value Jesse Natalie (1): mesa: Consider mesa format in addition to internal format for mip/cube completeness Karol Herbst (3): rusticl/kernel: run opt/lower_memcpy later to fix a crash nir: rework and fix rotate lowering rusticl/kernel: check that local size on dispatch doesn't exceed limits Konstantin Seurer (4): ac/llvm: Enable helper invocations for quad OPs lavapipe: Fix DGC vertex buffer handling lavapipe: Mark vertex elements dirty if the stride changed lavapipe: Report the correct preprocess buffer size Lionel Landwerlin (4): anv: fix disabled Wa_14017076903/18022508906 anv: hide vendor ID for The Finals anv: fix pipeline executable properties with graphics libraries anv: implement undocumented tile cache flush requirements Lucas Stach (1): etnaviv: disable 64bpp render/sampler formats Matt Turner (4): symbols-check: Add _GLOBAL_OFFSET_TABLE_ nir: Fix cast util: Add DETECT_ARCH_HPPA macro util/tests: Disable half-float NaN test on hppa/old-mips Max R (1): d3d10umd: Fix compilation Mike Blumenkrantz (5): lavapipe: fix devenv icd filename zink: always force flushes when originating from api frontend zink: ignore tc buffer replacement info zink: fix buffer rebind early-out check zink: fix separate shader patch variable location adjustment Patrick Lerda (1): glsl/nir: fix gl_nir_cross_validate_outputs_to_inputs() memory leak Pavel Ondračka (1): r300: fix reusing of color varying slots for generic ones Pierre-Eric Pelloux-Prayer (2): ac/surface: don't oversize surf_size radeonsi: compute epitch when modifying surf_pitch Rhys Perry (3): radv: do nir_shader_gather_info after radv_nir_lower_rt_abi nir/lower_non_uniform: set non_uniform=false when lowering is not needed nir/lower_shader_calls: remove CF before nir_opt_if Samuel Pitoiset (2): radv: do not issue SQTT marker with DISPATCH_MESH_INDIRECT_MULTI radv: fix indirect dispatches on the compute queue on GFX7 Sviatoslav Peleshko (1): nir: Use alu source components count in nir_alu_srcs_negative_equal Tapani Pälli (4): anv: check for wa 16013994831 in emit_so_memcpy_end iris: expand pre-hiz data cache flush to gfx >= 125 anv: expand pre-hiz data cache flush to gfx >= 125 iris: replace constant cache invalidate with hdc flush Tatsuyuki Ishi (1): radv: never set DISABLE_WR_CONFIRM for CP DMA clears and copies Timur Kristóf (1): radv: Correctly select SDMA support for PRIME blit. Yiwei Zhang (4): vulkan/wsi/wayland: fix returns and avoid leaks for failed swapchain venus: fix pipeline layout lifetime venus: fix pipeline derivatives venus: fix to respect the final pipeline layout Yonggang Luo (1): compiler/spirv: The spirv shader is binary, should write in binary mode git tag: mesa-23.3.4 https://mesa.freedesktop.org/archive/mesa-23.3.4.tar.xz SHA256:
Re: Future direction of the Mesa Vulkan runtime (or "should we build a new gallium?")
I'll agree with Jose about Vulkan being a low-level abstraction, and to me the "opt-in" way seems like a much more balanced approach to achieving our goals — not only balanced between the goals themselves (code amount and time to implement aren't our only criteria to optimize), but also across the variety of hardware — as if something goes wrong with the watertight abstraction for a certain implementation, not only it'd take more time to find a solution, but issues of one driver risk wasting time of everyone as it'd often be necessary to make debatable changes to interfaces used by all drivers. I also need to further clarify my point regardless the design of what we want to encourage drivers to use, specifically about pipeline objects and dynamic state/ESO. Vulkan, as I see from all the perspectives I'm regularly interacting with it from — as an RHI programmer at a game studio, a translation layer developer (the Xenia Xbox 360 emulator), and now creating a driver for it — has not grown much thicker than it originally was. What has increased is its surface area — but where it's actually important: letting applications more precisely convey their intentions. I'd say it's even thinner and more transparent from this point of view now. We got nice things like inline uniform blocks, host image copy, push descriptors, descriptor buffers, and of course dynamic state — and they all pretty much directly correspond to some hardware concepts, that apps can utilize to do what they want with less indirection between their actual architecture and the hardware. Essentially, the application and the driver (and the rest of the chain — the specification, I'd like to retract my statement about "fighting" it, by the way, and the hardware controlled by that driver) can work more cooperatively now, towards their common goal of delivering what the app developer wants to provide to the user with as high quality and speed as realistically possible. They now have more ways of helping each other by communicating their intentions and capabilities to each other more completely and accurately. And it's important for us not to go *backwards*. This is why I think it's just fundamentally wrong to encourage drivers to layer pipeline objects and static state on top of dynamic state. An application would typically use static state when it: • Knows the potentially needed state setups in advance (like in a game with demands of materials preprocessed, or in a non-gaming/non-DCC app). • Wants to quickly apply a complete state configuration. • Maybe doesn't care much about the state used by previously done work, like drawing wildly different kinds of objects in a scene. At the same time, it'd choose dynamic if it: • Doesn't have upfront knowledge of possible states (like in an OpenGL/ D3D9/D3D11 translation layer or a console emulator, or with a highly flexible art pipeline in the game). • Wants to quickly make small, incremental state changes. • Maybe wills to mix state variables updated at different frequencies. Their use cases, and application's intentions they convey, are as opposite as the antonymous words "static" and "dynamic" they're called. Treating one like a specialization of the other is making the driver blind in the same way as back in 2016 when applications had no other option but to reduce everything to static state. (Of course with state spanning so many pipeline stages, applications would usually not just be picking one of the two extremes, and instead may want static for some cases/stages and dynamic for the other. This is also where the route Vulkan's development over the 8 years has taken is very wise: instead of forcing Escobar's axiom of choice upon applications, let them specify their intentions on a per-variable basis, and choose the appropriate amount of state grouping among monolithic pipelines, GPL with libraries containing one or multiple parts of a pipeline, and ESO.) The primary rule of game optimization is, if you can avoid doing something every frame, or, even worse, hundreds or thousands of times per frame, do whatever reuse you can to avoid that. If we know that's what the game wants to do — by providing a pipeline object with the state it wants to be static, a pipeline layout object — we should be aiding it. Just like if the the game tells us that it can't precompile something, the graphics stack should do the best it can in this situation — it would be wrong to add the overhead of running a time machine to 2016 to its draws either. After all, the driver's draw call code and the game's draw call code are both just draw call code with one common goal. So, it's important that whichever solution we end up with, it must not be a "broken telephone" degrading the cooperation between the application and the driver. And we should not forget that the communication between them is two-way, which includes: • Interface calls done by the app. • Limits and features exposed by the driver.
gfxstream + Mesa: the deal of the century?
tl;dr - Interested in porting the gfxstream virtualization solution to Mesa and wondering how everyone feels about it -- | Longer | -- Dear all, In an earnings call with investors, Santa Claus shared a grim outlook for the North Pole. The Q4 2023 holiday season's yield of milk and cookies -- the currency of the North Pole economy -- was once again drastically below expectations. As everyone knows, Santa Claus delivers presents to the people of the world, and in return is rewarded with milk and cookies. The milk and cookies are then used to feed the elves, who then create next year's presents. This arrangement has proved remarkably resilient since time immemorial: surviving war, feminine, plagues and even Episodes 7 through 9 of the Star Wars saga. However, the past few years have seen demand for a rather unusual present skyrocket. Letter after letter has arrived, saying: "Dear Santa, can we get gfxstream Vulkan ported to Mesa .. an upstream gfxsteam?!?" Jolly old Saint Nick, despite his considerable powers, is unable to deliver this most demanded present. As a result, cookie and milk reserves have fallen to dangerous levels. Santa Claus has been offered a leveraged buyout of the North Pole by Wall Street firm to fix the shortfall. The firm has a radical plan to cut costs and increase revenues: - instead of nutritious milk and cookies, the Elves will be fed doritos, sugary drinks and adderall - no more "naughty or nice" lists: presents will now be based on a monthly subscription fee. - selling ads on the sled - a dog food factory to 'process' the reindeer. There is still hope, my friends. In consultation with the Easter Bunny, we have come with a plan that just might work. With this RFC, we have a working port of gfxstream in Mesa. If all goes well, the Easter Bunny should be able to deliver gfxstream + Mesa in easter eggs, and generate an unprecedented amount of carrots in return. The carrots will be converted to milk and cookies, and the North Pole's economy will once again be thriving. But for this plan to work, we must port gfxstream into Mesa.. | Common Questions | Q: What is gfxstream? gfxstream is an API-virtualization solution https://android.googlesource.com/platform/hardware/google/gfxstream/ The history and lore of the project can be found here [a]. Q: Where will this be used and productized? Pretty much everywhere gfxstream is used now, namely: - Android Emulator [b] - Cuttlefish [c] - Google Play Games [d] - Fuchsia guests [e] - AAOS reference guests [f] - a few other more experimental projects These projects do reach O(million) users. Most projects aim for strong backwards compatibility: i.e, Android images from 2019 still work on the 2023 host emulator release, and efforts are made to make sure ABI changes don't break developers. However, in the non-emulator embedded space, gfxstream-ists do plan to ship the host driver in the guest. Q: Is gfxstream upstream in the Linux kernel sense? gfxstream is officially part of the virtio-gpu spec and supported on open-source VMMs like QEMU and crosvm. Recent virtio-gpu additions like blob resources [g] and context types [h] were first tested/enabled on gfxstream. Q: Where is the current MR? Current MR: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27246 The MR is very dirty at the moment, but do expect to Mesa-ify and clean it up further. It's around 216K LoC, but 70K LoC are pre-compiled shaders which we can do something about. We also have patches on top of the Vulkan-Docs repo (around 30K), which can be upstreamed elsewhere too. It's also one big mega-commit rather than 1000s of history preserving ones. But it should be enough to pass deqp-vk, and see the generalized structure. We endeavor to meet all coding standards of the Mesa project. But first we want see if there are any strong opinions on the general idea before investing in further Mesa-ification. Q: What are the reasons for porting gfxstream to Mesa? - Non-Android developers have expressed desire to decouple gfxstream from AOSP. AOSP is gigantic to download and developers also have to sign a CLA. Aside from Mesa, the only real option is Github, which we can all agree is a downgrade from Mesa. - The Mesa WSI layer uses gfxstream guest Linux support. Currently we just rip out parts of Mesa to make it work, but it's better to be in-tree. The other option was the ARM Mesa WSI layer, but Mesa WSI seemed better maintained. Q: What are the benefits to Mesa? As gfxstream becomes a Mesa driver, Mesa will be used more than ever in virtualized environments. We also see the benefits for improved maintenance of AOSP Mesa, which is now on a 3-year old commit. Having a project that is on the official support path of many Android projects will likely ensure Mesa Vulkan builds consistently in AOSP (plan is to use ANGLE for GLES). We have some Android.bp files
Re: Future direction of the Mesa Vulkan runtime (or "should we build a new gallium?")
On Wed, Jan 24, 2024 at 12:26 PM Zack Rusin wrote: > > On Wed, Jan 24, 2024 at 10:27 AM Faith Ekstrand wrote: > > > > Jose, > > > > Thanks for your thoughts! > > > > On Wed, Jan 24, 2024 at 4:30 AM Jose Fonseca wrote: > > > > > > I don't know much about the current Vulkan driver internals to have or provide an informed opinion on the path forward, but I'd like to share my backwards looking perspective. > > > > > > Looking back, Gallium was two things effectively: > > > (1) an abstraction layer, that's watertight (as in upper layers shouldn't reach through to lower layers) > > > (2) an ecosystem of reusable components (draw, util, tgsi, etc.) > > > > > > (1) was of course important -- and the discipline it imposed is what enabled to great simplifications -- but it also became a straight-jacket, as GPUs didn't stand still, and sooner or later the see-every-hardware-as-the-same lenses stop reflecting reality. > > > > > > If I had to pick one, I'd say that (2) is far more useful and practical.Take components like gallium's draw and other util modules. A driver can choose to use them or not. One could fork them within Mesa source tree, and only the drivers that opt-in into the fork would need to be tested/adapted/etc > > > > > > On the flip side, Vulkan API is already a pretty low level HW abstraction. It's also very flexible and extensible, so it's hard to provide a watertight abstraction underneath it without either taking the lowest common denominator, or having lots of optional bits of functionality governed by a myriad of caps like you alluded to. > > > > There is a third thing that isn't really recognized in your description: > > > > (3) A common "language" to talk about GPUs and data structures that > > represent that language > > > > This is precisely what the Vulkan runtime today doesn't have. Classic > > meta sucked because we were trying to implement GL in GL. u_blitter, > > on the other hand, is pretty fantastic because Gallium provides a much > > more sane interface to write those common components in terms of. > > > > So far, we've been trying to build those components in terms of the > > Vulkan API itself with calls jumping back into the dispatch table to > > try and get inside the driver. This is working but it's getting more > > and more fragile the more tools we add to that box. A lot of what I > > want to do with gallium2 or whatever we're calling it is to fix our > > layering problems so that calls go in one direction and we can > > untangle the jumble. I'm still not sure what I want that to look like > > but I think I want it to look a lot like Vulkan, just with a handier > > interface. > > Yes, that makes sense. When we were writing the initial components for > gallium (draw and cso) I really liked the general concept and thought > about trying to reuse them in the old, non-gallium Mesa drivers but > the obstacle was that there was no common interface to lay them on. > Using GL to implement GL was silly and using Vulkan to implement > Vulkan is not much better. > > Having said that my general thoughts on GPU abstractions largely match > what Jose has said. To me it's a question of whether a clean > abstraction: > - on top of which you can build an entire GPU driver toolkit (i.e. all > the components and helpers) > - that makes it trivial to figure up what needs to be done to write a > new driver and makes bootstrapping a new driver a lot simpler > - that makes it easier to reason about cross hardware concepts (it's a > lot easier to understand the entirety of the ecosystem if every driver > is not doing something unique to implement similar functionality) > is worth more than almost exponentially increasing the difficulty of: > - advancing the ecosystem (i.e. it might be easier to understand but > it's way harder to create clean abstractions across such different > hardware). > - driver maintenance (i.e. there will be a constant stream of > regressions hitting your driver as a result of other people working on > their drivers) > - general development (i.e. bug fixes/new features being held back > because they break some other driver) > > Some of those can certainly be titled one way or the other, e.g. the > driver maintenance con be somewhat eased by requiring that every > driver working on top of the new abstraction has to have a stable > Mesa-CI setup (be it lava or ci-tron, or whatever) but all of those > things need to be reasoned about. In my experience abstractions never > have uniform support because some people will value cons of them more > than they value the pros. So the entire process requires some very > steadfast individuals to keep going despite hearing that the effort is > dumb, at least until the benefits of the new approach are impossible > to deny. So you know... "how much do you believe in this approach > because some days will suck and you can't give up" ;) is probably the > question. Well, I've built my entire career out of doing things that others said
Re: Future direction of the Mesa Vulkan runtime (or "should we build a new gallium?")
On Wed, Jan 24, 2024 at 10:27 AM Faith Ekstrand wrote: > > Jose, > > Thanks for your thoughts! > > On Wed, Jan 24, 2024 at 4:30 AM Jose Fonseca > wrote: > > > > I don't know much about the current Vulkan driver internals to have or > > provide an informed opinion on the path forward, but I'd like to share my > > backwards looking perspective. > > > > Looking back, Gallium was two things effectively: > > (1) an abstraction layer, that's watertight (as in upper layers shouldn't > > reach through to lower layers) > > (2) an ecosystem of reusable components (draw, util, tgsi, etc.) > > > > (1) was of course important -- and the discipline it imposed is what > > enabled to great simplifications -- but it also became a straight-jacket, > > as GPUs didn't stand still, and sooner or later the > > see-every-hardware-as-the-same lenses stop reflecting reality. > > > > If I had to pick one, I'd say that (2) is far more useful and practical. > > Take components like gallium's draw and other util modules. A driver can > > choose to use them or not. One could fork them within Mesa source tree, > > and only the drivers that opt-in into the fork would need to be > > tested/adapted/etc > > > > On the flip side, Vulkan API is already a pretty low level HW abstraction. > > It's also very flexible and extensible, so it's hard to provide a > > watertight abstraction underneath it without either taking the lowest > > common denominator, or having lots of optional bits of functionality > > governed by a myriad of caps like you alluded to. > > There is a third thing that isn't really recognized in your description: > > (3) A common "language" to talk about GPUs and data structures that > represent that language > > This is precisely what the Vulkan runtime today doesn't have. Classic > meta sucked because we were trying to implement GL in GL. u_blitter, > on the other hand, is pretty fantastic because Gallium provides a much > more sane interface to write those common components in terms of. > > So far, we've been trying to build those components in terms of the > Vulkan API itself with calls jumping back into the dispatch table to > try and get inside the driver. This is working but it's getting more > and more fragile the more tools we add to that box. A lot of what I > want to do with gallium2 or whatever we're calling it is to fix our > layering problems so that calls go in one direction and we can > untangle the jumble. I'm still not sure what I want that to look like > but I think I want it to look a lot like Vulkan, just with a handier > interface. Yes, that makes sense. When we were writing the initial components for gallium (draw and cso) I really liked the general concept and thought about trying to reuse them in the old, non-gallium Mesa drivers but the obstacle was that there was no common interface to lay them on. Using GL to implement GL was silly and using Vulkan to implement Vulkan is not much better. Having said that my general thoughts on GPU abstractions largely match what Jose has said. To me it's a question of whether a clean abstraction: - on top of which you can build an entire GPU driver toolkit (i.e. all the components and helpers) - that makes it trivial to figure up what needs to be done to write a new driver and makes bootstrapping a new driver a lot simpler - that makes it easier to reason about cross hardware concepts (it's a lot easier to understand the entirety of the ecosystem if every driver is not doing something unique to implement similar functionality) is worth more than almost exponentially increasing the difficulty of: - advancing the ecosystem (i.e. it might be easier to understand but it's way harder to create clean abstractions across such different hardware). - driver maintenance (i.e. there will be a constant stream of regressions hitting your driver as a result of other people working on their drivers) - general development (i.e. bug fixes/new features being held back because they break some other driver) Some of those can certainly be titled one way or the other, e.g. the driver maintenance con be somewhat eased by requiring that every driver working on top of the new abstraction has to have a stable Mesa-CI setup (be it lava or ci-tron, or whatever) but all of those things need to be reasoned about. In my experience abstractions never have uniform support because some people will value cons of them more than they value the pros. So the entire process requires some very steadfast individuals to keep going despite hearing that the effort is dumb, at least until the benefits of the new approach are impossible to deny. So you know... "how much do you believe in this approach because some days will suck and you can't give up" ;) is probably the question. z
Re: Future direction of the Mesa Vulkan runtime (or "should we build a new gallium?")
Jose, Thanks for your thoughts! On Wed, Jan 24, 2024 at 4:30 AM Jose Fonseca wrote: > > I don't know much about the current Vulkan driver internals to have or > provide an informed opinion on the path forward, but I'd like to share my > backwards looking perspective. > > Looking back, Gallium was two things effectively: > (1) an abstraction layer, that's watertight (as in upper layers shouldn't > reach through to lower layers) > (2) an ecosystem of reusable components (draw, util, tgsi, etc.) > > (1) was of course important -- and the discipline it imposed is what enabled > to great simplifications -- but it also became a straight-jacket, as GPUs > didn't stand still, and sooner or later the see-every-hardware-as-the-same > lenses stop reflecting reality. > > If I had to pick one, I'd say that (2) is far more useful and practical. > Take components like gallium's draw and other util modules. A driver can > choose to use them or not. One could fork them within Mesa source tree, and > only the drivers that opt-in into the fork would need to be tested/adapted/etc > > On the flip side, Vulkan API is already a pretty low level HW abstraction. > It's also very flexible and extensible, so it's hard to provide a watertight > abstraction underneath it without either taking the lowest common > denominator, or having lots of optional bits of functionality governed by a > myriad of caps like you alluded to. There is a third thing that isn't really recognized in your description: (3) A common "language" to talk about GPUs and data structures that represent that language This is precisely what the Vulkan runtime today doesn't have. Classic meta sucked because we were trying to implement GL in GL. u_blitter, on the other hand, is pretty fantastic because Gallium provides a much more sane interface to write those common components in terms of. So far, we've been trying to build those components in terms of the Vulkan API itself with calls jumping back into the dispatch table to try and get inside the driver. This is working but it's getting more and more fragile the more tools we add to that box. A lot of what I want to do with gallium2 or whatever we're calling it is to fix our layering problems so that calls go in one direction and we can untangle the jumble. I'm still not sure what I want that to look like but I think I want it to look a lot like Vulkan, just with a handier interface. ~Faith > Not sure how useful this is in practice to you, but the lesson from my POV is > that opt-in reusable and shared libraries are always time well spent as they > can bend and adapt with the times, whereas no opt-out watertight abstractions > inherently have a shelf life. > > Jose > > On Fri, Jan 19, 2024 at 5:30 PM Faith Ekstrand wrote: >> >> Yeah, this one's gonna hit Phoronix... >> >> When we started writing Vulkan drivers back in the day, there was this >> notion that Vulkan was a low-level API that directly targets hardware. >> Vulkan drivers were these super thin things that just blasted packets >> straight into the hardware. What little code was common was small and >> pretty easy to just copy+paste around. It was a nice thought... >> >> What's happened in the intervening 8 years is that Vulkan has grown. A lot. >> >> We already have several places where we're doing significant layering. >> It started with sharing the WSI code and some Python for generating >> dispatch tables. Later we added common synchronization code and a few >> vkFoo2 wrappers. Then render passes and... >> >> https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27024 >> >> That's been my project the last couple weeks: A common VkPipeline >> implementation built on top of an ESO-like interface. The big >> deviation this MR makes from prior art is that I make no attempt at >> pretending it's a layered implementation. The vtable for shader >> objects looks like ESO but takes its own path when it's useful to do >> so. For instance, shader creation always consumes NIR and a handful of >> lowering passes are run for you. It's no st_glsl_to_nir but it is a >> bit opinionated. Also, a few of the bits that are missing from ESO >> such as robustness have been added to the interface. >> >> In my mind, this marks a pretty fundamental shift in how the Vulkan >> runtime works, at least in my mind. Previously, everything was >> designed to be a toolbox where you can kind of pick and choose what >> you want to use. Also, everything at least tried to act like a layer >> where you still implemented Vulkan but you could leave out bits like >> render passes if you implemented the new thing and were okay with the >> layer. With the ESO code, you implement something that isn't Vulkan >> entrypoints and the actual entrypoints live in the runtime. This lets >> us expand and adjust the interface as needed for our purposes as well >> as sanitize certain things even in the modern API. >> >> The result is that NVK is starting to feel like a
Re: Future direction of the Mesa Vulkan runtime (or "should we build a new gallium?")
I don't know much about the current Vulkan driver internals to have or provide an informed opinion on the path forward, but I'd like to share my backwards looking perspective. Looking back, Gallium was two things effectively: (1) an abstraction layer, that's watertight (as in upper layers shouldn't reach through to lower layers) (2) an ecosystem of reusable components (draw, util, tgsi, etc.) (1) was of course important -- and the discipline it imposed is what enabled to great simplifications -- but it also became a straight-jacket, as GPUs didn't stand still, and sooner or later the see-every-hardware-as-the-same lenses stop reflecting reality. If I had to pick one, I'd say that (2) is far more useful and practical. Take components like gallium's draw and other util modules. A driver can choose to use them or not. One could fork them within Mesa source tree, and only the drivers that opt-in into the fork would need to be tested/adapted/etc On the flip side, Vulkan API is already a pretty low level HW abstraction. It's also very flexible and extensible, so it's hard to provide a watertight abstraction underneath it without either taking the lowest common denominator, or having lots of optional bits of functionality governed by a myriad of caps like you alluded to. Not sure how useful this is in practice to you, but the lesson from my POV is that *opt-in* reusable and shared libraries are always time well spent as they can bend and adapt with the times, whereas *no opt-out* watertight abstractions inherently have a shelf life. Jose On Fri, Jan 19, 2024 at 5:30 PM Faith Ekstrand wrote: > Yeah, this one's gonna hit Phoronix... > > When we started writing Vulkan drivers back in the day, there was this > notion that Vulkan was a low-level API that directly targets hardware. > Vulkan drivers were these super thin things that just blasted packets > straight into the hardware. What little code was common was small and > pretty easy to just copy+paste around. It was a nice thought... > > What's happened in the intervening 8 years is that Vulkan has grown. A lot. > > We already have several places where we're doing significant layering. > It started with sharing the WSI code and some Python for generating > dispatch tables. Later we added common synchronization code and a few > vkFoo2 wrappers. Then render passes and... > > https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27024 > > That's been my project the last couple weeks: A common VkPipeline > implementation built on top of an ESO-like interface. The big > deviation this MR makes from prior art is that I make no attempt at > pretending it's a layered implementation. The vtable for shader > objects looks like ESO but takes its own path when it's useful to do > so. For instance, shader creation always consumes NIR and a handful of > lowering passes are run for you. It's no st_glsl_to_nir but it is a > bit opinionated. Also, a few of the bits that are missing from ESO > such as robustness have been added to the interface. > > In my mind, this marks a pretty fundamental shift in how the Vulkan > runtime works, at least in my mind. Previously, everything was > designed to be a toolbox where you can kind of pick and choose what > you want to use. Also, everything at least tried to act like a layer > where you still implemented Vulkan but you could leave out bits like > render passes if you implemented the new thing and were okay with the > layer. With the ESO code, you implement something that isn't Vulkan > entrypoints and the actual entrypoints live in the runtime. This lets > us expand and adjust the interface as needed for our purposes as well > as sanitize certain things even in the modern API. > > The result is that NVK is starting to feel like a gallium driver. > > So here's the question: do we like this? Do we want to push in this > direction? Should we start making more things work more this way? I'm > not looking for MRs just yet nor do I have more reworks directly > planned. I'm more looking for thoughts and opinions as to how the > various Vulkan driver teams feel about this. We'll leave the detailed > planning for the Mesa issue tracker. > > It's worth noting that, even though I said we've tried to keep things > layerish, there are other parts of the runtime that look like this. > The synchronization code is a good example. The vk_sync interface is > pretty significantly different from the Vulkan objects it's used to > implement. That's worked out pretty well, IMO. With as complicated as > something like pipelines or synchronization are, trying to keep the > illusion of a layer just isn't practical. > > So, do we like this? Should we be pushing more towards drivers being a > backed of the runtime instead of a user of it? > > Now, before anyone asks, no, I don't really want to build a multi-API > abstraction with a Vulkan state tracker. If we were doing this 5 years > ago and Zink didn't already exist, one might be able to make an >
