Re: [Mesa-dev] Plumbing explicit synchronization through the Linux ecosystem
On Tue, Mar 17, 2020 at 7:08 PM Jason Ekstrand wrote: > > On Tue, Mar 17, 2020 at 7:16 PM Jacob Lifshay > wrote: > > > > On Tue, Mar 17, 2020 at 11:14 AM Lucas Stach wrote: > > > > > > Am Dienstag, den 17.03.2020, 10:59 -0700 schrieb Jacob Lifshay: > > > > I think I found a userspace-accessible way to create sync_files and > > > > dma_fences that would fulfill the requirements: > > > > https://github.com/torvalds/linux/blob/master/drivers/dma-buf/sw_sync.c > > > > > > > > I'm just not sure if that's a good interface to use, since it appears > > > > to be designed only for debugging. Will have to check for additional > > > > requirements of signalling an error when the process that created the > > > > fence is killed. > > It is expressly only for debugging and testing. Exposing such an API > to userspace would break the finite time guarantees that are relied > upon to keep sync_file a secure API. Ok, I was figuring that was probably the case. > > > Something like that can certainly be lifted for general use if it makes > > > sense. But then with a software renderer I don't really see how fences > > > help you at all. With a software renderer you know exactly when the > > > frame is finished and you can just defer pushing it over to the next > > > pipeline element until that time. You won't gain any parallelism by > > > using fences as the CPU is busy doing the rendering and will not run > > > other stuff concurrently, right? > > > > There definitely may be other hardware and/or processes that can > > process some stuff concurrently with the main application, such as the > > compositor and or video encoding processes (for video capture). > > Additionally, from what I understand, sync_file is the standard way to > > export and import explicit synchronization between processes and > > between drivers on Linux, so it seems like a good idea to support it > > from an interoperability standpoint even if it turns out that there > > aren't any scheduling/timing benefits. > > There are different ways that one can handle interoperability, > however. One way is to try and make the software rasterizer look as > much like a GPU as possible: lots of threads to make things as > asynchronous as possible, "real" implementations of semaphores and > fences, etc. This is basically the route I've picked, though rather than making lots of native threads, I'm planning on having just one thread per core and have a work-stealing scheduler (inspired by Rust's rayon crate) schedule all the individual render/compute jobs, because that allows making a lot more jobs to allow finer load balancing. > Another is to let a SW rasterizer be a SW rasterizer: do > everything immediately, thread only so you can exercise all the CPU > cores, and minimally implement semaphores and fences well enough to > maintain compatibility. If you take the first approach, then we have > to solve all these problems with letting userspace create unsignaled > sync_files which it will signal later and figure out how to make it > safe. If you take the second approach, you'll only ever have to > return already signaled sync_files and there's no problem with the > sync_file finite time guarantees. The main issue with doing everything immediately is that a lot of the function calls that games expect to take a very short time (e.g. vkQueueSubmit) would instead take a much longer time, potentially causing problems. One idea for a safe userspace-backed sync_file is to have a step counter that counts down until the sync_file is ready, where if userspace doesn't tell it to count any steps in a certain amount of time, then the sync_file switches to the error state. This way, it will error shortly after a process deadlocks for some reason, while still having the finite-time guarantee. When the sync_file is created, the step counter would be set to the number of jobs that the fence is waiting on. It can also be set to pause the timeout to wait until another sync_file signals, to handle cases where a sync_file is waiting on a userspace process that is waiting on another sync_file. The main issue is that the kernel would have to make sure that the sync_file graph doesn't have loops, maybe by erroring all sync_files that it finds in the loop. Does that sound like a good idea? Jacob ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Plumbing explicit synchronization through the Linux ecosystem
On Tue, Mar 17, 2020 at 7:16 PM Jacob Lifshay wrote: > > On Tue, Mar 17, 2020 at 11:14 AM Lucas Stach wrote: > > > > Am Dienstag, den 17.03.2020, 10:59 -0700 schrieb Jacob Lifshay: > > > I think I found a userspace-accessible way to create sync_files and > > > dma_fences that would fulfill the requirements: > > > https://github.com/torvalds/linux/blob/master/drivers/dma-buf/sw_sync.c > > > > > > I'm just not sure if that's a good interface to use, since it appears > > > to be designed only for debugging. Will have to check for additional > > > requirements of signalling an error when the process that created the > > > fence is killed. It is expressly only for debugging and testing. Exposing such an API to userspace would break the finite time guarantees that are relied upon to keep sync_file a secure API. > > Something like that can certainly be lifted for general use if it makes > > sense. But then with a software renderer I don't really see how fences > > help you at all. With a software renderer you know exactly when the > > frame is finished and you can just defer pushing it over to the next > > pipeline element until that time. You won't gain any parallelism by > > using fences as the CPU is busy doing the rendering and will not run > > other stuff concurrently, right? > > There definitely may be other hardware and/or processes that can > process some stuff concurrently with the main application, such as the > compositor and or video encoding processes (for video capture). > Additionally, from what I understand, sync_file is the standard way to > export and import explicit synchronization between processes and > between drivers on Linux, so it seems like a good idea to support it > from an interoperability standpoint even if it turns out that there > aren't any scheduling/timing benefits. There are different ways that one can handle interoperability, however. One way is to try and make the software rasterizer look as much like a GPU as possible: lots of threads to make things as asynchronous as possible, "real" implementations of semaphores and fences, etc. Another is to let a SW rasterizer be a SW rasterizer: do everything immediately, thread only so you can exercise all the CPU cores, and minimally implement semaphores and fences well enough to maintain compatibility. If you take the first approach, then we have to solve all these problems with letting userspace create unsignaled sync_files which it will signal later and figure out how to make it safe. If you take the second approach, you'll only ever have to return already signaled sync_files and there's no problem with the sync_file finite time guarantees. --Jason ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Plumbing explicit synchronization through the Linux ecosystem
On Tue, Mar 17, 2020 at 11:14 AM Lucas Stach wrote: > > Am Dienstag, den 17.03.2020, 10:59 -0700 schrieb Jacob Lifshay: > > I think I found a userspace-accessible way to create sync_files and > > dma_fences that would fulfill the requirements: > > https://github.com/torvalds/linux/blob/master/drivers/dma-buf/sw_sync.c > > > > I'm just not sure if that's a good interface to use, since it appears > > to be designed only for debugging. Will have to check for additional > > requirements of signalling an error when the process that created the > > fence is killed. > > Something like that can certainly be lifted for general use if it makes > sense. But then with a software renderer I don't really see how fences > help you at all. With a software renderer you know exactly when the > frame is finished and you can just defer pushing it over to the next > pipeline element until that time. You won't gain any parallelism by > using fences as the CPU is busy doing the rendering and will not run > other stuff concurrently, right? There definitely may be other hardware and/or processes that can process some stuff concurrently with the main application, such as the compositor and or video encoding processes (for video capture). Additionally, from what I understand, sync_file is the standard way to export and import explicit synchronization between processes and between drivers on Linux, so it seems like a good idea to support it from an interoperability standpoint even if it turns out that there aren't any scheduling/timing benefits. Jacob ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Plumbing explicit synchronization through the Linux ecosystem
Le mardi 17 mars 2020 à 11:27 -0500, Jason Ekstrand a écrit : > On Tue, Mar 17, 2020 at 10:33 AM Nicolas Dufresne > wrote: > > Le lundi 16 mars 2020 à 23:15 +0200, Laurent Pinchart a écrit : > > > Hi Jason, > > > > > > On Mon, Mar 16, 2020 at 10:06:07AM -0500, Jason Ekstrand wrote: > > > > On Mon, Mar 16, 2020 at 5:20 AM Laurent Pinchart wrote: > > > > > On Wed, Mar 11, 2020 at 04:18:55PM -0400, Nicolas Dufresne wrote: > > > > > > (I know I'm going to be spammed by so many mailing list ...) > > > > > > > > > > > > Le mercredi 11 mars 2020 à 14:21 -0500, Jason Ekstrand a écrit : > > > > > > > On Wed, Mar 11, 2020 at 12:31 PM Jason Ekstrand > > > > > > > wrote: > > > > > > > > All, > > > > > > > > > > > > > > > > Sorry for casting such a broad net with this one. I'm sure most > > > > > > > > people > > > > > > > > who reply will get at least one mailing list rejection. > > > > > > > > However, this > > > > > > > > is an issue that affects a LOT of components and that's why it's > > > > > > > > thorny to begin with. Please pardon the length of this e-mail > > > > > > > > as > > > > > > > > well; I promise there's a concrete point/proposal at the end. > > > > > > > > > > > > > > > > > > > > > > > > Explicit synchronization is the future of graphics and media. > > > > > > > > At > > > > > > > > least, that seems to be the consensus among all the graphics > > > > > > > > people > > > > > > > > I've talked to. I had a chat with one of the lead Android > > > > > > > > graphics > > > > > > > > engineers recently who told me that doing explicit sync from > > > > > > > > the start > > > > > > > > was one of the best engineering decisions Android ever made. > > > > > > > > It's > > > > > > > > also the direction being taken by more modern APIs such as > > > > > > > > Vulkan. > > > > > > > > > > > > > > > > > > > > > > > > ## What are implicit and explicit synchronization? > > > > > > > > > > > > > > > > For those that aren't familiar with this space, GPUs, media > > > > > > > > encoders, > > > > > > > > etc. are massively parallel and synchronization of some form is > > > > > > > > required to ensure that everything happens in the right order > > > > > > > > and > > > > > > > > avoid data races. Implicit synchronization is when bits of > > > > > > > > work (3D, > > > > > > > > compute, video encode, etc.) are implicitly based on the > > > > > > > > absolute > > > > > > > > CPU-time order in which API calls occur. Explicit > > > > > > > > synchronization is > > > > > > > > when the client (whatever that means in any given context) > > > > > > > > provides > > > > > > > > the dependency graph explicitly via some sort of synchronization > > > > > > > > primitives. If you're still confused, consider the following > > > > > > > > examples: > > > > > > > > > > > > > > > > With OpenGL and EGL, almost everything is implicit sync. Say > > > > > > > > you have > > > > > > > > two OpenGL contexts sharing an image where one writes to it and > > > > > > > > the > > > > > > > > other textures from it. The way the OpenGL spec works, the > > > > > > > > client has > > > > > > > > to make the API calls to render to the image before (in CPU > > > > > > > > time) it > > > > > > > > makes the API calls which texture from the image. As long as > > > > > > > > it does > > > > > > > > this (and maybe inserts a glFlush?), the driver will ensure > > > > > > > > that the > > > > > > > > rendering completes before the texturing happens and you get > > > > > > > > correct > > > > > > > > contents. > > > > > > > > > > > > > > > > Implicit synchronization can also happen across processes. > > > > > > > > Wayland, > > > > > > > > for instance, is currently built on implicit sync where the > > > > > > > > client > > > > > > > > does their rendering and then does a hand-off (via > > > > > > > > wl_surface::commit) > > > > > > > > to tell the compositor it's done at which point the compositor > > > > > > > > can now > > > > > > > > texture from the surface. The hand-off ensures that the > > > > > > > > client's > > > > > > > > OpenGL API calls happen before the server's OpenGL API calls. > > > > > > > > > > > > > > > > A good example of explicit synchronization is the Vulkan API. > > > > > > > > There, > > > > > > > > a client (or multiple clients) can simultaneously build command > > > > > > > > buffers in different threads where one of those command buffers > > > > > > > > renders to an image and the other textures from it and then > > > > > > > > submit > > > > > > > > both of them at the same time with instructions to the driver > > > > > > > > for > > > > > > > > which order to execute them in. The execution order is > > > > > > > > described via > > > > > > > > the VkSemaphore primitive. With the new > > > > > > > > VK_KHR_timeline_semaphore > > > > > > > > extension, you can even submit the work which does the texturing > > > > > > > > BEFORE the work which does the rendering and
Re: [Mesa-dev] Plumbing explicit synchronization through the Linux ecosystem
Am Dienstag, den 17.03.2020, 10:59 -0700 schrieb Jacob Lifshay: > On Tue, Mar 17, 2020 at 10:21 AM Lucas Stach wrote: > > Am Dienstag, den 17.03.2020, 10:12 -0700 schrieb Jacob Lifshay: > > > One related issue with explicit sync using sync_file is that combined > > > CPUs/GPUs (the CPU cores *are* the GPU cores) that do all the > > > rendering in userspace (like llvmpipe but for Vulkan and with extra > > > instructions for GPU tasks) but need to synchronize with other > > > drivers/processes is that there should be some way to create an > > > explicit fence/semaphore from userspace and later signal it. This > > > seems to conflict with the requirement for a sync_file to complete in > > > finite time, since the user process could be stopped or killed. > > > > > > Any ideas? > > > > Finite just means "not infinite". If you stop the process that's doing > > part of the pipeline processing you block the pipeline, you get to keep > > the pieces in that case. > > Seems reasonable. > > > That's one of the issues with implicit sync > > that explicit may solve: a single client taking way too much time to > > render something can block the whole pipeline up until the display > > flip. With explicit sync the compositor can just decide to use the last > > client buffer if the latest buffer isn't ready by some deadline. > > > > With regard to the process getting killed: whatever you sync primitive > > is, you need to make sure to signal the fence (possibly with an error > > condition set) when you are not going to make progress anymore. So > > whatever your means to creating the sync_fd from your software renderer > > is, it needs to signal any outstanding fences on the sync_fd when the > > fd is closed. > > I think I found a userspace-accessible way to create sync_files and > dma_fences that would fulfill the requirements: > https://github.com/torvalds/linux/blob/master/drivers/dma-buf/sw_sync.c > > I'm just not sure if that's a good interface to use, since it appears > to be designed only for debugging. Will have to check for additional > requirements of signalling an error when the process that created the > fence is killed. Something like that can certainly be lifted for general use if it makes sense. But then with a software renderer I don't really see how fences help you at all. With a software renderer you know exactly when the frame is finished and you can just defer pushing it over to the next pipeline element until that time. You won't gain any parallelism by using fences as the CPU is busy doing the rendering and will not run other stuff concurrently, right? Regards, Lucas ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Plumbing explicit synchronization through the Linux ecosystem
On Tue, Mar 17, 2020 at 10:21 AM Lucas Stach wrote: > > Am Dienstag, den 17.03.2020, 10:12 -0700 schrieb Jacob Lifshay: > > One related issue with explicit sync using sync_file is that combined > > CPUs/GPUs (the CPU cores *are* the GPU cores) that do all the > > rendering in userspace (like llvmpipe but for Vulkan and with extra > > instructions for GPU tasks) but need to synchronize with other > > drivers/processes is that there should be some way to create an > > explicit fence/semaphore from userspace and later signal it. This > > seems to conflict with the requirement for a sync_file to complete in > > finite time, since the user process could be stopped or killed. > > > > Any ideas? > > Finite just means "not infinite". If you stop the process that's doing > part of the pipeline processing you block the pipeline, you get to keep > the pieces in that case. Seems reasonable. > That's one of the issues with implicit sync > that explicit may solve: a single client taking way too much time to > render something can block the whole pipeline up until the display > flip. With explicit sync the compositor can just decide to use the last > client buffer if the latest buffer isn't ready by some deadline. > > With regard to the process getting killed: whatever you sync primitive > is, you need to make sure to signal the fence (possibly with an error > condition set) when you are not going to make progress anymore. So > whatever your means to creating the sync_fd from your software renderer > is, it needs to signal any outstanding fences on the sync_fd when the > fd is closed. I think I found a userspace-accessible way to create sync_files and dma_fences that would fulfill the requirements: https://github.com/torvalds/linux/blob/master/drivers/dma-buf/sw_sync.c I'm just not sure if that's a good interface to use, since it appears to be designed only for debugging. Will have to check for additional requirements of signalling an error when the process that created the fence is killed. Jacob > > Regards, > Lucas > ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Plumbing explicit synchronization through the Linux ecosystem
Am Dienstag, den 17.03.2020, 11:33 -0400 schrieb Nicolas Dufresne: > Le lundi 16 mars 2020 à 23:15 +0200, Laurent Pinchart a écrit : > > Hi Jason, > > > > On Mon, Mar 16, 2020 at 10:06:07AM -0500, Jason Ekstrand wrote: > > > On Mon, Mar 16, 2020 at 5:20 AM Laurent Pinchart wrote: > > > > On Wed, Mar 11, 2020 at 04:18:55PM -0400, Nicolas Dufresne wrote: > > > > > (I know I'm going to be spammed by so many mailing list ...) > > > > > > > > > > Le mercredi 11 mars 2020 à 14:21 -0500, Jason Ekstrand a écrit : > > > > > > On Wed, Mar 11, 2020 at 12:31 PM Jason Ekstrand > > > > > > wrote: > > > > > > > All, > > > > > > > > > > > > > > Sorry for casting such a broad net with this one. I'm sure most > > > > > > > people > > > > > > > who reply will get at least one mailing list rejection. However, > > > > > > > this > > > > > > > is an issue that affects a LOT of components and that's why it's > > > > > > > thorny to begin with. Please pardon the length of this e-mail as > > > > > > > well; I promise there's a concrete point/proposal at the end. > > > > > > > > > > > > > > > > > > > > > Explicit synchronization is the future of graphics and media. At > > > > > > > least, that seems to be the consensus among all the graphics > > > > > > > people > > > > > > > I've talked to. I had a chat with one of the lead Android > > > > > > > graphics > > > > > > > engineers recently who told me that doing explicit sync from the > > > > > > > start > > > > > > > was one of the best engineering decisions Android ever made. It's > > > > > > > also the direction being taken by more modern APIs such as Vulkan. > > > > > > > > > > > > > > > > > > > > > ## What are implicit and explicit synchronization? > > > > > > > > > > > > > > For those that aren't familiar with this space, GPUs, media > > > > > > > encoders, > > > > > > > etc. are massively parallel and synchronization of some form is > > > > > > > required to ensure that everything happens in the right order and > > > > > > > avoid data races. Implicit synchronization is when bits of work > > > > > > > (3D, > > > > > > > compute, video encode, etc.) are implicitly based on the absolute > > > > > > > CPU-time order in which API calls occur. Explicit > > > > > > > synchronization is > > > > > > > when the client (whatever that means in any given context) > > > > > > > provides > > > > > > > the dependency graph explicitly via some sort of synchronization > > > > > > > primitives. If you're still confused, consider the following > > > > > > > examples: > > > > > > > > > > > > > > With OpenGL and EGL, almost everything is implicit sync. Say you > > > > > > > have > > > > > > > two OpenGL contexts sharing an image where one writes to it and > > > > > > > the > > > > > > > other textures from it. The way the OpenGL spec works, the > > > > > > > client has > > > > > > > to make the API calls to render to the image before (in CPU time) > > > > > > > it > > > > > > > makes the API calls which texture from the image. As long as it > > > > > > > does > > > > > > > this (and maybe inserts a glFlush?), the driver will ensure that > > > > > > > the > > > > > > > rendering completes before the texturing happens and you get > > > > > > > correct > > > > > > > contents. > > > > > > > > > > > > > > Implicit synchronization can also happen across processes. > > > > > > > Wayland, > > > > > > > for instance, is currently built on implicit sync where the client > > > > > > > does their rendering and then does a hand-off (via > > > > > > > wl_surface::commit) > > > > > > > to tell the compositor it's done at which point the compositor > > > > > > > can now > > > > > > > texture from the surface. The hand-off ensures that the client's > > > > > > > OpenGL API calls happen before the server's OpenGL API calls. > > > > > > > > > > > > > > A good example of explicit synchronization is the Vulkan API. > > > > > > > There, > > > > > > > a client (or multiple clients) can simultaneously build command > > > > > > > buffers in different threads where one of those command buffers > > > > > > > renders to an image and the other textures from it and then submit > > > > > > > both of them at the same time with instructions to the driver for > > > > > > > which order to execute them in. The execution order is described > > > > > > > via > > > > > > > the VkSemaphore primitive. With the new VK_KHR_timeline_semaphore > > > > > > > extension, you can even submit the work which does the texturing > > > > > > > BEFORE the work which does the rendering and the driver will sort > > > > > > > it > > > > > > > out. > > > > > > > > > > > > > > The #1 problem with implicit synchronization (which explicit > > > > > > > solves) > > > > > > > is that it leads to a lot of over-synchronization both in client > > > > > > > space > > > > > > > and in driver/device space. The client has to synchronize a lot > > > > > > > more > > > > > > > because it has to ensure that the
Re: [Mesa-dev] Plumbing explicit synchronization through the Linux ecosystem
Am Dienstag, den 17.03.2020, 10:12 -0700 schrieb Jacob Lifshay: > One related issue with explicit sync using sync_file is that combined > CPUs/GPUs (the CPU cores *are* the GPU cores) that do all the > rendering in userspace (like llvmpipe but for Vulkan and with extra > instructions for GPU tasks) but need to synchronize with other > drivers/processes is that there should be some way to create an > explicit fence/semaphore from userspace and later signal it. This > seems to conflict with the requirement for a sync_file to complete in > finite time, since the user process could be stopped or killed. > > Any ideas? Finite just means "not infinite". If you stop the process that's doing part of the pipeline processing you block the pipeline, you get to keep the pieces in that case. That's one of the issues with implicit sync that explicit may solve: a single client taking way too much time to render something can block the whole pipeline up until the display flip. With explicit sync the compositor can just decide to use the last client buffer if the latest buffer isn't ready by some deadline. With regard to the process getting killed: whatever you sync primitive is, you need to make sure to signal the fence (possibly with an error condition set) when you are not going to make progress anymore. So whatever your means to creating the sync_fd from your software renderer is, it needs to signal any outstanding fences on the sync_fd when the fd is closed. Regards, Lucas ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Plumbing explicit synchronization through the Linux ecosystem
On Tue, Mar 17, 2020 at 12:13 PM Jacob Lifshay wrote: > > One related issue with explicit sync using sync_file is that combined > CPUs/GPUs (the CPU cores *are* the GPU cores) that do all the > rendering in userspace (like llvmpipe but for Vulkan and with extra > instructions for GPU tasks) but need to synchronize with other > drivers/processes is that there should be some way to create an > explicit fence/semaphore from userspace and later signal it. This > seems to conflict with the requirement for a sync_file to complete in > finite time, since the user process could be stopped or killed. Yeah... That's going to be a problem. The only way I could see that working is if you created a sync_file that had a timeout associated with it. However, then you run into the issue where you may have corruption if stuff doesn't complete on time. Then again, you're not really dealing with an external unit and so the latency cost of going across the window system protocol probably isn't massively different from the latency cost of triggering the sync_file. Maybe the answer there is to just do everything in-order and not worry about synchronization? ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Plumbing explicit synchronization through the Linux ecosystem
On Tue., Mar. 17, 2020, 06:02 Michel Dänzer, wrote: > On 2020-03-16 7:33 p.m., Marek Olšák wrote: > > On Mon, Mar 16, 2020 at 5:57 AM Michel Dänzer > wrote: > >> On 2020-03-16 4:50 a.m., Marek Olšák wrote: > >>> The synchronization works because the Mesa driver waits for idle > (drains > >>> the GFX pipeline) at the end of command buffers and there is only 1 > >>> graphics queue, so everything is ordered. > >>> > >>> The GFX pipeline runs asynchronously to the command buffer, meaning the > >>> command buffer only starts draws and doesn't wait for completion. If > the > >>> Mesa driver didn't wait at the end of the command buffer, the command > >>> buffer would finish and a different process could start execution of > its > >>> own command buffer while shaders of the previous process are still > >> running. > >>> > >>> If the Mesa driver submits a command buffer internally (because it's > >> full), > >>> it doesn't wait, so the GFX pipeline doesn't notice that a command > buffer > >>> ended and a new one started. > >>> > >>> The waiting at the end of command buffers happens only when the flush > is > >>> external (Swap buffers, glFlush). > >>> > >>> It's a performance problem, because the GFX queue is blocked until the > >> GFX > >>> pipeline is drained at the end of every frame at least. > >>> > >>> So explicit fences for SwapBuffers would help. > >> > >> Not sure what difference it would make, since the same thing needs to be > >> done for explicit fences as well, doesn't it? > > > > No. Explicit fences don't require userspace to wait for idle in the > command > > buffer. Fences are signalled when the last draw is complete and caches > are > > flushed. Before that happens, any command buffer that is not dependent on > > the fence can start execution. There is never a need for the GPU to be > idle > > if there is enough independent work to do. > > I don't think explicit fences in the context of this discussion imply > using that different fence signalling mechanism though. My understanding > is that the API proposed by Jason allows implicit fences to be used as > explicit ones and vice versa, so presumably they have to use the same > signalling mechanism. > > > Anyway, maybe the different fence signalling mechanism you describe > could be used by the amdgpu kernel driver in general, then Mesa could > drop the waits for idle and get the benefits with implicit sync as well? > Yes. If there is any waiting, or should be done in the GPU scheduler, not in the command buffer, so that independent command buffers can use the GFX queue. Marek > > -- > Earthling Michel Dänzer | https://redhat.com > Libre software enthusiast | Mesa and X developer > ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Plumbing explicit synchronization through the Linux ecosystem
One related issue with explicit sync using sync_file is that combined CPUs/GPUs (the CPU cores *are* the GPU cores) that do all the rendering in userspace (like llvmpipe but for Vulkan and with extra instructions for GPU tasks) but need to synchronize with other drivers/processes is that there should be some way to create an explicit fence/semaphore from userspace and later signal it. This seems to conflict with the requirement for a sync_file to complete in finite time, since the user process could be stopped or killed. Any ideas? Jacob Lifshay ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Plumbing explicit synchronization through the Linux ecosystem
On Tue, Mar 17, 2020 at 10:33 AM Nicolas Dufresne wrote: > > Le lundi 16 mars 2020 à 23:15 +0200, Laurent Pinchart a écrit : > > Hi Jason, > > > > On Mon, Mar 16, 2020 at 10:06:07AM -0500, Jason Ekstrand wrote: > > > On Mon, Mar 16, 2020 at 5:20 AM Laurent Pinchart wrote: > > > > On Wed, Mar 11, 2020 at 04:18:55PM -0400, Nicolas Dufresne wrote: > > > > > (I know I'm going to be spammed by so many mailing list ...) > > > > > > > > > > Le mercredi 11 mars 2020 à 14:21 -0500, Jason Ekstrand a écrit : > > > > > > On Wed, Mar 11, 2020 at 12:31 PM Jason Ekstrand > > > > > > wrote: > > > > > > > All, > > > > > > > > > > > > > > Sorry for casting such a broad net with this one. I'm sure most > > > > > > > people > > > > > > > who reply will get at least one mailing list rejection. However, > > > > > > > this > > > > > > > is an issue that affects a LOT of components and that's why it's > > > > > > > thorny to begin with. Please pardon the length of this e-mail as > > > > > > > well; I promise there's a concrete point/proposal at the end. > > > > > > > > > > > > > > > > > > > > > Explicit synchronization is the future of graphics and media. At > > > > > > > least, that seems to be the consensus among all the graphics > > > > > > > people > > > > > > > I've talked to. I had a chat with one of the lead Android > > > > > > > graphics > > > > > > > engineers recently who told me that doing explicit sync from the > > > > > > > start > > > > > > > was one of the best engineering decisions Android ever made. It's > > > > > > > also the direction being taken by more modern APIs such as Vulkan. > > > > > > > > > > > > > > > > > > > > > ## What are implicit and explicit synchronization? > > > > > > > > > > > > > > For those that aren't familiar with this space, GPUs, media > > > > > > > encoders, > > > > > > > etc. are massively parallel and synchronization of some form is > > > > > > > required to ensure that everything happens in the right order and > > > > > > > avoid data races. Implicit synchronization is when bits of work > > > > > > > (3D, > > > > > > > compute, video encode, etc.) are implicitly based on the absolute > > > > > > > CPU-time order in which API calls occur. Explicit > > > > > > > synchronization is > > > > > > > when the client (whatever that means in any given context) > > > > > > > provides > > > > > > > the dependency graph explicitly via some sort of synchronization > > > > > > > primitives. If you're still confused, consider the following > > > > > > > examples: > > > > > > > > > > > > > > With OpenGL and EGL, almost everything is implicit sync. Say you > > > > > > > have > > > > > > > two OpenGL contexts sharing an image where one writes to it and > > > > > > > the > > > > > > > other textures from it. The way the OpenGL spec works, the > > > > > > > client has > > > > > > > to make the API calls to render to the image before (in CPU time) > > > > > > > it > > > > > > > makes the API calls which texture from the image. As long as it > > > > > > > does > > > > > > > this (and maybe inserts a glFlush?), the driver will ensure that > > > > > > > the > > > > > > > rendering completes before the texturing happens and you get > > > > > > > correct > > > > > > > contents. > > > > > > > > > > > > > > Implicit synchronization can also happen across processes. > > > > > > > Wayland, > > > > > > > for instance, is currently built on implicit sync where the client > > > > > > > does their rendering and then does a hand-off (via > > > > > > > wl_surface::commit) > > > > > > > to tell the compositor it's done at which point the compositor > > > > > > > can now > > > > > > > texture from the surface. The hand-off ensures that the client's > > > > > > > OpenGL API calls happen before the server's OpenGL API calls. > > > > > > > > > > > > > > A good example of explicit synchronization is the Vulkan API. > > > > > > > There, > > > > > > > a client (or multiple clients) can simultaneously build command > > > > > > > buffers in different threads where one of those command buffers > > > > > > > renders to an image and the other textures from it and then submit > > > > > > > both of them at the same time with instructions to the driver for > > > > > > > which order to execute them in. The execution order is described > > > > > > > via > > > > > > > the VkSemaphore primitive. With the new VK_KHR_timeline_semaphore > > > > > > > extension, you can even submit the work which does the texturing > > > > > > > BEFORE the work which does the rendering and the driver will sort > > > > > > > it > > > > > > > out. > > > > > > > > > > > > > > The #1 problem with implicit synchronization (which explicit > > > > > > > solves) > > > > > > > is that it leads to a lot of over-synchronization both in client > > > > > > > space > > > > > > > and in driver/device space. The client has to synchronize a lot > > > > > > > more > > > > > > > because it has to ensure that the API calls happen in a
Re: [Mesa-dev] Plumbing explicit synchronization through the Linux ecosystem
Le lundi 16 mars 2020 à 23:15 +0200, Laurent Pinchart a écrit : > Hi Jason, > > On Mon, Mar 16, 2020 at 10:06:07AM -0500, Jason Ekstrand wrote: > > On Mon, Mar 16, 2020 at 5:20 AM Laurent Pinchart wrote: > > > On Wed, Mar 11, 2020 at 04:18:55PM -0400, Nicolas Dufresne wrote: > > > > (I know I'm going to be spammed by so many mailing list ...) > > > > > > > > Le mercredi 11 mars 2020 à 14:21 -0500, Jason Ekstrand a écrit : > > > > > On Wed, Mar 11, 2020 at 12:31 PM Jason Ekstrand > > > > > wrote: > > > > > > All, > > > > > > > > > > > > Sorry for casting such a broad net with this one. I'm sure most > > > > > > people > > > > > > who reply will get at least one mailing list rejection. However, > > > > > > this > > > > > > is an issue that affects a LOT of components and that's why it's > > > > > > thorny to begin with. Please pardon the length of this e-mail as > > > > > > well; I promise there's a concrete point/proposal at the end. > > > > > > > > > > > > > > > > > > Explicit synchronization is the future of graphics and media. At > > > > > > least, that seems to be the consensus among all the graphics people > > > > > > I've talked to. I had a chat with one of the lead Android graphics > > > > > > engineers recently who told me that doing explicit sync from the > > > > > > start > > > > > > was one of the best engineering decisions Android ever made. It's > > > > > > also the direction being taken by more modern APIs such as Vulkan. > > > > > > > > > > > > > > > > > > ## What are implicit and explicit synchronization? > > > > > > > > > > > > For those that aren't familiar with this space, GPUs, media > > > > > > encoders, > > > > > > etc. are massively parallel and synchronization of some form is > > > > > > required to ensure that everything happens in the right order and > > > > > > avoid data races. Implicit synchronization is when bits of work > > > > > > (3D, > > > > > > compute, video encode, etc.) are implicitly based on the absolute > > > > > > CPU-time order in which API calls occur. Explicit synchronization > > > > > > is > > > > > > when the client (whatever that means in any given context) provides > > > > > > the dependency graph explicitly via some sort of synchronization > > > > > > primitives. If you're still confused, consider the following > > > > > > examples: > > > > > > > > > > > > With OpenGL and EGL, almost everything is implicit sync. Say you > > > > > > have > > > > > > two OpenGL contexts sharing an image where one writes to it and the > > > > > > other textures from it. The way the OpenGL spec works, the client > > > > > > has > > > > > > to make the API calls to render to the image before (in CPU time) it > > > > > > makes the API calls which texture from the image. As long as it > > > > > > does > > > > > > this (and maybe inserts a glFlush?), the driver will ensure that the > > > > > > rendering completes before the texturing happens and you get correct > > > > > > contents. > > > > > > > > > > > > Implicit synchronization can also happen across processes. Wayland, > > > > > > for instance, is currently built on implicit sync where the client > > > > > > does their rendering and then does a hand-off (via > > > > > > wl_surface::commit) > > > > > > to tell the compositor it's done at which point the compositor can > > > > > > now > > > > > > texture from the surface. The hand-off ensures that the client's > > > > > > OpenGL API calls happen before the server's OpenGL API calls. > > > > > > > > > > > > A good example of explicit synchronization is the Vulkan API. > > > > > > There, > > > > > > a client (or multiple clients) can simultaneously build command > > > > > > buffers in different threads where one of those command buffers > > > > > > renders to an image and the other textures from it and then submit > > > > > > both of them at the same time with instructions to the driver for > > > > > > which order to execute them in. The execution order is described > > > > > > via > > > > > > the VkSemaphore primitive. With the new VK_KHR_timeline_semaphore > > > > > > extension, you can even submit the work which does the texturing > > > > > > BEFORE the work which does the rendering and the driver will sort it > > > > > > out. > > > > > > > > > > > > The #1 problem with implicit synchronization (which explicit solves) > > > > > > is that it leads to a lot of over-synchronization both in client > > > > > > space > > > > > > and in driver/device space. The client has to synchronize a lot > > > > > > more > > > > > > because it has to ensure that the API calls happen in a particular > > > > > > order. The driver/device have to synchronize a lot more because > > > > > > they > > > > > > never know what is going to end up being a synchronization point as > > > > > > an > > > > > > API call on another thread/process may occur at any time. As we > > > > > > move > > > > > > to more and more multi-threaded programming this
Re: [Mesa-dev] Plumbing explicit synchronization through the Linux ecosystem
On Tue, Mar 17, 2020 at 3:01 AM Simon Ser wrote: > > On Monday, March 16, 2020 5:04 PM, Jason Ekstrand > wrote: > > > Hopefully, that will provide some motivation for other compositors > > (kwin, gnome-shell, etc.) because they now have a real user of it in > > an upstream driver for a major desktop platform and not just a few > > weston examples. However, someone is going to have to drive the > > actual development in those compositors. I'd be very happy if more > > people got involved, :-) > > FWIW, a wlroots pull request is in progress [0]. The plan is first to > accept fence FDs from clients, then send them our fences as a second > step. What exactly are the semantics there? Are you going to somehow wait inside wlroots for the buffer to be 100% idle or are you expecting the client to somehow use explicit for sending buffers implicit to wait for idle? If it's the latter, that's not going to work. --Jason ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Plumbing explicit synchronization through the Linux ecosystem
On 2020-03-16 7:33 p.m., Marek Olšák wrote: > On Mon, Mar 16, 2020 at 5:57 AM Michel Dänzer wrote: >> On 2020-03-16 4:50 a.m., Marek Olšák wrote: >>> The synchronization works because the Mesa driver waits for idle (drains >>> the GFX pipeline) at the end of command buffers and there is only 1 >>> graphics queue, so everything is ordered. >>> >>> The GFX pipeline runs asynchronously to the command buffer, meaning the >>> command buffer only starts draws and doesn't wait for completion. If the >>> Mesa driver didn't wait at the end of the command buffer, the command >>> buffer would finish and a different process could start execution of its >>> own command buffer while shaders of the previous process are still >> running. >>> >>> If the Mesa driver submits a command buffer internally (because it's >> full), >>> it doesn't wait, so the GFX pipeline doesn't notice that a command buffer >>> ended and a new one started. >>> >>> The waiting at the end of command buffers happens only when the flush is >>> external (Swap buffers, glFlush). >>> >>> It's a performance problem, because the GFX queue is blocked until the >> GFX >>> pipeline is drained at the end of every frame at least. >>> >>> So explicit fences for SwapBuffers would help. >> >> Not sure what difference it would make, since the same thing needs to be >> done for explicit fences as well, doesn't it? > > No. Explicit fences don't require userspace to wait for idle in the command > buffer. Fences are signalled when the last draw is complete and caches are > flushed. Before that happens, any command buffer that is not dependent on > the fence can start execution. There is never a need for the GPU to be idle > if there is enough independent work to do. I don't think explicit fences in the context of this discussion imply using that different fence signalling mechanism though. My understanding is that the API proposed by Jason allows implicit fences to be used as explicit ones and vice versa, so presumably they have to use the same signalling mechanism. Anyway, maybe the different fence signalling mechanism you describe could be used by the amdgpu kernel driver in general, then Mesa could drop the waits for idle and get the benefits with implicit sync as well? -- Earthling Michel Dänzer | https://redhat.com Libre software enthusiast | Mesa and X developer ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Plumbing explicit synchronization through the Linux ecosystem
On Monday, March 16, 2020 5:04 PM, Jason Ekstrand wrote: > Hopefully, that will provide some motivation for other compositors > (kwin, gnome-shell, etc.) because they now have a real user of it in > an upstream driver for a major desktop platform and not just a few > weston examples. However, someone is going to have to drive the > actual development in those compositors. I'd be very happy if more > people got involved, :-) FWIW, a wlroots pull request is in progress [0]. The plan is first to accept fence FDs from clients, then send them our fences as a second step. [0]: https://github.com/swaywm/wlroots/pull/2070 ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Plumbing explicit synchronization through the Linux ecosystem
On Mon, Mar 16, 2020 at 10:37:04PM -0500, Jason Ekstrand wrote: > On Mon, Mar 16, 2020 at 6:39 PM Roman Gilg wrote: > > > > On Wed, Mar 11, 2020 at 8:21 PM Jason Ekstrand wrote: > > > > > > On Wed, Mar 11, 2020 at 12:31 PM Jason Ekstrand > > > wrote: > > > > > > > > All, > > > > > > > > Sorry for casting such a broad net with this one. I'm sure most people > > > > who reply will get at least one mailing list rejection. However, this > > > > is an issue that affects a LOT of components and that's why it's > > > > thorny to begin with. Please pardon the length of this e-mail as > > > > well; I promise there's a concrete point/proposal at the end. > > > > > > > > > > > > Explicit synchronization is the future of graphics and media. At > > > > least, that seems to be the consensus among all the graphics people > > > > I've talked to. I had a chat with one of the lead Android graphics > > > > engineers recently who told me that doing explicit sync from the start > > > > was one of the best engineering decisions Android ever made. It's > > > > also the direction being taken by more modern APIs such as Vulkan. > > > > > > > > > > > > ## What are implicit and explicit synchronization? > > > > > > > > For those that aren't familiar with this space, GPUs, media encoders, > > > > etc. are massively parallel and synchronization of some form is > > > > required to ensure that everything happens in the right order and > > > > avoid data races. Implicit synchronization is when bits of work (3D, > > > > compute, video encode, etc.) are implicitly based on the absolute > > > > CPU-time order in which API calls occur. Explicit synchronization is > > > > when the client (whatever that means in any given context) provides > > > > the dependency graph explicitly via some sort of synchronization > > > > primitives. If you're still confused, consider the following > > > > examples: > > > > > > > > With OpenGL and EGL, almost everything is implicit sync. Say you have > > > > two OpenGL contexts sharing an image where one writes to it and the > > > > other textures from it. The way the OpenGL spec works, the client has > > > > to make the API calls to render to the image before (in CPU time) it > > > > makes the API calls which texture from the image. As long as it does > > > > this (and maybe inserts a glFlush?), the driver will ensure that the > > > > rendering completes before the texturing happens and you get correct > > > > contents. > > > > > > > > Implicit synchronization can also happen across processes. Wayland, > > > > for instance, is currently built on implicit sync where the client > > > > does their rendering and then does a hand-off (via wl_surface::commit) > > > > to tell the compositor it's done at which point the compositor can now > > > > texture from the surface. The hand-off ensures that the client's > > > > OpenGL API calls happen before the server's OpenGL API calls. > > > > > > > > A good example of explicit synchronization is the Vulkan API. There, > > > > a client (or multiple clients) can simultaneously build command > > > > buffers in different threads where one of those command buffers > > > > renders to an image and the other textures from it and then submit > > > > both of them at the same time with instructions to the driver for > > > > which order to execute them in. The execution order is described via > > > > the VkSemaphore primitive. With the new VK_KHR_timeline_semaphore > > > > extension, you can even submit the work which does the texturing > > > > BEFORE the work which does the rendering and the driver will sort it > > > > out. > > > > > > > > The #1 problem with implicit synchronization (which explicit solves) > > > > is that it leads to a lot of over-synchronization both in client space > > > > and in driver/device space. The client has to synchronize a lot more > > > > because it has to ensure that the API calls happen in a particular > > > > order. The driver/device have to synchronize a lot more because they > > > > never know what is going to end up being a synchronization point as an > > > > API call on another thread/process may occur at any time. As we move > > > > to more and more multi-threaded programming this synchronization (on > > > > the client-side especially) becomes more and more painful. > > > > > > > > > > > > ## Current status in Linux > > > > > > > > Implicit synchronization in Linux works via a the kernel's internal > > > > dma_buf and dma_fence data structures. A dma_fence is a tiny object > > > > which represents the "done" status for some bit of work. Typically, > > > > dma_fences are created as a by-product of someone submitting some bit > > > > of work (say, 3D rendering) to the kernel. The dma_buf object has a > > > > set of dma_fences on it representing shared (read) and exclusive > > > > (write) access to the object. When work is submitted which, for > > > > instance renders to the dma_buf, it's queued waiting on all the fences > >
Re: [Mesa-dev] 2020 X.Org Board of Directors Elections Nomination period is NOW
Just a quick reminder that both board nomination and membership renewal periods are still opening: - Send board nominations to elections AT x DOT org - Got to https://members.x.org/ to renew your membership (or become one to begin with!) Cheers, Daniel On Sun, Mar 8, 2020 at 8:51 PM Daniel Vetter wrote: > > We are seeking nominations for candidates for election to the X.Org > Foundation Board of Directors. All X.Org Foundation members are > eligible for election to the board. > > Nominations for the 202 election are now open and will remain open > until 23:59 UTC on 29th March 2020. > > The Board consists of directors elected from the membership. Each > year, an election is held to bring the total number of directors to > eight. The four members receiving the highest vote totals will serve > as directors for two year terms. > > The directors who received two year terms starting in 2019 wereSamuel > Iglesias Gonsálvez, Manasi D Navare, Lyude Paul and Daniel Vetter. > They will continue to serve until their term ends in 2021. Current > directors whose term expires in 2020 are Eric Anholt, Bryce > Harrington, Keith Packard and Harry Wentland. > > A director is expected to participate in the fortnightly IRC meeting > to discuss current business and to attend the annual meeting of the > X.Org Foundation, which will be held at a location determined in > advance by the Board of Directors. > > A member may nominate themselves or any other member they feel is > qualified. Nominations should be sent to the Election Committee at > elections at x.org. > > Nominees shall be required to be current members of the X.Org > Foundation, and submit a personal statement of up to 200 words that > will be provided to prospective voters. The collected statements, > along with the statement of contribution to the X.Org Foundation in > the member's account page on http://members.x.org, will be made > available to all voters to help them make their voting decisions. > > Nominations, membership applications or renewals and completed > personal statements must be received no later than 23:59 UTC on 02 > April 2020. > > The slate of candidates will be published 6 April 2020 and candidate > Q will begin then. The deadline for Xorg membership applications and > renewals is 02 April 2020. > > Cheers, Daniel, on behalf of the X.Org BoD > > PS: I cc'ed the usual dev lists since not many members put in the renewal yet. > -- > Daniel Vetter > Software Engineer, Intel Corporation > +41 (0) 79 365 57 48 - http://blog.ffwll.ch -- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
