Re: [Intel-gfx] [RFC] xf86-video-intel: enable hw-generated binding tables
Hi all! On Thursday, April 24, 2014 01:17:14 PM Ville Syrjälä wrote: On Thu, Apr 24, 2014 at 11:25:15AM +0300, Abdiel Janulgue wrote: On Thursday, April 24, 2014 07:06:34 AM Chris Wilson wrote: On Thu, Apr 24, 2014 at 09:08:14AM +0300, Abdiel Janulgue wrote: Anyway I haven't tried the work-around where we explictly only disable the BT and RS on the other user-space clients (xorg driver in this case) when Mesa is using RS instead of forcing the reset of the clients to use RS format. I'll try that first and let you know if it works. I hate to break the bad news. Tried this just now - still get hangs :( So I guess, all userspace clients* does need to use RS-format if we use this feature. GPGPU workloads seems to be special use-case where the RS hwbinding table format can be disabled. Otherwise, I guess we are stuck with this inflexibility. The spec also says this: When switching between HW and SW binding table generation, SW must issue a state cache invalidate. So it does look like they were expecting people to switch between the two modes. Do you have any igt test for RS? Maybe add an option into rendercopy to make use of RS? Then we could write some tests that try to hit this problem w/o requiring Mesa. I've modified igt rendercopy to test switching RS format on and off. Good news and bad news! Good news is that switching between RS binding tables and SW format works seamlessly in BDW! Bad news for HSW because the problems I describe above only happens on that chip. This is a HW issue then. =Abdiel [1] On the other hand, it doesn't seem all that bad though. The RS hw-binding table format are only needed for clients that submit vertex and pixel shader commands. I've identified currently just UXA and SNA that seem to use this besides Mesa. OpenCL is not affected. If it does, it might be more efficient to do that in the kernel? It has to be done in the kernel in order for interoperability with third party clients. -Chris ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [Intel-gfx] [RFC] xf86-video-intel: enable hw-generated binding tables
On Thu, Apr 24, 2014 at 09:08:14AM +0300, Abdiel Janulgue wrote: Anyway I haven't tried the work-around where we explictly only disable the BT and RS on the other user-space clients (xorg driver in this case) when Mesa is using RS instead of forcing the reset of the clients to use RS format. I'll try that first and let you know if it works. If it does, it might be more efficient to do that in the kernel? It has to be done in the kernel in order for interoperability with third party clients. -Chris -- Chris Wilson, Intel Open Source Technology Centre ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [Intel-gfx] [RFC] xf86-video-intel: enable hw-generated binding tables
On Thursday, April 24, 2014 07:06:34 AM Chris Wilson wrote: On Thu, Apr 24, 2014 at 09:08:14AM +0300, Abdiel Janulgue wrote: Anyway I haven't tried the work-around where we explictly only disable the BT and RS on the other user-space clients (xorg driver in this case) when Mesa is using RS instead of forcing the reset of the clients to use RS format. I'll try that first and let you know if it works. I hate to break the bad news. Tried this just now - still get hangs :( So I guess, all userspace clients* does need to use RS-format if we use this feature. GPGPU workloads seems to be special use-case where the RS hwbinding table format can be disabled. Otherwise, I guess we are stuck with this inflexibility. [1] On the other hand, it doesn't seem all that bad though. The RS hw-binding table format are only needed for clients that submit vertex and pixel shader commands. I've identified currently just UXA and SNA that seem to use this besides Mesa. OpenCL is not affected. If it does, it might be more efficient to do that in the kernel? It has to be done in the kernel in order for interoperability with third party clients. -Chris ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [Intel-gfx] [RFC] xf86-video-intel: enable hw-generated binding tables
On Thu, Apr 24, 2014 at 11:25:15AM +0300, Abdiel Janulgue wrote: On Thursday, April 24, 2014 07:06:34 AM Chris Wilson wrote: On Thu, Apr 24, 2014 at 09:08:14AM +0300, Abdiel Janulgue wrote: Anyway I haven't tried the work-around where we explictly only disable the BT and RS on the other user-space clients (xorg driver in this case) when Mesa is using RS instead of forcing the reset of the clients to use RS format. I'll try that first and let you know if it works. I hate to break the bad news. Tried this just now - still get hangs :( So I guess, all userspace clients* does need to use RS-format if we use this feature. GPGPU workloads seems to be special use-case where the RS hwbinding table format can be disabled. Otherwise, I guess we are stuck with this inflexibility. [1] On the other hand, it doesn't seem all that bad though. The RS hw-binding table format are only needed for clients that submit vertex and pixel shader commands. I've identified currently just UXA and SNA that seem to use this besides Mesa. OpenCL is not affected. libva? rendercopy in igt? This kind of abi break is really ugly :( -Daniel -- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [Intel-gfx] [RFC] xf86-video-intel: enable hw-generated binding tables
On Thu, Apr 24, 2014 at 11:25:15AM +0300, Abdiel Janulgue wrote: On Thursday, April 24, 2014 07:06:34 AM Chris Wilson wrote: On Thu, Apr 24, 2014 at 09:08:14AM +0300, Abdiel Janulgue wrote: Anyway I haven't tried the work-around where we explictly only disable the BT and RS on the other user-space clients (xorg driver in this case) when Mesa is using RS instead of forcing the reset of the clients to use RS format. I'll try that first and let you know if it works. I hate to break the bad news. Tried this just now - still get hangs :( So I guess, all userspace clients* does need to use RS-format if we use this feature. GPGPU workloads seems to be special use-case where the RS hwbinding table format can be disabled. Otherwise, I guess we are stuck with this inflexibility. The spec also says this: When switching between HW and SW binding table generation, SW must issue a state cache invalidate. So it does look like they were expecting people to switch between the two modes. Do you have any igt test for RS? Maybe add an option into rendercopy to make use of RS? Then we could write some tests that try to hit this problem w/o requiring Mesa. [1] On the other hand, it doesn't seem all that bad though. The RS hw-binding table format are only needed for clients that submit vertex and pixel shader commands. I've identified currently just UXA and SNA that seem to use this besides Mesa. OpenCL is not affected. If it does, it might be more efficient to do that in the kernel? It has to be done in the kernel in order for interoperability with third party clients. -Chris -- Ville Syrjälä Intel OTC ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [Intel-gfx] [RFC] xf86-video-intel: enable hw-generated binding tables
On Thursday, April 24, 2014 01:17:14 PM Ville Syrjälä wrote: On Thu, Apr 24, 2014 at 11:25:15AM +0300, Abdiel Janulgue wrote: On Thursday, April 24, 2014 07:06:34 AM Chris Wilson wrote: On Thu, Apr 24, 2014 at 09:08:14AM +0300, Abdiel Janulgue wrote: Anyway I haven't tried the work-around where we explictly only disable the BT and RS on the other user-space clients (xorg driver in this case) when Mesa is using RS instead of forcing the reset of the clients to use RS format. I'll try that first and let you know if it works. I hate to break the bad news. Tried this just now - still get hangs :( So I guess, all userspace clients* does need to use RS-format if we use this feature. GPGPU workloads seems to be special use-case where the RS hwbinding table format can be disabled. Otherwise, I guess we are stuck with this inflexibility. The spec also says this: When switching between HW and SW binding table generation, SW must issue a state cache invalidate. So it does look like they were expecting people to switch between the two modes. Yep, the previous work-around did include a state cache invalidate. Do you have any igt test for RS? Maybe add an option into rendercopy to make use of RS? Then we could write some tests that try to hit this problem w/o requiring Mesa. Seems to be a good idea. I'll try to reproduce the problem with igt [1] On the other hand, it doesn't seem all that bad though. The RS hw-binding table format are only needed for clients that submit vertex and pixel shader commands. I've identified currently just UXA and SNA that seem to use this besides Mesa. OpenCL is not affected. If it does, it might be more efficient to do that in the kernel? It has to be done in the kernel in order for interoperability with third party clients. -Chris ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [Intel-gfx] [RFC] xf86-video-intel: enable hw-generated binding tables
On Tuesday, April 22, 2014 07:20:58 PM Daniel Vetter wrote: On Tue, Apr 22, 2014 at 04:23:12PM +0100, Chris Wilson wrote: On Tue, Apr 22, 2014 at 06:16:34PM +0300, Abdiel Janulgue wrote: This patch series enables resource streamer for xf86-video-intel UXA. Fixes i965 Mesa driver that makes use of resource streamer optimization to survive a full piglit run without bricking the machine. Previously, I get random hungs when doing a full piglit round when enabling RS. After months of head-scratching, I noticed I didn't quite pay attention to this small detail in bspec: The binding table generator feature has a simple all or nothing model. If HW generated binding tables are enabled, the driver must enable the pool and use 3D_HW_BINDING_TABLE_POINTER_* commands. I realized that our xf86-video-intel driver is piping out 3D commands as well that used the older manual-generated binding table format that caused a conflict when Mesa is using the hw-generated binding table format. This has to be per-context right? Otherwise how do you intend to coordinate multiple clients submitting to the kernel using incompatible command sets? Even in the restricted X/DRI sense, how do you intend to negotiate which to use? Hm, I've thought that the MI_BB_START should synchronize with the asynchronous RS parsing/processing? Is there no way to disable RS again for the next batch in a different context? I've already tried disabling RS at the end of every batch so that next batch in different context can continue to use older non-RS format. That does not work either and still causes hangs. What I've seen so far, it seems GPU does not like switching the format of commands from RS-format to non-RS format. It's either one way or the other. If switched on, it affects subsequent contexes henceforth expecting RS-format commands until the GPU gets reset. That's probably the note in bspec: the binding table generator feature has a simple all or nothing model. If there's no way to solve this with contexts or some manual reset trick using LRIs then we're pretty decently screwed up. Worst case we need to stop the gpu and reset it to keep backwards compat :( ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [Intel-gfx] [RFC] xf86-video-intel: enable hw-generated binding tables
On Wed, Apr 23, 2014 at 1:21 PM, Abdiel Janulgue abdiel.janul...@linux.intel.com wrote: I've already tried disabling RS at the end of every batch so that next batch in different context can continue to use older non-RS format. That does not work either and still causes hangs. What I've seen so far, it seems GPU does not like switching the format of commands from RS-format to non-RS format. It's either one way or the other. If switched on, it affects subsequent contexes henceforth expecting RS-format commands until the GPU gets reset. That's probably the note in bspec: the binding table generator feature has a simple all or nothing model. Oh hooray, that's just awesome :( So we indeed need to stop the gpu and reset it if there's a non-RS render batch after any RS render batch. Which also means that we need to enable this for _all_ userspace to avoid completely disastrous performance. So uxa, sna, libva, maybe opencl ... I guess before we engage in this endeavor we need to track this down with the hardware people. -Daniel -- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [Intel-gfx] [RFC] xf86-video-intel: enable hw-generated binding tables
On Wed, Apr 23, 2014 at 06:52:09PM +0200, Daniel Vetter wrote: On Wed, Apr 23, 2014 at 1:21 PM, Abdiel Janulgue abdiel.janul...@linux.intel.com wrote: I've already tried disabling RS at the end of every batch so that next batch in different context can continue to use older non-RS format. That does not work either and still causes hangs. What I've seen so far, it seems GPU does not like switching the format of commands from RS-format to non-RS format. It's either one way or the other. If switched on, it affects subsequent contexes henceforth expecting RS-format commands until the GPU gets reset. That's probably the note in bspec: the binding table generator feature has a simple all or nothing model. Oh hooray, that's just awesome :( So we indeed need to stop the gpu and reset it if there's a non-RS render batch after any RS render batch. I'm not sure I buy it. There's an example in the spec showing how to disable the RS around eg. GPGPU workloads and re-enable it for 3D workloads even within one batch. I guess it's possible that the GPGPU pipeline mode is somehow special here, but since the RS state is supposed to be saved to the context I'm thinking you should be able to disable it whenever you want. What's really confusing with that example is the fact that it first re-enables the binding tables and then the RS, but then there's a note after that saying you have to those steps in the opposite order. Anyhoo, I'm wondering if the problems are simply due to disabling RS but leaving the binding tables enabled? So one idea might be that when we switch from executing an RS batch to a non-RS batch, we should disable the RS and binding tables manually. We would then have to demand that any user batch which needs the binding tables enabled must enable them, even if if we just executed a batch that already used the binding tables. And when we need to disable the RS and binding tables we could either have the kernel do that autmagically when it notices a RS-non-RS transition, or we could also demand that all user batches must disable the binding tables at the end. But maybe demanding that from every batch would incur some extra overhead? In any case it should be fairly easy to try that and see if it cures the hangs. Or are you already doing things that way? -- Ville Syrjälä Intel OTC ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [Intel-gfx] [RFC] xf86-video-intel: enable hw-generated binding tables
On Tue, Apr 22, 2014 at 06:16:34PM +0300, Abdiel Janulgue wrote: This patch series enables resource streamer for xf86-video-intel UXA. Fixes i965 Mesa driver that makes use of resource streamer optimization to survive a full piglit run without bricking the machine. Previously, I get random hungs when doing a full piglit round when enabling RS. After months of head-scratching, I noticed I didn't quite pay attention to this small detail in bspec: The binding table generator feature has a simple all or nothing model. If HW generated binding tables are enabled, the driver must enable the pool and use 3D_HW_BINDING_TABLE_POINTER_* commands. I realized that our xf86-video-intel driver is piping out 3D commands as well that used the older manual-generated binding table format that caused a conflict when Mesa is using the hw-generated binding table format. This has to be per-context right? Otherwise how do you intend to coordinate multiple clients submitting to the kernel using incompatible command sets? Even in the restricted X/DRI sense, how do you intend to negotiate which to use? -Chris -- Chris Wilson, Intel Open Source Technology Centre ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [Intel-gfx] [RFC] xf86-video-intel: enable hw-generated binding tables
On Tue, Apr 22, 2014 at 04:23:12PM +0100, Chris Wilson wrote: On Tue, Apr 22, 2014 at 06:16:34PM +0300, Abdiel Janulgue wrote: This patch series enables resource streamer for xf86-video-intel UXA. Fixes i965 Mesa driver that makes use of resource streamer optimization to survive a full piglit run without bricking the machine. Previously, I get random hungs when doing a full piglit round when enabling RS. After months of head-scratching, I noticed I didn't quite pay attention to this small detail in bspec: The binding table generator feature has a simple all or nothing model. If HW generated binding tables are enabled, the driver must enable the pool and use 3D_HW_BINDING_TABLE_POINTER_* commands. I realized that our xf86-video-intel driver is piping out 3D commands as well that used the older manual-generated binding table format that caused a conflict when Mesa is using the hw-generated binding table format. This has to be per-context right? Otherwise how do you intend to coordinate multiple clients submitting to the kernel using incompatible command sets? Even in the restricted X/DRI sense, how do you intend to negotiate which to use? Hm, I've thought that the MI_BB_START should synchronize with the asynchronous RS parsing/processing? Is there no way to disable RS again for the next batch in a different context? If there's no way to solve this with contexts or some manual reset trick using LRIs then we're pretty decently screwed up. Worst case we need to stop the gpu and reset it to keep backwards compat :( -Daniel -- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx
