[PATCH 00/48] Etnaviv changes RFCv1->RFCv2
Hi Jon, 2015-10-20 11:09 GMT+02:00 Jon Nettleton : > On Tue, Oct 20, 2015 at 11:00 AM, Lucas Stach > wrote: >> Am Dienstag, den 20.10.2015, 09:20 +0200 schrieb Christian Gmeiner: >>> Hi Lucas, >>> >>> 2015-10-13 10:25 GMT+02:00 Lucas Stach : >>> > Am Mittwoch, den 30.09.2015, 09:53 +0200 schrieb Christian Gmeiner: >>> >> Hi Lucas, >>> >> >>> >> 2015-09-28 12:39 GMT+02:00 Lucas Stach : >>> >> > Hi Christian, >>> >> > >>> >> > Am Montag, den 28.09.2015, 11:46 +0200 schrieb Christian Gmeiner: >>> >> >> Hi Lucas. >>> >> >> >>> >> >> I think I have run into a cache flush / cache coherency issue. I will >>> >> >> try to reproduce this issue with a small example and will >>> >> >> keep you updated. >>> >> > >>> >> > What are the symptoms of the issue you are hitting? Maybe I can >>> >> > reproduce or see if I have an idea right away. >>> >> > >>> >> >>> >> With the help of the etnaviv_2d_test in my libdrm repo on github I was >>> >> able >>> >> to test different bo flags. >>> >> >>> >> ETNA_BO_UNCACHED and ETNA_BO_CACHED are working as expected. >>> >> The rendering result looks as expected. If I try ETNA_BO_WC the rendering >>> >> result looks different for every run. >>> >> >>> >> debian at cubox:~/libdrm$ tests/etnaviv/etnaviv_2d_test /dev/dri/card1 >>> >> Version: 1.0.0 >>> >> Name: etnaviv >>> >> Date: 20150910 >>> >> Description: etnaviv DRM >>> >> bo cpu prep: 0 >>> >> debian at cubox:~/libdrm$ md5sum /tmp/etna.bmp >>> >> 052880d433e1bf495e268206addd4087 /tmp/etna.bmp >>> >> debian at cubox:~/libdrm$ tests/etnaviv/etnaviv_2d_test /dev/dri/card1 >>> >> Version: 1.0.0 >>> >> Name: etnaviv >>> >> Date: 20150910 >>> >> Description: etnaviv DRM >>> >> bo cpu prep: 0 >>> >> debian at cubox:~/libdrm$ md5sum /tmp/etna.bmp >>> >> f1a02a52d81c0b79b098877e6b7d9303 /tmp/etna.bmp >>> >> debian at cubox:~/libdrm$ tests/etnaviv/etnaviv_2d_test /dev/dri/card1 >>> >> Version: 1.0.0 >>> >> Name: etnaviv >>> >> Date: 20150910 >>> >> Description: etnaviv DRM >>> >> bo cpu prep: 0 >>> >> debian at cubox:~/libdrm$ md5sum /tmp/etna.bmp >>> >> de5a428eb1f6567849ef40a944a995b8 /tmp/etna.bmp >>> >> >>> >> etna_cmd_stream_finish() waits until the submitted command stream was >>> >> processed by the GPU. I tried to use etna_bo_cpu_prep(..) but I that did >>> >> not >>> >> help. >>> >> >>> >> I am doing something wrong? Should this work in theory? >>> >> >>> > For the record: this is caused by the "shared attribute override enable" >>> > bit in the AUX_CTRL register of the PL310 cache controller not being >>> > set, which makes it non-compliant with the ARMv7 ARM specified behavior >>> > of conflicting memory aliases. >>> > The etnaviv kernel driver makes sure to clean the cachable alias before >>> > handing out the pages, but without this bit set the L2 controller will >>> > turn the userspace bufferable reads into "cachable, no allocate" which >>> > will lead to userspace hitting stale, non-evicted cache lines. >>> > >>> > This can be worked around by adding a "arm,shared-override" property to >>> > the L2 cache DT node. This will only work if the kernel is booted in >>> > secure state and will fault on a NS kernel. So this should be considered >>> > a hack and the bootloader/firmware should make sure to set this bit. >>> > >>> >>> Is the kernel able to detect this faulty environment? It would be great >>> if we can warn the user about this issue and 'convert' all ETNA_BO_WC >>> request to ETNA_BO_CACHED or ETNA_BO_UNCACHED. >>> >>> Else there might be some bug reports in the future where something is >>> broken due to a bad environment and these kind of bugs are hard to >>> sort out. >>> >> I don't think we should work around a platform issue in individual >> drivers. I mean etnaviv makes the issue really visible, but not having >> this bit set in the PL310 controller makes the whole platform >> non-conforming to what is specified in the ARMv7 ARM, so the platform >> may exhibit all kinds of subtle breakages anyway. >> >> We may check for this bit in the PL310 initialization and warn the user >> that a firmware update might be needed to fix this. This should be >> enough to sort out bug reports caused by this specific issue. >> > > I agree, although we can also point them to the device-tree option to > workaround the problem temporarily. > > Christian, fyi I have fixed this in u-boot for all the SolidRun platforms. Great - will update all my devices. greets -- Christian Gmeiner, MSc https://soundcloud.com/christian-gmeiner
[PATCH 00/48] Etnaviv changes RFCv1->RFCv2
Hi Lucas, 2015-10-20 11:00 GMT+02:00 Lucas Stach : > Am Dienstag, den 20.10.2015, 09:20 +0200 schrieb Christian Gmeiner: >> Hi Lucas, >> >> 2015-10-13 10:25 GMT+02:00 Lucas Stach : >> > Am Mittwoch, den 30.09.2015, 09:53 +0200 schrieb Christian Gmeiner: >> >> Hi Lucas, >> >> >> >> 2015-09-28 12:39 GMT+02:00 Lucas Stach : >> >> > Hi Christian, >> >> > >> >> > Am Montag, den 28.09.2015, 11:46 +0200 schrieb Christian Gmeiner: >> >> >> Hi Lucas. >> >> >> >> >> >> I think I have run into a cache flush / cache coherency issue. I will >> >> >> try to reproduce this issue with a small example and will >> >> >> keep you updated. >> >> > >> >> > What are the symptoms of the issue you are hitting? Maybe I can >> >> > reproduce or see if I have an idea right away. >> >> > >> >> >> >> With the help of the etnaviv_2d_test in my libdrm repo on github I was >> >> able >> >> to test different bo flags. >> >> >> >> ETNA_BO_UNCACHED and ETNA_BO_CACHED are working as expected. >> >> The rendering result looks as expected. If I try ETNA_BO_WC the rendering >> >> result looks different for every run. >> >> >> >> debian at cubox:~/libdrm$ tests/etnaviv/etnaviv_2d_test /dev/dri/card1 >> >> Version: 1.0.0 >> >> Name: etnaviv >> >> Date: 20150910 >> >> Description: etnaviv DRM >> >> bo cpu prep: 0 >> >> debian at cubox:~/libdrm$ md5sum /tmp/etna.bmp >> >> 052880d433e1bf495e268206addd4087 /tmp/etna.bmp >> >> debian at cubox:~/libdrm$ tests/etnaviv/etnaviv_2d_test /dev/dri/card1 >> >> Version: 1.0.0 >> >> Name: etnaviv >> >> Date: 20150910 >> >> Description: etnaviv DRM >> >> bo cpu prep: 0 >> >> debian at cubox:~/libdrm$ md5sum /tmp/etna.bmp >> >> f1a02a52d81c0b79b098877e6b7d9303 /tmp/etna.bmp >> >> debian at cubox:~/libdrm$ tests/etnaviv/etnaviv_2d_test /dev/dri/card1 >> >> Version: 1.0.0 >> >> Name: etnaviv >> >> Date: 20150910 >> >> Description: etnaviv DRM >> >> bo cpu prep: 0 >> >> debian at cubox:~/libdrm$ md5sum /tmp/etna.bmp >> >> de5a428eb1f6567849ef40a944a995b8 /tmp/etna.bmp >> >> >> >> etna_cmd_stream_finish() waits until the submitted command stream was >> >> processed by the GPU. I tried to use etna_bo_cpu_prep(..) but I that did >> >> not >> >> help. >> >> >> >> I am doing something wrong? Should this work in theory? >> >> >> > For the record: this is caused by the "shared attribute override enable" >> > bit in the AUX_CTRL register of the PL310 cache controller not being >> > set, which makes it non-compliant with the ARMv7 ARM specified behavior >> > of conflicting memory aliases. >> > The etnaviv kernel driver makes sure to clean the cachable alias before >> > handing out the pages, but without this bit set the L2 controller will >> > turn the userspace bufferable reads into "cachable, no allocate" which >> > will lead to userspace hitting stale, non-evicted cache lines. >> > >> > This can be worked around by adding a "arm,shared-override" property to >> > the L2 cache DT node. This will only work if the kernel is booted in >> > secure state and will fault on a NS kernel. So this should be considered >> > a hack and the bootloader/firmware should make sure to set this bit. >> > >> >> Is the kernel able to detect this faulty environment? It would be great >> if we can warn the user about this issue and 'convert' all ETNA_BO_WC >> request to ETNA_BO_CACHED or ETNA_BO_UNCACHED. >> >> Else there might be some bug reports in the future where something is >> broken due to a bad environment and these kind of bugs are hard to >> sort out. >> > I don't think we should work around a platform issue in individual > drivers. I mean etnaviv makes the issue really visible, but not having > this bit set in the PL310 controller makes the whole platform > non-conforming to what is specified in the ARMv7 ARM, so the platform > may exhibit all kinds of subtle breakages anyway. > I am fine with that. > We may check for this bit in the PL310 initialization and warn the user > that a firmware update might be needed to fix this. This should be > enough to sort out bug reports caused by this specific issue. > That is all I want. If you have a patch for that put me on CC and I can test/review it. greets -- Christian Gmeiner, MSc https://soundcloud.com/christian-gmeiner
[PATCH 00/48] Etnaviv changes RFCv1->RFCv2
On Tue, Oct 20, 2015 at 11:00 AM, Lucas Stach wrote: > Am Dienstag, den 20.10.2015, 09:20 +0200 schrieb Christian Gmeiner: >> Hi Lucas, >> >> 2015-10-13 10:25 GMT+02:00 Lucas Stach : >> > Am Mittwoch, den 30.09.2015, 09:53 +0200 schrieb Christian Gmeiner: >> >> Hi Lucas, >> >> >> >> 2015-09-28 12:39 GMT+02:00 Lucas Stach : >> >> > Hi Christian, >> >> > >> >> > Am Montag, den 28.09.2015, 11:46 +0200 schrieb Christian Gmeiner: >> >> >> Hi Lucas. >> >> >> >> >> >> I think I have run into a cache flush / cache coherency issue. I will >> >> >> try to reproduce this issue with a small example and will >> >> >> keep you updated. >> >> > >> >> > What are the symptoms of the issue you are hitting? Maybe I can >> >> > reproduce or see if I have an idea right away. >> >> > >> >> >> >> With the help of the etnaviv_2d_test in my libdrm repo on github I was >> >> able >> >> to test different bo flags. >> >> >> >> ETNA_BO_UNCACHED and ETNA_BO_CACHED are working as expected. >> >> The rendering result looks as expected. If I try ETNA_BO_WC the rendering >> >> result looks different for every run. >> >> >> >> debian at cubox:~/libdrm$ tests/etnaviv/etnaviv_2d_test /dev/dri/card1 >> >> Version: 1.0.0 >> >> Name: etnaviv >> >> Date: 20150910 >> >> Description: etnaviv DRM >> >> bo cpu prep: 0 >> >> debian at cubox:~/libdrm$ md5sum /tmp/etna.bmp >> >> 052880d433e1bf495e268206addd4087 /tmp/etna.bmp >> >> debian at cubox:~/libdrm$ tests/etnaviv/etnaviv_2d_test /dev/dri/card1 >> >> Version: 1.0.0 >> >> Name: etnaviv >> >> Date: 20150910 >> >> Description: etnaviv DRM >> >> bo cpu prep: 0 >> >> debian at cubox:~/libdrm$ md5sum /tmp/etna.bmp >> >> f1a02a52d81c0b79b098877e6b7d9303 /tmp/etna.bmp >> >> debian at cubox:~/libdrm$ tests/etnaviv/etnaviv_2d_test /dev/dri/card1 >> >> Version: 1.0.0 >> >> Name: etnaviv >> >> Date: 20150910 >> >> Description: etnaviv DRM >> >> bo cpu prep: 0 >> >> debian at cubox:~/libdrm$ md5sum /tmp/etna.bmp >> >> de5a428eb1f6567849ef40a944a995b8 /tmp/etna.bmp >> >> >> >> etna_cmd_stream_finish() waits until the submitted command stream was >> >> processed by the GPU. I tried to use etna_bo_cpu_prep(..) but I that did >> >> not >> >> help. >> >> >> >> I am doing something wrong? Should this work in theory? >> >> >> > For the record: this is caused by the "shared attribute override enable" >> > bit in the AUX_CTRL register of the PL310 cache controller not being >> > set, which makes it non-compliant with the ARMv7 ARM specified behavior >> > of conflicting memory aliases. >> > The etnaviv kernel driver makes sure to clean the cachable alias before >> > handing out the pages, but without this bit set the L2 controller will >> > turn the userspace bufferable reads into "cachable, no allocate" which >> > will lead to userspace hitting stale, non-evicted cache lines. >> > >> > This can be worked around by adding a "arm,shared-override" property to >> > the L2 cache DT node. This will only work if the kernel is booted in >> > secure state and will fault on a NS kernel. So this should be considered >> > a hack and the bootloader/firmware should make sure to set this bit. >> > >> >> Is the kernel able to detect this faulty environment? It would be great >> if we can warn the user about this issue and 'convert' all ETNA_BO_WC >> request to ETNA_BO_CACHED or ETNA_BO_UNCACHED. >> >> Else there might be some bug reports in the future where something is >> broken due to a bad environment and these kind of bugs are hard to >> sort out. >> > I don't think we should work around a platform issue in individual > drivers. I mean etnaviv makes the issue really visible, but not having > this bit set in the PL310 controller makes the whole platform > non-conforming to what is specified in the ARMv7 ARM, so the platform > may exhibit all kinds of subtle breakages anyway. > > We may check for this bit in the PL310 initialization and warn the user > that a firmware update might be needed to fix this. This should be > enough to sort out bug reports caused by this specific issue. > I agree, although we can also point them to the device-tree option to workaround the problem temporarily. Christian, fyi I have fixed this in u-boot for all the SolidRun platforms.
[PATCH 00/48] Etnaviv changes RFCv1->RFCv2
Am Dienstag, den 20.10.2015, 09:20 +0200 schrieb Christian Gmeiner: > Hi Lucas, > > 2015-10-13 10:25 GMT+02:00 Lucas Stach : > > Am Mittwoch, den 30.09.2015, 09:53 +0200 schrieb Christian Gmeiner: > >> Hi Lucas, > >> > >> 2015-09-28 12:39 GMT+02:00 Lucas Stach : > >> > Hi Christian, > >> > > >> > Am Montag, den 28.09.2015, 11:46 +0200 schrieb Christian Gmeiner: > >> >> Hi Lucas. > >> >> > >> >> I think I have run into a cache flush / cache coherency issue. I will > >> >> try to reproduce this issue with a small example and will > >> >> keep you updated. > >> > > >> > What are the symptoms of the issue you are hitting? Maybe I can > >> > reproduce or see if I have an idea right away. > >> > > >> > >> With the help of the etnaviv_2d_test in my libdrm repo on github I was able > >> to test different bo flags. > >> > >> ETNA_BO_UNCACHED and ETNA_BO_CACHED are working as expected. > >> The rendering result looks as expected. If I try ETNA_BO_WC the rendering > >> result looks different for every run. > >> > >> debian at cubox:~/libdrm$ tests/etnaviv/etnaviv_2d_test /dev/dri/card1 > >> Version: 1.0.0 > >> Name: etnaviv > >> Date: 20150910 > >> Description: etnaviv DRM > >> bo cpu prep: 0 > >> debian at cubox:~/libdrm$ md5sum /tmp/etna.bmp > >> 052880d433e1bf495e268206addd4087 /tmp/etna.bmp > >> debian at cubox:~/libdrm$ tests/etnaviv/etnaviv_2d_test /dev/dri/card1 > >> Version: 1.0.0 > >> Name: etnaviv > >> Date: 20150910 > >> Description: etnaviv DRM > >> bo cpu prep: 0 > >> debian at cubox:~/libdrm$ md5sum /tmp/etna.bmp > >> f1a02a52d81c0b79b098877e6b7d9303 /tmp/etna.bmp > >> debian at cubox:~/libdrm$ tests/etnaviv/etnaviv_2d_test /dev/dri/card1 > >> Version: 1.0.0 > >> Name: etnaviv > >> Date: 20150910 > >> Description: etnaviv DRM > >> bo cpu prep: 0 > >> debian at cubox:~/libdrm$ md5sum /tmp/etna.bmp > >> de5a428eb1f6567849ef40a944a995b8 /tmp/etna.bmp > >> > >> etna_cmd_stream_finish() waits until the submitted command stream was > >> processed by the GPU. I tried to use etna_bo_cpu_prep(..) but I that did > >> not > >> help. > >> > >> I am doing something wrong? Should this work in theory? > >> > > For the record: this is caused by the "shared attribute override enable" > > bit in the AUX_CTRL register of the PL310 cache controller not being > > set, which makes it non-compliant with the ARMv7 ARM specified behavior > > of conflicting memory aliases. > > The etnaviv kernel driver makes sure to clean the cachable alias before > > handing out the pages, but without this bit set the L2 controller will > > turn the userspace bufferable reads into "cachable, no allocate" which > > will lead to userspace hitting stale, non-evicted cache lines. > > > > This can be worked around by adding a "arm,shared-override" property to > > the L2 cache DT node. This will only work if the kernel is booted in > > secure state and will fault on a NS kernel. So this should be considered > > a hack and the bootloader/firmware should make sure to set this bit. > > > > Is the kernel able to detect this faulty environment? It would be great > if we can warn the user about this issue and 'convert' all ETNA_BO_WC > request to ETNA_BO_CACHED or ETNA_BO_UNCACHED. > > Else there might be some bug reports in the future where something is > broken due to a bad environment and these kind of bugs are hard to > sort out. > I don't think we should work around a platform issue in individual drivers. I mean etnaviv makes the issue really visible, but not having this bit set in the PL310 controller makes the whole platform non-conforming to what is specified in the ARMv7 ARM, so the platform may exhibit all kinds of subtle breakages anyway. We may check for this bit in the PL310 initialization and warn the user that a firmware update might be needed to fix this. This should be enough to sort out bug reports caused by this specific issue. Regards, Lucas -- Pengutronix e.K. | Lucas Stach | Industrial Linux Solutions | http://www.pengutronix.de/ |
[PATCH 00/48] Etnaviv changes RFCv1->RFCv2
Hi Lucas, 2015-10-13 10:25 GMT+02:00 Lucas Stach : > Am Mittwoch, den 30.09.2015, 09:53 +0200 schrieb Christian Gmeiner: >> Hi Lucas, >> >> 2015-09-28 12:39 GMT+02:00 Lucas Stach : >> > Hi Christian, >> > >> > Am Montag, den 28.09.2015, 11:46 +0200 schrieb Christian Gmeiner: >> >> Hi Lucas. >> >> >> >> I think I have run into a cache flush / cache coherency issue. I will >> >> try to reproduce this issue with a small example and will >> >> keep you updated. >> > >> > What are the symptoms of the issue you are hitting? Maybe I can >> > reproduce or see if I have an idea right away. >> > >> >> With the help of the etnaviv_2d_test in my libdrm repo on github I was able >> to test different bo flags. >> >> ETNA_BO_UNCACHED and ETNA_BO_CACHED are working as expected. >> The rendering result looks as expected. If I try ETNA_BO_WC the rendering >> result looks different for every run. >> >> debian at cubox:~/libdrm$ tests/etnaviv/etnaviv_2d_test /dev/dri/card1 >> Version: 1.0.0 >> Name: etnaviv >> Date: 20150910 >> Description: etnaviv DRM >> bo cpu prep: 0 >> debian at cubox:~/libdrm$ md5sum /tmp/etna.bmp >> 052880d433e1bf495e268206addd4087 /tmp/etna.bmp >> debian at cubox:~/libdrm$ tests/etnaviv/etnaviv_2d_test /dev/dri/card1 >> Version: 1.0.0 >> Name: etnaviv >> Date: 20150910 >> Description: etnaviv DRM >> bo cpu prep: 0 >> debian at cubox:~/libdrm$ md5sum /tmp/etna.bmp >> f1a02a52d81c0b79b098877e6b7d9303 /tmp/etna.bmp >> debian at cubox:~/libdrm$ tests/etnaviv/etnaviv_2d_test /dev/dri/card1 >> Version: 1.0.0 >> Name: etnaviv >> Date: 20150910 >> Description: etnaviv DRM >> bo cpu prep: 0 >> debian at cubox:~/libdrm$ md5sum /tmp/etna.bmp >> de5a428eb1f6567849ef40a944a995b8 /tmp/etna.bmp >> >> etna_cmd_stream_finish() waits until the submitted command stream was >> processed by the GPU. I tried to use etna_bo_cpu_prep(..) but I that did not >> help. >> >> I am doing something wrong? Should this work in theory? >> > For the record: this is caused by the "shared attribute override enable" > bit in the AUX_CTRL register of the PL310 cache controller not being > set, which makes it non-compliant with the ARMv7 ARM specified behavior > of conflicting memory aliases. > The etnaviv kernel driver makes sure to clean the cachable alias before > handing out the pages, but without this bit set the L2 controller will > turn the userspace bufferable reads into "cachable, no allocate" which > will lead to userspace hitting stale, non-evicted cache lines. > > This can be worked around by adding a "arm,shared-override" property to > the L2 cache DT node. This will only work if the kernel is booted in > secure state and will fault on a NS kernel. So this should be considered > a hack and the bootloader/firmware should make sure to set this bit. > Is the kernel able to detect this faulty environment? It would be great if we can warn the user about this issue and 'convert' all ETNA_BO_WC request to ETNA_BO_CACHED or ETNA_BO_UNCACHED. Else there might be some bug reports in the future where something is broken due to a bad environment and these kind of bugs are hard to sort out. greets -- Christian Gmeiner, MSc https://soundcloud.com/christian-gmeiner
[PATCH 00/48] Etnaviv changes RFCv1->RFCv2
Hi Jon, On Tue, Oct 20, 2015 at 7:09 AM, Jon Nettleton wrote: > I agree, although we can also point them to the device-tree option to > workaround the problem temporarily. > > Christian, fyi I have fixed this in u-boot for all the SolidRun platforms. This is fixed in mainline U-boot for mx6: http://git.denx.de/?p=u-boot.git;a=commitdiff;h=b4ed9f86df441d7ac304f70cc711c257e7c8ebf1;hp=9927d60f4aadcecbd3143400d01ad4500438ea4f
[PATCH 00/48] Etnaviv changes RFCv1->RFCv2
Am Mittwoch, den 30.09.2015, 09:53 +0200 schrieb Christian Gmeiner: > Hi Lucas, > > 2015-09-28 12:39 GMT+02:00 Lucas Stach : > > Hi Christian, > > > > Am Montag, den 28.09.2015, 11:46 +0200 schrieb Christian Gmeiner: > >> Hi Lucas. > >> > >> I think I have run into a cache flush / cache coherency issue. I will > >> try to reproduce this issue with a small example and will > >> keep you updated. > > > > What are the symptoms of the issue you are hitting? Maybe I can > > reproduce or see if I have an idea right away. > > > > With the help of the etnaviv_2d_test in my libdrm repo on github I was able > to test different bo flags. > > ETNA_BO_UNCACHED and ETNA_BO_CACHED are working as expected. > The rendering result looks as expected. If I try ETNA_BO_WC the rendering > result looks different for every run. > > debian at cubox:~/libdrm$ tests/etnaviv/etnaviv_2d_test /dev/dri/card1 > Version: 1.0.0 > Name: etnaviv > Date: 20150910 > Description: etnaviv DRM > bo cpu prep: 0 > debian at cubox:~/libdrm$ md5sum /tmp/etna.bmp > 052880d433e1bf495e268206addd4087 /tmp/etna.bmp > debian at cubox:~/libdrm$ tests/etnaviv/etnaviv_2d_test /dev/dri/card1 > Version: 1.0.0 > Name: etnaviv > Date: 20150910 > Description: etnaviv DRM > bo cpu prep: 0 > debian at cubox:~/libdrm$ md5sum /tmp/etna.bmp > f1a02a52d81c0b79b098877e6b7d9303 /tmp/etna.bmp > debian at cubox:~/libdrm$ tests/etnaviv/etnaviv_2d_test /dev/dri/card1 > Version: 1.0.0 > Name: etnaviv > Date: 20150910 > Description: etnaviv DRM > bo cpu prep: 0 > debian at cubox:~/libdrm$ md5sum /tmp/etna.bmp > de5a428eb1f6567849ef40a944a995b8 /tmp/etna.bmp > > etna_cmd_stream_finish() waits until the submitted command stream was > processed by the GPU. I tried to use etna_bo_cpu_prep(..) but I that did not > help. > > I am doing something wrong? Should this work in theory? > For the record: this is caused by the "shared attribute override enable" bit in the AUX_CTRL register of the PL310 cache controller not being set, which makes it non-compliant with the ARMv7 ARM specified behavior of conflicting memory aliases. The etnaviv kernel driver makes sure to clean the cachable alias before handing out the pages, but without this bit set the L2 controller will turn the userspace bufferable reads into "cachable, no allocate" which will lead to userspace hitting stale, non-evicted cache lines. This can be worked around by adding a "arm,shared-override" property to the L2 cache DT node. This will only work if the kernel is booted in secure state and will fault on a NS kernel. So this should be considered a hack and the bootloader/firmware should make sure to set this bit. Regards, Lucas -- Pengutronix e.K. | Lucas Stach | Industrial Linux Solutions | http://www.pengutronix.de/ |
[PATCH 00/48] Etnaviv changes RFCv1->RFCv2
Am Mittwoch, den 30.09.2015, 09:53 +0200 schrieb Christian Gmeiner:
> Hi Lucas,
>
> 2015-09-28 12:39 GMT+02:00 Lucas Stach :
> > Hi Christian,
> >
> > Am Montag, den 28.09.2015, 11:46 +0200 schrieb Christian Gmeiner:
> >> Hi Lucas.
> >>
> >> I think I have run into a cache flush / cache coherency issue. I will
> >> try to reproduce this issue with a small example and will
> >> keep you updated.
> >
> > What are the symptoms of the issue you are hitting? Maybe I can
> > reproduce or see if I have an idea right away.
> >
>
> With the help of the etnaviv_2d_test in my libdrm repo on github I was able
> to test different bo flags.
>
> ETNA_BO_UNCACHED and ETNA_BO_CACHED are working as expected.
> The rendering result looks as expected. If I try ETNA_BO_WC the rendering
> result looks different for every run.
>
I have an idea what's going wrong here. Will keep you updated.
Regards,
Lucas
> debian at cubox:~/libdrm$ tests/etnaviv/etnaviv_2d_test /dev/dri/card1
> Version: 1.0.0
> Name: etnaviv
> Date: 20150910
> Description: etnaviv DRM
> bo cpu prep: 0
> debian at cubox:~/libdrm$ md5sum /tmp/etna.bmp
> 052880d433e1bf495e268206addd4087 /tmp/etna.bmp
> debian at cubox:~/libdrm$ tests/etnaviv/etnaviv_2d_test /dev/dri/card1
> Version: 1.0.0
> Name: etnaviv
> Date: 20150910
> Description: etnaviv DRM
> bo cpu prep: 0
> debian at cubox:~/libdrm$ md5sum /tmp/etna.bmp
> f1a02a52d81c0b79b098877e6b7d9303 /tmp/etna.bmp
> debian at cubox:~/libdrm$ tests/etnaviv/etnaviv_2d_test /dev/dri/card1
> Version: 1.0.0
> Name: etnaviv
> Date: 20150910
> Description: etnaviv DRM
> bo cpu prep: 0
> debian at cubox:~/libdrm$ md5sum /tmp/etna.bmp
> de5a428eb1f6567849ef40a944a995b8 /tmp/etna.bmp
>
> etna_cmd_stream_finish() waits until the submitted command stream was
> processed by the GPU. I tried to use etna_bo_cpu_prep(..) but I that did not
> help.
>
> I am doing something wrong? Should this work in theory?
>
> diff --git a/tests/etnaviv/etnaviv_2d_test.c b/tests/etnaviv/etnaviv_2d_test.c
> index e1ee8a8..037da5b 100644
> --- a/tests/etnaviv/etnaviv_2d_test.c
> +++ b/tests/etnaviv/etnaviv_2d_test.c
> @@ -200,7 +200,7 @@ int main(int argc, char *argv[])
> goto fail;
> }
>
> - bmp = etna_bo_new(dev, bmp_size, ETNA_BO_UNCACHED);
> + bmp = etna_bo_new(dev, bmp_size, ETNA_BO_WC);
> if (!bmp) {
> ret = 5;
> goto fail;
> @@ -218,8 +218,10 @@ int main(int argc, char *argv[])
>
> etna_cmd_stream_finish(stream);
>
> + int state = etna_bo_cpu_prep(bmp, DRM_ETNA_PREP_READ);
> + printf("bo cpu prep: %d\n", state);
> bmp_dump32(etna_bo_map(bmp), width, height, false, "/tmp/etna.bmp");
> -
> + etna_bo_cpu_fini(bmp);
> fail:
> if (stream)
> etna_cmd_stream_del(stream);
>
> Greets
> --
> Christian Gmeiner, MSc
>
> https://soundcloud.com/christian-gmeiner
--
Pengutronix e.K. | Lucas Stach |
Industrial Linux Solutions | http://www.pengutronix.de/ |
[PATCH 00/48] Etnaviv changes RFCv1->RFCv2
Hi Lucas,
2015-09-28 12:39 GMT+02:00 Lucas Stach :
> Hi Christian,
>
> Am Montag, den 28.09.2015, 11:46 +0200 schrieb Christian Gmeiner:
>> Hi Lucas.
>>
>> I think I have run into a cache flush / cache coherency issue. I will
>> try to reproduce this issue with a small example and will
>> keep you updated.
>
> What are the symptoms of the issue you are hitting? Maybe I can
> reproduce or see if I have an idea right away.
>
With the help of the etnaviv_2d_test in my libdrm repo on github I was able
to test different bo flags.
ETNA_BO_UNCACHED and ETNA_BO_CACHED are working as expected.
The rendering result looks as expected. If I try ETNA_BO_WC the rendering
result looks different for every run.
debian at cubox:~/libdrm$ tests/etnaviv/etnaviv_2d_test /dev/dri/card1
Version: 1.0.0
Name: etnaviv
Date: 20150910
Description: etnaviv DRM
bo cpu prep: 0
debian at cubox:~/libdrm$ md5sum /tmp/etna.bmp
052880d433e1bf495e268206addd4087 /tmp/etna.bmp
debian at cubox:~/libdrm$ tests/etnaviv/etnaviv_2d_test /dev/dri/card1
Version: 1.0.0
Name: etnaviv
Date: 20150910
Description: etnaviv DRM
bo cpu prep: 0
debian at cubox:~/libdrm$ md5sum /tmp/etna.bmp
f1a02a52d81c0b79b098877e6b7d9303 /tmp/etna.bmp
debian at cubox:~/libdrm$ tests/etnaviv/etnaviv_2d_test /dev/dri/card1
Version: 1.0.0
Name: etnaviv
Date: 20150910
Description: etnaviv DRM
bo cpu prep: 0
debian at cubox:~/libdrm$ md5sum /tmp/etna.bmp
de5a428eb1f6567849ef40a944a995b8 /tmp/etna.bmp
etna_cmd_stream_finish() waits until the submitted command stream was
processed by the GPU. I tried to use etna_bo_cpu_prep(..) but I that did not
help.
I am doing something wrong? Should this work in theory?
diff --git a/tests/etnaviv/etnaviv_2d_test.c b/tests/etnaviv/etnaviv_2d_test.c
index e1ee8a8..037da5b 100644
--- a/tests/etnaviv/etnaviv_2d_test.c
+++ b/tests/etnaviv/etnaviv_2d_test.c
@@ -200,7 +200,7 @@ int main(int argc, char *argv[])
goto fail;
}
- bmp = etna_bo_new(dev, bmp_size, ETNA_BO_UNCACHED);
+ bmp = etna_bo_new(dev, bmp_size, ETNA_BO_WC);
if (!bmp) {
ret = 5;
goto fail;
@@ -218,8 +218,10 @@ int main(int argc, char *argv[])
etna_cmd_stream_finish(stream);
+ int state = etna_bo_cpu_prep(bmp, DRM_ETNA_PREP_READ);
+ printf("bo cpu prep: %d\n", state);
bmp_dump32(etna_bo_map(bmp), width, height, false, "/tmp/etna.bmp");
-
+ etna_bo_cpu_fini(bmp);
fail:
if (stream)
etna_cmd_stream_del(stream);
Greets
--
Christian Gmeiner, MSc
https://soundcloud.com/christian-gmeiner
[PATCH 00/48] Etnaviv changes RFCv1->RFCv2
Hi Christian, Am Montag, den 28.09.2015, 11:46 +0200 schrieb Christian Gmeiner: > Hi Lucas. > > I think I have run into a cache flush / cache coherency issue. I will > try to reproduce this issue with a small example and will > keep you updated. What are the symptoms of the issue you are hitting? Maybe I can reproduce or see if I have an idea right away. Regards, Lucas -- Pengutronix e.K. | Lucas Stach | Industrial Linux Solutions | http://www.pengutronix.de/ |
[PATCH 00/48] Etnaviv changes RFCv1->RFCv2
Hi Lucas. I think I have run into a cache flush / cache coherency issue. I will try to reproduce this issue with a small example and will keep you updated. Greets -- Christian Gmeiner, MSc https://soundcloud.com/christian-gmeiner 2015-09-25 13:57 GMT+02:00 Lucas Stach : > Hi Russell, Christian and whoever might be interested, > > This series contains the individual changes between RFCv1 and RFCv2 > of the etnaviv driver. This should make it easier to review the patches > for people already familiar with the first round of etnaviv patches. > > Note that I opted to sort the patches a bit different than how they are > laid out in the etnaviv-for-upstream git branch to make logical > connections between patches a bit more clear and to not repeat patches > with only trivial changes from RFCv1 in this series. > > Regards, > Lucas > > Christian Gmeiner (5): > staging: etnaviv: quiten down kernel log output > staging: etnaviv: fix 'ret' may be used uninitialized in this function > staging: etnaviv: fix error: 'etnaviv_gpu_hw_resume' defined but not > used > staging: etnaviv: debugfs: add possibility to dump kernel buffer > staging: etnaviv: change etnaviv_buffer_init() to return prefetch > > Lucas Stach (10): > staging: etnaviv: remove compat MMU code > staging: etnaviv: rename last remaining bits from msm to etnaviv > staging: etnaviv: add proper license header to all files > staging: etnaviv: implement simple hang recovery > staging: etnaviv: map all buffers to the GPU > staging: etnaviv: implement cache maintenance on cpu_(prep|fini) > staging: etnaviv: remove submit type > staging: etnaviv: rewrite submit interface to use copy from user > staging: etnaviv: don't use GEM buffer for internal ring buffer > staging: etnaviv: remove CMDSTREAM GEM allocation from UAPI > > Russell King (33): > staging: etnaviv: avoid holding struct_mutex over dma_alloc_coherent() > staging: etnaviv: restructure iommu handling > staging: etnaviv: clean up public API (part 2) > staging: etnaviv: rename last remaining msm_* symbols > staging: etnaviv: add Dove GPU subsystem compatible > staging: etnaviv: fix missing error cleanups in etnaviv_load() > staging: etnaviv: fix off-by-one for iommu aperture end > staging: etnaviv: avoid lockdep circular dependency warning > staging: etnaviv: fix gpu debugfs show implementation > staging: etnaviv: use vm_insert_page() rather than vm_insert_mixed() > staging: etnaviv: etnaviv_gem_fault: reduce struct_mutex exposure > staging: etnaviv: give etnaviv_gem_mmap_offset() a sane behaviour > staging: etnaviv: allow etnaviv_ioctl_gem_info() locking to be > interruptible > staging: etnaviv: make context a per-GPU thing > staging: etnaviv: switch to per-GPU fence completion implementation > staging: etnaviv: provide etnaviv_queue_work() > staging: etnaviv: use standard kernel types rather than stdint.h types > staging: etnaviv: no need to initialise a list_head > staging: etnaviv: fix oops caused by scanning for free blocks > staging: etnaviv: clean up etnaviv_iommu_unmap_gem() signature > staging: etnaviv: increase page table size to maximum > staging: etnaviv: fix BUG_ON when removing module > staging: etnaviv: provide a helper to load the GPU clock field > staging: etnaviv: rename GPU clock functions > staging: etnaviv: fix runtime resume > staging: etnaviv: drop event ring buffer tracking > staging: etnaviv: improve efficiency of command parser > staging: etnaviv: no point looking up the mapping for cmdstream bos > staging: etnaviv: copy submit command and bos in one go > staging: etnaviv: remove cmd buffer offset validation in > submit_reloc() > staging: etnaviv: move mapping teardown into etnaviv_gem_free_object() > staging: etnaviv: add support for GEM_WAIT ioctl > staging: etnaviv: avoid pinning pages in CMA > > drivers/staging/etnaviv/etnaviv_buffer.c | 123 > drivers/staging/etnaviv/etnaviv_cmd_parser.c | 56 ++-- > drivers/staging/etnaviv/etnaviv_drv.c| 217 --- > drivers/staging/etnaviv/etnaviv_drv.h| 54 ++-- > drivers/staging/etnaviv/etnaviv_gem.c| 370 > drivers/staging/etnaviv/etnaviv_gem.h| 40 +-- > drivers/staging/etnaviv/etnaviv_gem_prime.c | 10 +- > drivers/staging/etnaviv/etnaviv_gem_submit.c | 219 +++ > drivers/staging/etnaviv/etnaviv_gpu.c| 403 > --- > drivers/staging/etnaviv/etnaviv_gpu.h| 107 --- > drivers/staging/etnaviv/etnaviv_iommu.c | 113 > drivers/staging/etnaviv/etnaviv_iommu.h | 2 + > drivers/staging/etnaviv/etnaviv_mmu.c| 75 ++--- > drivers/staging/etnaviv/etnaviv_mmu.h| 15 +- > include/uapi/drm/etnaviv_drm.h | 100 +++ > 15 files changed, 1086 insertions(+), 818 deletions(-) > > -- > 2.5.1 >
[PATCH 00/48] Etnaviv changes RFCv1->RFCv2
Hi Russell, Christian and whoever might be interested, This series contains the individual changes between RFCv1 and RFCv2 of the etnaviv driver. This should make it easier to review the patches for people already familiar with the first round of etnaviv patches. Note that I opted to sort the patches a bit different than how they are laid out in the etnaviv-for-upstream git branch to make logical connections between patches a bit more clear and to not repeat patches with only trivial changes from RFCv1 in this series. Regards, Lucas Christian Gmeiner (5): staging: etnaviv: quiten down kernel log output staging: etnaviv: fix 'ret' may be used uninitialized in this function staging: etnaviv: fix error: 'etnaviv_gpu_hw_resume' defined but not used staging: etnaviv: debugfs: add possibility to dump kernel buffer staging: etnaviv: change etnaviv_buffer_init() to return prefetch Lucas Stach (10): staging: etnaviv: remove compat MMU code staging: etnaviv: rename last remaining bits from msm to etnaviv staging: etnaviv: add proper license header to all files staging: etnaviv: implement simple hang recovery staging: etnaviv: map all buffers to the GPU staging: etnaviv: implement cache maintenance on cpu_(prep|fini) staging: etnaviv: remove submit type staging: etnaviv: rewrite submit interface to use copy from user staging: etnaviv: don't use GEM buffer for internal ring buffer staging: etnaviv: remove CMDSTREAM GEM allocation from UAPI Russell King (33): staging: etnaviv: avoid holding struct_mutex over dma_alloc_coherent() staging: etnaviv: restructure iommu handling staging: etnaviv: clean up public API (part 2) staging: etnaviv: rename last remaining msm_* symbols staging: etnaviv: add Dove GPU subsystem compatible staging: etnaviv: fix missing error cleanups in etnaviv_load() staging: etnaviv: fix off-by-one for iommu aperture end staging: etnaviv: avoid lockdep circular dependency warning staging: etnaviv: fix gpu debugfs show implementation staging: etnaviv: use vm_insert_page() rather than vm_insert_mixed() staging: etnaviv: etnaviv_gem_fault: reduce struct_mutex exposure staging: etnaviv: give etnaviv_gem_mmap_offset() a sane behaviour staging: etnaviv: allow etnaviv_ioctl_gem_info() locking to be interruptible staging: etnaviv: make context a per-GPU thing staging: etnaviv: switch to per-GPU fence completion implementation staging: etnaviv: provide etnaviv_queue_work() staging: etnaviv: use standard kernel types rather than stdint.h types staging: etnaviv: no need to initialise a list_head staging: etnaviv: fix oops caused by scanning for free blocks staging: etnaviv: clean up etnaviv_iommu_unmap_gem() signature staging: etnaviv: increase page table size to maximum staging: etnaviv: fix BUG_ON when removing module staging: etnaviv: provide a helper to load the GPU clock field staging: etnaviv: rename GPU clock functions staging: etnaviv: fix runtime resume staging: etnaviv: drop event ring buffer tracking staging: etnaviv: improve efficiency of command parser staging: etnaviv: no point looking up the mapping for cmdstream bos staging: etnaviv: copy submit command and bos in one go staging: etnaviv: remove cmd buffer offset validation in submit_reloc() staging: etnaviv: move mapping teardown into etnaviv_gem_free_object() staging: etnaviv: add support for GEM_WAIT ioctl staging: etnaviv: avoid pinning pages in CMA drivers/staging/etnaviv/etnaviv_buffer.c | 123 drivers/staging/etnaviv/etnaviv_cmd_parser.c | 56 ++-- drivers/staging/etnaviv/etnaviv_drv.c| 217 --- drivers/staging/etnaviv/etnaviv_drv.h| 54 ++-- drivers/staging/etnaviv/etnaviv_gem.c| 370 drivers/staging/etnaviv/etnaviv_gem.h| 40 +-- drivers/staging/etnaviv/etnaviv_gem_prime.c | 10 +- drivers/staging/etnaviv/etnaviv_gem_submit.c | 219 +++ drivers/staging/etnaviv/etnaviv_gpu.c| 403 --- drivers/staging/etnaviv/etnaviv_gpu.h| 107 --- drivers/staging/etnaviv/etnaviv_iommu.c | 113 drivers/staging/etnaviv/etnaviv_iommu.h | 2 + drivers/staging/etnaviv/etnaviv_mmu.c| 75 ++--- drivers/staging/etnaviv/etnaviv_mmu.h| 15 +- include/uapi/drm/etnaviv_drm.h | 100 +++ 15 files changed, 1086 insertions(+), 818 deletions(-) -- 2.5.1
