[PATCH 1/4] drm/amd/powerplay: update how to use metrics table on Arcturus

2020-06-05 Thread Evan Quan
Retrieve only those interested metrics data instead of the whole metrics table. By this, the memory copy can be dropped. Change-Id: Ice2b2ba4647301119130be0ba65bb587f19d38ae Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/arcturus_ppt.c | 213 -- .../gpu/drm/amd

[PATCH 2/4] drm/amd/powerplay: update how to use metrics table on Navi10

2020-06-05 Thread Evan Quan
Retrieve only those data interested instead of the whole table. Change-Id: I78a3ecce1075e73b96c2ee0b13aacc469bb09021 Signed-off-by: Evan Quan --- .../gpu/drm/amd/powerplay/inc/amdgpu_smu.h| 1 + drivers/gpu/drm/amd/powerplay/navi10_ppt.c| 180 -- 2 files changed, 127

[PATCH 08/16] drm/amd/powerplay: postpone operations not must for hw setup to late_init

2020-06-03 Thread Evan Quan
So that we do not need to perform those unnecessary operations again on resume. Change-Id: I90f8a8d68762b5f88d7477934128a17bf67e3341 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 57 +++--- 1 file changed, 28 insertions(+), 29 deletions(-) diff --git

[PATCH 16/16] drm/amd/powerplay: skip BACO feature on DPMs disablement

2020-06-03 Thread Evan Quan
Instead of disabling and reenabling it later. Change-Id: I90775202178f3b7695f42f39ce240bbfd51a1346 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 72 ++ 1 file changed, 31 insertions(+), 41 deletions(-) diff --git a/drivers/gpu/drm/amd/powerplay

[PATCH 15/16] drm/amd/powerplay: add firmware cleanup on sw_fini

2020-06-03 Thread Evan Quan
To avoid possible memory leak. Change-Id: I4740eac7fc2c6e934ec8f503e5a98057f0902f4a Signed-off-by: Evan Quan Reviewed-by: Alex Deucher --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 2 ++ drivers/gpu/drm/amd/powerplay/arcturus_ppt.c | 1 + drivers/gpu/drm/amd/powerplay/inc/amdgpu_smu.h

[PATCH 03/16] drm/amd/powerplay: implement a common API for dpms disablement

2020-06-03 Thread Evan Quan
So that code can be shared between .hw_fini and .suspend. Change-Id: I4a0eeb7cdecbf5b24fac3d0fe1d8fcb1ca9f0b0a Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 178 + 1 file changed, 77 insertions(+), 101 deletions(-) diff --git a/drivers/gpu/drm

[PATCH 09/16] drm/amd/powerplay: move those operations not needed for resume out

2020-06-03 Thread Evan Quan
Since smu_smc_table_hw_init() is needed for both .hw_init and .resume. By doing this, we can drop unnecessary operations on resume. Change-Id: I2af6277efaa9adba2de69161e20e54c4aa10a411 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 40 +++--- 1 file

[PATCH 12/16] drm/amd/powerplay: better namings

2020-06-03 Thread Evan Quan
And some minor changes as dropping unused parameter and label internal used API as static. Change-Id: I0af0aea029dc4fc7d8e150ab6ec984e9a5f1a74a Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 12 +--- drivers/gpu/drm/amd/powerplay/inc/amdgpu_smu.h | 2

[PATCH 05/16] drm/amd/powerplay: clean up the APIs for bootup clocks

2020-06-03 Thread Evan Quan
Combine and simplify the logics for retrieving bootup clocks. Change-Id: Ifca28c454f3769dece0cc705ba054ff34db0ab60 Signed-off-by: Evan Quan Reviewed-by: Alex Deucher --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c| 4 - drivers/gpu/drm/amd/powerplay/arcturus_ppt.c | 1 - .../gpu/drm

[PATCH 01/16] drm/amd/powerplay: eliminate asic type check

2020-06-03 Thread Evan Quan
By moving ASIC specific code into its own file. Change-Id: Ib6775c1c8c5211ea45db6c3fb604a8279411ab37 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c| 59 +++ drivers/gpu/drm/amd/powerplay/arcturus_ppt.c | 8 +-- drivers/gpu/drm/amd/powerplay

[PATCH 06/16] drm/amd/powerplay: clean up the APIs for pptable setup

2020-06-03 Thread Evan Quan
Combine and simplify the logics for setup pptable. Change-Id: I062f15eab586050593afd960432c4c70fbdd5d41 Signed-off-by: Evan Quan Acked-by: Alex Deucher --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c| 17 drivers/gpu/drm/amd/powerplay/arcturus_ppt.c | 66 - .../gpu/drm/amd

[PATCH 07/16] drm/amd/powerplay: clean up the overdrive settings

2020-06-03 Thread Evan Quan
Eliminate the buffer allocation and drop the unnecessary overdrive table uploading. Change-Id: I8ba5383a330e6d5355cea219147500c1b4a43f47 Signed-off-by: Evan Quan Acked-by: Alex Deucher --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c| 2 +- .../gpu/drm/amd/powerplay/inc/amdgpu_smu.h| 2

[PATCH 13/16] drm/amd/powerplay: max code sharing between .hw_fini and .suspend

2020-06-03 Thread Evan Quan
Thus redundant code can be dropped. Change-Id: I672f84ed5856da53b7f8f915b2f24ca11cd4b228 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 44 +++--- 1 file changed, 23 insertions(+), 21 deletions(-) diff --git a/drivers/gpu/drm/amd/powerplay

[PATCH 04/16] drm/amd/powerplay: centralize all buffer allocation in sw_init phase

2020-06-03 Thread Evan Quan
To fit common design. And this can simplify the buffer deallocation. Change-Id: Iee682e76aadb5f34861d69d5794ced44f0a78789 Signed-off-by: Evan Quan Reviewed-by: Alex Deucher --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 342 ++--- drivers/gpu/drm/amd/powerplay/smu_v11_0.c

[PATCH 10/16] drm/amd/powerplay: max code sharing between .hw_init and .resume

2020-06-03 Thread Evan Quan
Then redundant code can be dropped. Change-Id: Icbafbb7ffc8189a09f4236786aea6702ee73f9f4 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 42 ++ 1 file changed, 18 insertions(+), 24 deletions(-) diff --git a/drivers/gpu/drm/amd/powerplay

[PATCH 14/16] drm/amd/powerplay: allocate the struct amdgpu_irq_src on the stack

2020-06-03 Thread Evan Quan
Since it is only several bytes in size. Change-Id: Ie9df0db543fdd4cf5b963a286ef40dee03c436bf Signed-off-by: Evan Quan Reviewed-by: Alex Deucher --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 3 --- drivers/gpu/drm/amd/powerplay/inc/amdgpu_smu.h | 2 +- drivers/gpu/drm/amd/powerplay

[PATCH 11/16] drm/amd/powerplay: resort those operations performed in hw setup

2020-06-03 Thread Evan Quan
Those common operations(for all ASICs) are placed first and followed by ASIC specific ones. While the display related are placed at the last. Change-Id: Id45caee98273c8c0b9c1c9f2713fcf8106e02000 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 61

[PATCH 02/16] drm/amd/powerplay: drop unused APIs and unnecessary checks

2020-06-03 Thread Evan Quan
Minor code cleanups. Change-Id: I6d240241e78cae17288c1d49dbae6ab1796b1128 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c| 74 --- .../gpu/drm/amd/powerplay/inc/amdgpu_smu.h| 2 - 2 files changed, 16 insertions(+), 60 deletions(-) diff --git

[PATCH] drm/amd/powerplay: use work queue to perform throttling logging

2020-06-02 Thread Evan Quan
As IO operations(access to SMU internals) and possible sleep are involved in throttling logging. Workqueue can handle them well. Otherwise we may hit "scheduling while atomic" error. Change-Id: I454d593e965e54b13fdf04c112abb0a022204278 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd

[PATCH 5/9] drm/amd/powerplay: clean up the APIs for pptable setup

2020-06-01 Thread Evan Quan
Combine and simplify the logics for setup pptable. Change-Id: I062f15eab586050593afd960432c4c70fbdd5d41 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c| 17 drivers/gpu/drm/amd/powerplay/arcturus_ppt.c | 66 - .../gpu/drm/amd/powerplay/inc

[PATCH 4/9] drm/amd/powerplay: clean up the APIs for bootup clocks

2020-06-01 Thread Evan Quan
Combine and simplify the logics for retrieving bootup clocks. Change-Id: Ifca28c454f3769dece0cc705ba054ff34db0ab60 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c| 4 - drivers/gpu/drm/amd/powerplay/arcturus_ppt.c | 1 - .../gpu/drm/amd/powerplay/inc

[PATCH 7/9] drm/amd/powerplay: clean up the SMU hw setup operations

2020-06-01 Thread Evan Quan
Postpone some operations which are not must for hw setup to late_init. Thus, code sharing is possible between hw_init/fini and suspend/resume. Also this makes code more clean and readable. Change-Id: Id3996fd9e2dbf2ff59d8a6032cc5f6730db1295c Signed-off-by: Evan Quan --- drivers/gpu/drm/amd

[PATCH 9/9] drm/amd/powerplay: add firmware cleanup on sw_fini

2020-06-01 Thread Evan Quan
To avoid possible memory leak. Change-Id: I4740eac7fc2c6e934ec8f503e5a98057f0902f4a Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 2 ++ drivers/gpu/drm/amd/powerplay/arcturus_ppt.c | 1 + drivers/gpu/drm/amd/powerplay/inc/amdgpu_smu.h | 1 + drivers/gpu/drm/amd

[PATCH 6/9] drm/amd/powerplay: clean up the overdrive settings

2020-06-01 Thread Evan Quan
Eliminate the buffer allocation and drop the unnecessary overdrive table uploading. Change-Id: I8ba5383a330e6d5355cea219147500c1b4a43f47 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c| 2 +- .../gpu/drm/amd/powerplay/inc/amdgpu_smu.h| 2 +- drivers/gpu/drm/amd

[PATCH 8/9] drm/amd/powerplay: drop unnecessary dynamic buffer allocation

2020-06-01 Thread Evan Quan
Since the structure comes with only several bytes. Change-Id: Ie9df0db543fdd4cf5b963a286ef40dee03c436bf Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 3 --- drivers/gpu/drm/amd/powerplay/inc/amdgpu_smu.h | 2 +- drivers/gpu/drm/amd/powerplay/smu_v11_0.c

[PATCH 3/9] drm/amd/powerplay: centralize all buffer allocation in sw_init phase

2020-06-01 Thread Evan Quan
To fit common design. And this can simplify the buffer deallocation. Change-Id: Iee682e76aadb5f34861d69d5794ced44f0a78789 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 330 ++--- drivers/gpu/drm/amd/powerplay/smu_v11_0.c | 105 --- 2 files

[PATCH 1/9] drm/amd/powerplay: drop unnecessary CHIP_ARCTURUS guard

2020-06-01 Thread Evan Quan
These APIs internally guard they will not break ARCTURUS. Change-Id: Ib6775c1c8c5211ea45db6c3fb604a8279411ab37 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 38 +--- drivers/gpu/drm/amd/powerplay/arcturus_ppt.c | 8 ++--- 2 files changed, 20

[PATCH 2/9] drm/amd/powerplay: some cosmetic fixes

2020-06-01 Thread Evan Quan
To make code more clean and readable by moving ASIC specific code to its own file, more code sharing and dropping unused code. Change-Id: I6b299f9e98c7678b48281cbed9beb17b644bb4cc Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 213 - drivers/gpu

[PATCH 1/3] drm/amd/powerplay: stop thermal IRQs on suspend

2020-05-28 Thread Evan Quan
Added missing thermal IRQs disablement on suspend. Change-Id: I959a1d56930de434cc8534334220d3faeadf79f8 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/drivers/gpu/drm/amd/powerplay/amdgpu_smu.c b/drivers/gpu/drm

[PATCH 2/3] drm/amd/powerplay: use the common APIs for IRQ disablement/enablement

2020-05-28 Thread Evan Quan
Also the new logics for MP1 SW IRQs disablement/enablement are added. Change-Id: I57ef8f21ab3d51aa0d557f511d89f5fa2ce08144 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/smu_v11_0.c | 79 --- 1 file changed, 57 insertions(+), 22 deletions(-) diff --git a/drivers

[PATCH 3/3] drm/amd/powerplay: give better names for the thermal IRQ related APIs

2020-05-28 Thread Evan Quan
Thermal control is performed by PMFW. What handled in driver is just whether or not to enable the alert(to driver). Change-Id: Icf857054b74f021e7fee2bf3aa9b314aa0d5ef09 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 8 drivers/gpu/drm/amd/powerplay

[PATCH 3/4] drm/amd/powerplay: enable thermal throttling logging support V2

2020-05-27 Thread Evan Quan
, the logging will be performed every minute to avoid log flooding. V2: simplify the implemention by ratelimited printk Change-Id: Ia4534700ee183c16b3231b461208bd0277f3b6a7 Signed-off-by: Evan Quan Acked-by: Christian König Reviewed-by: Alex Deucher --- drivers/gpu/drm/amd/powerplay/smu_v11_0.c | 11

[PATCH 4/4] drm/amdgpu: added a sysfs interface for thermal throttling related V3

2020-05-27 Thread Evan Quan
-by: Evan Quan Reviewed-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu.h| 3 + drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 11 drivers/gpu/drm/amd/amdgpu/amdgpu_pm.c | 71 ++ drivers/gpu/drm/amd/powerplay/smu_v11_0.c | 10 ++- 4 files changed, 89

[PATCH 2/4] drm/amd/powerplay: implement ASIC specific thermal throttling logging

2020-05-27 Thread Evan Quan
Enable this for Arcturus only for now. Change-Id: I4ad4fbb4f932d748849574e0f4dde01bb7db487a Signed-off-by: Evan Quan Reviewed-by: Alex Deucher --- drivers/gpu/drm/amd/powerplay/arcturus_ppt.c | 44 +++ .../gpu/drm/amd/powerplay/inc/amdgpu_smu.h| 1 + drivers/gpu/drm/amd

[PATCH 1/4] drm/amd/powerplay: update Arcturus smu-driver headers

2020-05-27 Thread Evan Quan
To fit the latest 54.24.0 PMFW. Change-Id: I38d08da6bf5120a7902b7366e09ca3fbc2c25f54 Signed-off-by: Evan Quan Reviewed-by: Alex Deucher Reviewed-by: Kenneth Feng --- .../drm/amd/powerplay/inc/smu11_driver_if_arcturus.h | 12 ++-- drivers/gpu/drm/amd/powerplay/inc/smu_v11_0.h

[PATCH] drm/amd/powerplay: check whether SMU IP is enabled before access

2020-05-25 Thread Evan Quan
Since on early phase of bringup, the SMU IP may be not enabled or supported. Without this, we may hit null pointer dereference on accessing smu->adev. Change-Id: I644175e926cd4fef8259f89002d6f8eda04fe42c Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c |

[PATCH] drm/amdkfd: report the real PCI bus number

2020-05-20 Thread Evan Quan
Since the PCI bus number retrieved by PCI_BUS_NUM(pdev->devfn) is wrong. Change-Id: I882a8531a65cdf91be20e34a034aca1f43f658b4 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/amdkfd/kfd_iommu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/gpu/drm/amd/amd

[PATCH] drm/amd/powerplay: unify the prompts on thermal interrupts

2020-05-20 Thread Evan Quan
The prompts will contain pci address(segment/bus/port/function), severity(warn or error) and some keywords(GPU, amdgpu). Also this address the issue that pci bus retrieved by PCI_BUS_NUM(adev->pdev->devfn) is wrong. Change-Id: I714d1dffb30a6cf76dcede087cf5d9302f683ed8 Signed-off-by: Eva

[PATCH] drm/amd/powerplay: ack the SMUToHost interrupt on receive V2

2020-05-12 Thread Evan Quan
There will be no further interrupt without proper ack for current one. V2: fix typo to really set ACK bit only Change-Id: Iad5adcaf7dd5c3a773b3d93ee0922a424dba8ac8 Signed-off-by: Evan Quan Reviewed-by: Alex Deucher --- drivers/gpu/drm/amd/powerplay/smu_v11_0.c | 6 ++ 1 file changed, 6

[PATCH] drm/amd/powerplay: ack the SMUToHost interrupt on receive

2020-05-12 Thread Evan Quan
There will be no further interrupt without proper ack for current one. Change-Id: Iad5adcaf7dd5c3a773b3d93ee0922a424dba8ac8 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/smu_v11_0.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/drivers/gpu/drm/amd/powerplay/smu_v11_0.c

[PATCH 1/2] drm/amd/powerplay: try to do a graceful shutdown on SW CTF

2020-05-09 Thread Evan Quan
Normally this(SW CTF) should not happen. And by doing graceful shutdown we can prevent further damage. Change-Id: I84f0c0643f78dcb5251089fe83305c460b815b62 Signed-off-by: Evan Quan --- .../gpu/drm/amd/powerplay/hwmgr/smu_helper.c | 21 +++ drivers/gpu/drm/amd/powerplay

[PATCH 2/2] drm/amd/powerplay: shutdown on HW CTF

2020-05-09 Thread Evan Quan
To prevent further damage. Change-Id: Iae0399001694bb7446bcc6071cd03d8fa47c9d02 Signed-off-by: Evan Quan --- .../gpu/drm/amd/powerplay/hwmgr/smu_helper.c | 16 +-- drivers/gpu/drm/amd/powerplay/smu_v11_0.c | 20 ++- 2 files changed, 33 insertions(+), 3 deletions

[PATCH] drm/amd/powerplay: report correct AC/DC event based on ctxid V2

2020-05-08 Thread Evan Quan
'ctxid' is used to distinguish different events raised from SMC. 0x3 and 0x4 are for AC and DC power mode. V2: update the way to retrieve the ctxid and change the log level to debug Change-Id: I7dbcb053fe9cea7c70e0a33afc2115b3745f6186 Signed-off-by: Evan Quan Reviewed-by: Alex Deucher

[PATCH 2/2] drm/amd/powerplay: report correct AC/DC event based on ctxid

2020-05-08 Thread Evan Quan
'ctxid' is used to distinguish different events raised from SMC. 0x3 and 0x4 are for AC and DC power mode. Change-Id: I7dbcb053fe9cea7c70e0a33afc2115b3745f6186 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/smu_v11_0.c | 15 +-- 1 file changed, 13 insertions(+), 2

[PATCH 1/2] drm/amdgpu: decode the ctxid for SMC to host IH

2020-05-08 Thread Evan Quan
Driver needs that to tell the different events raised from SMC. Change-Id: I0e44e22527182fbb45a2db4fc3b1cd73fb17ba33 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h | 1 + drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 1 + drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 1 + 3 files

[PATCH] drm/amdgpu: enable hibernate support on Navi1X

2020-05-07 Thread Evan Quan
BACO is needed to support hibernate on Navi1X. Change-Id: Ia283612919e9a5b4e1e198ab2cc0bc15f7f94760 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/amdgpu/amdgpu.h| 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c| 2 ++ drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 2 +- 3 files

[PATCH 1/3] drm/amdgpu: disable MGCG/MGLS also on gfx CG ungate

2020-04-30 Thread Evan Quan
Otherwise, MGCG/MGLS will be left enabled. Change-Id: I620c5a58b814ae4a0f6005bbc467d7cc2b6d0d66 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c b/drivers/gpu/drm/amd

[PATCH 3/3] drm/amd/powerplay: perform PG ungate prior to CG ungate

2020-04-30 Thread Evan Quan
Since gfxoff should be disabled first before trying to access those GC registers. Change-Id: I5bf0fbe01fa05c89bbb392ef40e11c07edfee039 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amd_powerplay.c | 6 +++--- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c| 6 +++--- 2 files changed

[PATCH 2/3] drm/amdgpu: drop unnecessary cancel_delayed_work_sync on PG ungate

2020-04-30 Thread Evan Quan
As this is already properly handled in amdgpu_gfx_off_ctrl(). In fact, this unnecessary cancel_delayed_work_sync may leave a small time window for race condition and is dangerous. Change-Id: I601b7b86850db3ed4f4990fd796de663401451ca Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/amdgpu

[PATCH] drm/amdgpu: put the audio codec into suspend state before gpu reset V3

2020-04-28 Thread Evan Quan
V2: limit this for BACO and mode1 reset only V3: try 1st to use pm_runtime_autosuspend_expiration() to query how much time is left. Use default setting on failure Change-Id: I33d85e6fcad1882eb33f9cde8916d57be8d5a87a Signed-off-by: Evan Quan Reviewed-by: Alex Deucher --- drivers/gpu/drm/

[PATCH] drm/amdgpu: address the static checker warnings V2

2020-04-28 Thread Evan Quan
ever fails on this Change-Id: I9c22b57abc9f512114112f93fb035f1fecf26beb Signed-off-by: Evan Quan Reported-by: Dan Carpenter --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgp

[PATCH] drm/amdgpu: address the static checker warnings

2020-04-24 Thread Evan Quan
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c:4199 amdgpu_device_gpu_recover() error: we previously assumed 'hive' could be null (see line 4196) This is introduced by "drm/amdgpu: optimize the gpu reset for XGMI setup V2". Change-Id: I9c22b57abc9f512114112f93fb035f1fecf26beb Signed-of

[PATCH 2/2] drma/dmgpu: drop redundant cg/pg ungate on runpm enter

2020-04-24 Thread Evan Quan
: I489ca456770d3fe482b685f132400202467f712b Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 08eeb0d2c149..71278942f9f0 100644 --- a/drivers/gpu

[PATCH 1/2] drm/amdgpu: move kfd suspend after ip_suspend_phase1

2020-04-24 Thread Evan Quan
This sequence change should be safe as what did in ip_suspend_phase1 is to suspend DCE only. And this is a prerequisite for coming redundant cg/pg ungate dropping. Change-Id: Ie125e234f8f743d327cf8e0389e4872a312184c0 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 4

[PATCH] drm/amdgpu: put the audio codec into suspend state before gpu reset V2

2020-04-21 Thread Evan Quan
V2: limit this for BACO and mode1 reset only Change-Id: I33d85e6fcad1882eb33f9cde8916d57be8d5a87a Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 70 ++ 1 file changed, 70 insertions(+) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/

[PATCH] drm/amdgpu: put the audio codec into suspend state before gpu reset

2020-04-21 Thread Evan Quan
Change-Id: I33d85e6fcad1882eb33f9cde8916d57be8d5a87a Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 60 ++ 1 file changed, 60 insertions(+) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 2d

[PATCH 3/4] drm/amdgpu: optimize the gpu reset for XGMI setup V2

2020-04-20 Thread Evan Quan
the devices in a single 'for' loop. V2: added missing hive->hive_lock protection Change-Id: I84dca425f1ae778c4b4b8bc3a0d2b9a3d1b50043 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 76 +++--- 1 file changed, 25 insertions(+), 51 deletions(-) d

[PATCH 4/4] drm/amdgpu: code cleanup around gpu reset

2020-04-20 Thread Evan Quan
Make code more readable. Change-Id: I28444f285b23aac16be421e3447d0de6c3a57ee8 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 14 -- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm

[PATCH 2/4] drm/amdgpu: correct cancel_delayed_work_sync on gpu reset

2020-04-20 Thread Evan Quan
As for XGMI setup, it should be performed on other devices from the hive also. Change-Id: I08554c27216efa21c2c46c0b3379d856b5264c9e Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/gpu/drm/amd/amdgpu

[PATCH 1/4] drm/amdgpu: correct fbdev suspend on gpu reset

2020-04-20 Thread Evan Quan
As for XGMI setup, it needs to be performed on all the devices from the same hive. Change-Id: I25e6364d31f0b34938cf424a410628aa54dd2edd Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/gpu/drm/amd

[PATCH 0/4] Some XGMI related gpu reset fixes and cleanups

2020-04-20 Thread Evan Quan
Patch 1 and 2 are the necessary fixes for XGMI setup. Since these operations are needed for other devices from the same hive. That's missing now. Patch 3 are 4 are basically code cosmetic. Evan Quan (4): drm/amdgpu: correct fbdev suspend on gpu reset drm/amdgpu: correct

[PATCH] drm/amdgpu: load SMU IP for onevf mode on Navi1x V2

2020-04-16 Thread Evan Quan
SMU IP needs to be loaded for onevf mode. Otherwise, there may be accesses without initialization. V2: added SMU IP for all sriov cases since there is necessary checks in IP operations(hw_init/fini) Change-Id: I513aa4140f1169ca048b64985cafe9c7577afca7 Signed-off-by: Evan Quan --- drivers

[PATCH] drm/amd/powerplay: update Arcturus smu-driver if header

2020-04-13 Thread Evan Quan
To fit the latest PMFW. Change-Id: I1ebc5a4c3386e07a9fed73e4bec2c5d1ab8d4b23 Signed-off-by: Evan Quan --- .../amd/powerplay/inc/smu11_driver_if_arcturus.h | 15 +-- drivers/gpu/drm/amd/powerplay/inc/smu_v11_0.h | 2 +- 2 files changed, 14 insertions(+), 3 deletions(-) diff

[PATCH 2/2] drm/amd/powerplay: bump the NAVI10 smu-driver if version

2020-04-13 Thread Evan Quan
To fit the latest SMC firmware 42.53 and eliminate the warning on driver loading. Change-Id: I3f524d03e53ec6778b7118ef72652a538eec4ace Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/inc/smu_v11_0.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/gpu/drm

[PATCH 1/2] drm/amd/powerplay: revise the way to retrieve the board parameters

2020-04-13 Thread Evan Quan
It can support different NV1x ASIC better. And this can guard no member got missing. Change-Id: Id5e6608f6be1b31ef1c0a5c1d399da295524ff43 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/include/atomfirmware.h | 102 + drivers/gpu/drm/amd/powerplay/navi10_ppt.c | 99

[PATCH 2/2] drm/amd/powerplay: properly set the dpm_enabled state

2020-04-13 Thread Evan Quan
On the ASIC powered down(in baco or system suspend), the dpm_enabled will be set as false. Then all access (e.g. df state setting issued on RAS error event) to SMU will be blocked. Change-Id: I128c681b6f520d24d4247e438773b2b4b497d43b Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay

[PATCH 1/2] drm/amd/powerplay: correct i2c eeprom init/fini sequence

2020-04-13 Thread Evan Quan
: I8444c05b51e67347841641fe5768d3126d089792 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c| 22 ++- .../gpu/drm/amd/powerplay/inc/amdgpu_smu.h| 5 - drivers/gpu/drm/amd/powerplay/smu_internal.h | 5 + 3 files changed, 17 insertions(+), 15 deletions(-) diff --git a/drivers/gpu

[PATCH] drm/amdgpu: fix wrong vram lost counter increment V2

2020-04-12 Thread Evan Quan
Vram lost counter is wrongly increased by two during baco reset. V2: assumed vram lost for mode1 reset on all ASICs Change-Id: I8b9959a5d1632abc774ba07d56cf295bdd8288eb Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 20 ++-- drivers/gpu/drm/amd/amdgpu

[PATCH] drm/amdgpu: fix wrong vram lost counter increment

2020-04-10 Thread Evan Quan
Vram lost counter is wrongly increased by two during baco reset. Change-Id: I8b9959a5d1632abc774ba07d56cf295bdd8288eb Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 36 -- drivers/gpu/drm/amd/amdgpu/cik.c | 2 -- drivers/gpu/drm/amd

[PATCH] drm/amd/powerplay: error out on forcing clock setting not supported

2020-04-03 Thread Evan Quan
a disservice. Change-Id: Ibffdb157639809707aa71049a935e53d1b4d06eb Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/arcturus_ppt.c | 47 +++- 1 file changed, 46 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/amd/powerplay/arcturus_ppt.c b/drivers/gpu/drm/amd

[PATCH 2/2] drm/amd/powerplay: move the ASIC specific nbio operation out of smu_v11_0.c

2020-03-27 Thread Evan Quan
This is ASIC specific and should be placed in _ppt.c of each ASIC. Change-Id: If2b29ccf0dfc0dc90e1636a806b13ce2efed58c6 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/arcturus_ppt.c | 15 ++- drivers/gpu/drm/amd/powerplay/navi10_ppt.c | 18 -- drivers

[PATCH 1/2] drm/amd/powerplay: drop redundant BIF doorbell interrupt operations

2020-03-27 Thread Evan Quan
This is already done in soc15.c. And this is really ASIC specific and should not be placed here. Change-Id: I7c9ee3cce07463849a29c7bcbcb493f817626274 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/smu_v11_0.c | 14 -- 1 file changed, 14 deletions(-) diff --git a/drivers

[PATCH] drm/amdgpu: fix non-pointer dereference for non-RAS supported

2020-03-27 Thread Evan Quan
nge-Id: I053089a8e44749db23ec882d14f876ded9c1332d Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c index 9e9e0f7747b7..8a78db648442 100644 --- a/drive

[PATCH 3/9] drm/amd/powerpaly: drop unused APIs

2020-03-26 Thread Evan Quan
Drop unused smu7 message APIs. Change-Id: I8fc13b626fad04241f2b9353419282eaa14923af Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/smumgr/smu7_smumgr.c | 14 -- drivers/gpu/drm/amd/powerplay/smumgr/smu7_smumgr.h | 3 --- 2 files changed, 17 deletions(-) diff --git

[PATCH 2/9] drm/amd/powerplay: avoid calling SMU7 specific SMU message implemention

2020-03-26 Thread Evan Quan
Prepare for coming lock protection for SMU message issuing. Change-Id: Ie1365f350abcde4c4474e9bbdaf7742349a0534d Signed-off-by: Evan Quan --- .../gpu/drm/amd/powerplay/hwmgr/smu7_hwmgr.c | 10 +++ .../drm/amd/powerplay/hwmgr/smu7_thermal.c| 8 +++--- .../drm/amd/powerplay/smumgr

[PATCH 1/9] drm/amd/powerplay: avoid calling CI specific SMU message implemention

2020-03-26 Thread Evan Quan
Prepare for coming lock protection for SMU message issuing. Change-Id: If5f7615483d9967f9512fba49cc8454a1300ba6c Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/smumgr/ci_smumgr.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/amd/powerplay

[PATCH 5/9] drm/amd/powerplay: avoid calling SMU9 specific SMU message implemention

2020-03-26 Thread Evan Quan
Prepare for coming lock protection for SMU message issuing. Change-Id: Id2c0504c60358e6ff2beee83058e36812962e934 Signed-off-by: Evan Quan --- .../drm/amd/powerplay/smumgr/vega10_smumgr.c | 24 +++--- .../drm/amd/powerplay/smumgr/vega12_smumgr.c | 32 +-- 2 files

[PATCH 4/9] drm/amd/powerplay: avoid calling SMU8 specific SMU message implemention

2020-03-26 Thread Evan Quan
Prepare for coming lock protection for SMU message issuing. Change-Id: Ia4a07898410a661d8603474ec253859278f00d3c Signed-off-by: Evan Quan --- .../gpu/drm/amd/powerplay/hwmgr/smu8_hwmgr.c | 2 +- .../drm/amd/powerplay/smumgr/smu8_smumgr.c| 30 +-- 2 files changed, 16

[PATCH 6/9] drm/amd/powerplay: avoid calling SMU10 specific SMU message implemention

2020-03-26 Thread Evan Quan
Prepare for coming lock protection for SMU message issuing. Change-Id: I05ec712b05dcb2831c948a1df26695619ba8014a Signed-off-by: Evan Quan --- .../drm/amd/powerplay/smumgr/smu10_smumgr.c| 18 +- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/drivers/gpu/drm/amd

[PATCH] drm/amd/powerplay: unload mp1 for Arcturus RAS baco reset

2020-03-25 Thread Evan Quan
This sequence is recommended by PMFW team for the baco reset with PMFW reloaded. And it seems able to address the random failure seen on Arcturus. Change-Id: Ic5a83ad46466a12533e9f704ad06348852156e78 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/smu_v11_0.c | 6 ++ 1 file

[PATCH] drm/amd/powerplay: correct the bootup power source for Navi1X

2020-03-19 Thread Evan Quan
PMFW may boots those ASICs with DC mode. Need to set it back to AC mode. Change-Id: I56ffd0e747f778aa013da43a8693ddfb5da31e3c Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 15 +++ drivers/gpu/drm/amd/powerplay/inc/amdgpu_smu.h | 1 + drivers/gpu/drm

[PATCH] drm/amdgpu: reenable runtime pm on navi12

2020-03-11 Thread Evan Quan
The runtime pm is verified as working now on navi12. Change-Id: I20393633678297308c9651237bbfdc854a3cff94 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers

[PATCH] drm/amdgpu: add fbdev suspend/resume on gpu reset

2020-03-11 Thread Evan Quan
This can fix the baco reset failure seen on Navi10. And this should be a low risk fix as the same sequence is already used for system suspend/resume. Change-Id: Idb4d02c5fcbbd5b7817195ee04c7af34c346a053 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 4 1 file

[PATCH] drm/amdgpu: check for the existence of RAS dir before creating

2020-03-09 Thread Evan Quan
To address the error message below: debugfs: Directory 'ras' with parent '/' already present! Change-Id: I2539e89fdfe4e22055c3be5a48a8c0adad315f91 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers

[PATCH] drm/amd/powerplay: bump the NAVI10 smu-driver if version

2020-03-06 Thread Evan Quan
To fit the latest SMC firmware 53.27 and eliminate the warning on driver loading. Change-Id: I3f524d03e53ec6778b7118ef72652a538eec4ace Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/inc/smu_v11_0.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/gpu/drm

[PATCH] drm/amd/powerplay: revise the way to retrieve the board parameters

2020-03-06 Thread Evan Quan
It can support different NV1x ASIC better. And this can guard no member got missing. Change-Id: Id5e6608f6be1b31ef1c0a5c1d399da295524ff43 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/include/atomfirmware.h | 102 + drivers/gpu/drm/amd/powerplay/navi10_ppt.c | 99

[PATCH] drm/amd/powerplay: honor the BACO setting from VBIOS

2020-03-04 Thread Evan Quan
Should not enable BACO always on Vega20. That will break the VBIOS setting. Change-Id: I2609fc3f86dc785d385d9320574446810b5b693d Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/hwmgr/vega20_hwmgr.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/drivers/gpu/drm/amd/powerplay

[PATCH] drm/amdgpu: update psp firmwares loading sequence V2

2020-02-24 Thread Evan Quan
For those ASICs with DF Cstate management centralized to PMFW, TMR setup should be performed between pmfw loading and other non-psp firmwares loading. V2: skip possible SMU firmware reloading Change-Id: I8986ddb4d9ffe63ed0823d1dce8d9d52812a1240 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd

[PATCH] drm/amdgpu: update psp firmwares loading sequence

2020-02-24 Thread Evan Quan
For those ASICs with DF Cstate management centralized to PMFW, TMR setup should be performed between pmfw loading and other non-psp firmwares loading. Change-Id: I8986ddb4d9ffe63ed0823d1dce8d9d52812a1240 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c | 61

[PATCH] drm/amdgpu: drop the non-sense firmware version check on arcturus

2020-02-17 Thread Evan Quan
As the firmware versions of arcturus are different from other gfx9 ASICs. And the warning("CP firmware version too old, please update!") caused by this check can be eliminated. Change-Id: I04ab0e435b6ff9712be502e6ebb8d6e1f0a776d8 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/amdgp

[PATCH] drm/amd/powerplay: correct the way for checking SMU_FEATURE_BACO_BIT support

2020-02-11 Thread Evan Quan
Since 'smu_feature_is_enabled(smu, SMU_FEATURE_BACO_BIT)' will always return false considering the 'smu_system_features_control(smu, false)' disabled all SMU features. Change-Id: I73956ffa51d6da8375c7c377895a221e13d31594 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c

[PATCH] drm/amd/powerplay: always refetch the enabled features status on dpm enablement

2020-02-10 Thread Evan Quan
Otherwise, the cached dpm features status may be inconsistent under some case(e.g. baco reset of Navi asic). Change-Id: Ia7d7089e82cbe4ccaa9fadce1f2f0043c42c73df Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/smu_v11_0.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions

[PATCH] drm/amd/powerplay: update smu11_driver_if_navi10.h

2020-02-09 Thread Evan Quan
To pair the latest SMU firmwares. Change-Id: I5262c750fa08bc6268b43e3420e110e9ee71ccf6 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/inc/smu11_driver_if_navi10.h | 3 ++- drivers/gpu/drm/amd/powerplay/inc/smu_v11_0.h | 4 ++-- 2 files changed, 4 insertions(+), 3

[PATCH] drm/amd/powerplay: handle features disablement for baco reset in SMU FW

2020-02-06 Thread Evan Quan
SMU FW will handle the features disablement for baco reset on Arcturus. Change-Id: Ifd87a09de2fca0c67c041afbd5e711973769119a Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c | 53 +- 1 file changed, 42 insertions(+), 11 deletions(-) diff --git

[PATCH] drm/amd/powerplay: fix navi10 system intermittent reboot issue

2020-01-30 Thread Evan Quan
This workaround is needed only for Navi10 12 Gbps SKUs. Change-Id: I4bfcb8a8dbff785a159e6a1ed413d93063403ab3 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c| 18 +++ .../gpu/drm/amd/powerplay/inc/amdgpu_smu.h| 1 + drivers/gpu/drm/amd/powerplay/inc

[PATCH] drm/amd/powerplay: a quick fix for the deadlock issue below

2020-01-15 Thread Evan Quan
SMU". Change-Id: I6ae2ed89d40a9475ae37b58e22108f4d6da974bb Signed-off-by: Evan Quan Reported-by: Rui Teng --- drivers/gpu/drm/amd/amdgpu/amdgpu_dpm.c | 40 +++-- 1 file changed, 25 insertions(+), 15 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dpm.c b/d

[PATCH] drm/amd/powerplay: cover the powerplay implementation details V2

2020-01-07 Thread Evan Quan
This can save users much troubles. As they do not actually need to care whether swSMU or traditional powerplay routine should be used. V2: apply the fixes to vi.c and cik.c also Change-Id: I827fe4cac0d4fd487782168a7fe73e15756a7109 Signed-off-by: Evan Quan Reviewed-by: Alex Deucher --- drivers

[PATCH] drm/amd/powerplay: cover the powerplay implementation details

2020-01-07 Thread Evan Quan
This can save users much troubles. As they do not actually need to care whether swSMU or traditional powerplay routine should be used. Change-Id: I827fe4cac0d4fd487782168a7fe73e15756a7109 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 12 +- drivers/gpu/drm/amd

[PATCH] drm/amd/powerplay: refine code to support no-dpm case

2020-01-03 Thread Evan Quan
With "dpm=0", there will be no DPM enabled. The code needs to be refined to support this. Change-Id: Ief1561e69a24472135e0499073a27ea5fd8174e2 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amd_powerplay.c | 5 - drivers/gpu/drm/amd/powerplay/amdgpu_smu

[PATCH] drm/amd/powerplay: issue proper hdp flush for table transferring

2020-01-03 Thread Evan Quan
Guard the content consistence between the view of GPU and CPU during the table transferring. Change-Id: Ib3cebb97a1c8fb302eb040483bbaf089ae00c6a9 Signed-off-by: Evan Quan --- drivers/gpu/drm/amd/powerplay/amdgpu_smu.c| 15 ++- .../gpu/drm/amd/powerplay/smumgr/smu10_smumgr.c

<    4   5   6   7   8   9   10   11   12   13   >