[Intel-gfx] [PATCH i-g-t 3/9] lib/kunit: Fix misplaced igt_kunit() doc

2023-10-03 Thread Janusz Krzysztofik
: Janusz Krzysztofik --- lib/igt_kmod.c | 17 - 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c index df0e650d49..426ae5b26f 100644 --- a/lib/igt_kmod.c +++ b/lib/igt_kmod.c @@ -762,15 +762,6 @@ static void *modprobe_task(void *arg

[Intel-gfx] [PATCH i-g-t 4/9] lib/kunit: Parse KTAP report from the main process thread

2023-10-03 Thread Janusz Krzysztofik
been processed, - fix trailing newlines missing from error messages, - add more debug statements, - integrate common code around kunit_result_free() into it. v2: Interrupt blocking read() on modprobe failure. Signed-off-by: Janusz Krzysztofik Acked-by: Mauro Carvalho Chehab # v2 --- lib

[Intel-gfx] [PATCH i-g-t 5/9] lib/kunit: Omit suite name prefix if the same as subtest name

2023-10-03 Thread Janusz Krzysztofik
is into account when composing names for IGT dynamic sub-subtest names and drop the leading test suite name component when it is the same as subtest name. Signed-off-by: Janusz Krzysztofik Reviewed-by: Mauro Carvalho Chehab --- lib/igt_kmod.c | 11 --- 1 file changed, 8 insertions(+),

[Intel-gfx] [PATCH i-g-t 8/9] lib/kunit: Fetch a list of test cases in advance

2023-10-03 Thread Janusz Krzysztofik
list of test cases obtained in advance, then we stop parsing KTAP report as soon as we get a result from the last test case from the list. Signed-off-by: Janusz Krzysztofik --- lib/igt_kmod.c | 217 +++-- 1 file changed, 157 insertions(+), 60 deletions(-)

[Intel-gfx] [PATCH i-g-t 7/9] lib/ktap: Drop workaround for missing top level KTAP headers

2023-10-03 Thread Janusz Krzysztofik
has fixed that issue, that workaround is no longer needed. Drop it. Signed-off-by: Janusz Krzysztofik --- lib/igt_ktap.c | 12 lib/tests/igt_ktap_parser.c | 3 +-- 2 files changed, 1 insertion(+), 14 deletions(-) diff --git a/lib/igt_ktap.c b/lib/igt_ktap.c index

[Intel-gfx] [PATCH i-g-t 6/9] tests/kms_selftest: Let subtest names match suite names

2023-10-03 Thread Janusz Krzysztofik
to kunit and the i915 module potentially providing three test suites: mock, live and perf). [1] https://docs.kernel.org/dev-tools/kunit/style.html Signed-off-by: Janusz Krzysztofik --- tests/kms_selftest.c | 37 - 1 file changed, 16 insertions(+), 21 deletions

[Intel-gfx] [PATCH i-g-t 9/9] lib/kunit: Execute kunit test cases only when needed

2023-10-03 Thread Janusz Krzysztofik
of dynamic sub-subtests, and do that only from the first actually executed dynamic sub-subtest. Signed-off-by: Janusz Krzysztofik --- lib/igt_kmod.c | 59 -- 1 file changed, 33 insertions(+), 26 deletions(-) diff --git a/lib/igt_kmod.c b/lib/igt_km

Re: [Intel-gfx] [PATCH i-g-t 8/9] lib/kunit: Fetch a list of test cases in advance

2023-10-06 Thread Janusz Krzysztofik
Hi Kamil, Thanks for review. On Thursday, 5 October 2023 23:28:44 CEST Kamil Konieczny wrote: > Hi Janusz, > > On 2023-10-03 at 11:10:53 +0200, Janusz Krzysztofik wrote: > > Recent improvements to the kernel kunit framework allow us to obtain a > > list of test cases pro

[Intel-gfx] [PATCH i-g-t v2 00/11] Kunit fixes and improvements

2023-10-09 Thread Janusz Krzysztofik
- still break the loop of test cases on unexpected return codes from kunit_kmsg_get_result(), - fix typos (Kamil), - update commit descriptions. Janusz Krzysztofik (11): lib/kunit: Fix handling of potential errors from F_GETFL lib/kunit: Be more verbose on errors lib/kunit: Fix

[Intel-gfx] [PATCH i-g-t v2 02/11] lib/kunit: Be more verbose on errors

2023-10-09 Thread Janusz Krzysztofik
Use a more verbose variant of igt_fail() when failing a dynamic sub- subtest on kernel taint. Also, print a debug message on string duplication failure. Signed-off-by: Janusz Krzysztofik Reviewed-by: Kamil Konieczny --- lib/igt_kmod.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions

[Intel-gfx] [PATCH i-g-t v2 01/11] lib/kunit: Fix handling of potential errors from F_GETFL

2023-10-09 Thread Janusz Krzysztofik
Function fcntl(..., F_GETFL, ...) that returns file status flags may also return a negative error code. Handle that error instead of blindly using the returned value as flags. Signed-off-by: Janusz Krzysztofik Reviewed-by: Kamil Konieczny --- lib/igt_kmod.c | 4 ++-- 1 file changed, 2

[Intel-gfx] [PATCH i-g-t v2 03/11] lib/kunit: Fix misplaced igt_kunit() doc

2023-10-09 Thread Janusz Krzysztofik
: Janusz Krzysztofik Reviewed-by: Kamil Konieczny --- lib/igt_kmod.c | 17 - 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c index df0e650d49..426ae5b26f 100644 --- a/lib/igt_kmod.c +++ b/lib/igt_kmod.c @@ -762,15 +762,6 @@ static void

[Intel-gfx] [PATCH i-g-t v2 04/11] lib/kunit: Parse KTAP report from the main process thread

2023-10-09 Thread Janusz Krzysztofik
been processed, - fix trailing newlines missing from error messages, - add more debug statements, - integrate common code around kunit_result_free() into it. v2: Interrupt blocking read() on modprobe failure. Signed-off-by: Janusz Krzysztofik Acked-by: Mauro Carvalho Chehab # v2 --- lib

[Intel-gfx] [PATCH i-g-t v2 05/11] lib/kunit: Omit suite name prefix if the same as subtest name

2023-10-09 Thread Janusz Krzysztofik
is into account when composing names for IGT dynamic sub-subtest names and drop the leading test suite name component when it is the same as subtest name. Signed-off-by: Janusz Krzysztofik Reviewed-by: Mauro Carvalho Chehab --- lib/igt_kmod.c | 11 --- 1 file changed, 8 insertions(+),

[Intel-gfx] [PATCH i-g-t v2 07/11] lib/ktap: Drop workaround for missing top level KTAP headers

2023-10-09 Thread Janusz Krzysztofik
has fixed that issue, that workaround is no longer needed. Drop it. Signed-off-by: Janusz Krzysztofik --- lib/igt_ktap.c | 12 lib/tests/igt_ktap_parser.c | 3 +-- 2 files changed, 1 insertion(+), 14 deletions(-) diff --git a/lib/igt_ktap.c b/lib/igt_ktap.c index

[Intel-gfx] [PATCH i-g-t v2 06/11] tests/kms_selftest: Let subtest names match suite names

2023-10-09 Thread Janusz Krzysztofik
to kunit and the i915 module potentially providing three test suites: mock, live and perf). [1] https://docs.kernel.org/dev-tools/kunit/style.html Signed-off-by: Janusz Krzysztofik Reviewed-by: Kamil Konieczny --- tests/kms_selftest.c | 37 - 1 file changed

[Intel-gfx] [PATCH i-g-t v2 08/11] lib/kunit: Provide all results cleanup helper

2023-10-09 Thread Janusz Krzysztofik
Planned changes require a couple of loops around kunit_result_free(). Since we already have such loop, move it into a helper in preparation for future uses. Signed-off-by: Janusz Krzysztofik --- lib/igt_kmod.c | 20 ++-- 1 file changed, 14 insertions(+), 6 deletions(-) diff

[Intel-gfx] [PATCH i-g-t v2 09/11] lib/kunit: Prepare for KTAP parsing after modprobe completed

2023-10-09 Thread Janusz Krzysztofik
performed after the kunit test module is successfully loaded and there will be no concurrently running modprobe thread, we need to make synchronization of reads from /dev/kmsg with potential errors modprobe thread optional. Signed-off-by: Janusz Krzysztofik --- lib/igt_kmod.c | 50

[Intel-gfx] [PATCH i-g-t v2 10/11] lib/kunit: Fetch a list of test cases in advance

2023-10-09 Thread Janusz Krzysztofik
ly true in first dynamic sub-subtest, and we always free last result before looping to next sub-subtest, - still break the loop of test cases on unexpected return codes from kunit_kmsg_get_result(), - use kunit_results_free() helper, - fix typos (Kamil), - update commit description. S

[Intel-gfx] [PATCH i-g-t v2 11/11] lib/kunit: Execute kunit test cases only when needed

2023-10-09 Thread Janusz Krzysztofik
of dynamic sub-subtests, and do that only from the first actually executed dynamic sub-subtest. v2: Always pass last result to next dynamic sub-subtest, fetch first result right after loading the kunit test module for execution. Signed-off-by: Janusz Krzysztofik --- lib/igt_k

[Intel-gfx] [PATCH i-g-t] tests/gem_reset_stats: Don't allow request watchdog to interfere

2023-04-13 Thread Janusz Krzysztofik
chdog interfers with engine heartbeat and request preemption used by the test for exercising reset statistics. Disable request watchdog during the test execution. Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/8310 Signed-off-by: Janusz Krzysztofik --- tests/i915/gem_reset_stats.c | 7 +

[Intel-gfx] [RFC PATCH] x86/mm: Fix PAT bit missing from page protection modify mask

2023-04-24 Thread Janusz Krzysztofik
and the _PAGE_CHG_MASK symbol, primarly used by pte_modify(), is likely intentionally defined with that bit not set, keep that symbol unchanged. Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/7648 Fixes: 281d4078bec3 ("x86: Make page cache mode a real type") Si

[Intel-gfx] [PATCH v2] x86/mm: Fix PAT bit missing from page protection modify mask

2023-06-07 Thread Janusz Krzysztofik
CHG_MASK (Juergen). [1] https://gitlab.freedesktop.org/drm/igt-gpu-tools/-/commit/0f0754413f14 Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/7648 Fixes: 281d4078bec3 ("x86: Make page cache mode a real type") Signed-off-by: Janusz Krzysztofik Tested-by: Marek Marczykowski-Górecki Re

Re: [Intel-gfx] [PATCH v2] x86/mm: Fix PAT bit missing from page protection modify mask

2023-06-07 Thread Janusz Krzysztofik
On Wednesday, 7 June 2023 17:31:24 CEST Dave Hansen wrote: > On 6/7/23 08:23, Janusz Krzysztofik wrote: > > > > Extend bitmask used by pgprot_modify() for selecting bits to be preserved > > with _PAGE_PAT bit. However, since that bit can be reused as _PAGE_PSE, > > an

Re: [Intel-gfx] [PATCH v2] x86/mm: Fix PAT bit missing from page protection modify mask

2023-06-07 Thread Janusz Krzysztofik
On Wednesday, 7 June 2023 23:12:13 CEST Edgecombe, Rick P wrote: > On Wed, 2023-06-07 at 19:11 +0200, Janusz Krzysztofik wrote: > > On Wednesday, 7 June 2023 17:31:24 CEST Dave Hansen wrote: > > > On 6/7/23 08:23, Janusz Krzysztofik wrote: > > > > > > >

Re: [Intel-gfx] [PATCH v2] x86/mm: Fix PAT bit missing from page protection modify mask

2023-06-08 Thread Janusz Krzysztofik
On Thursday, 8 June 2023 00:47:36 CEST Edgecombe, Rick P wrote: > On Wed, 2023-06-07 at 23:33 +0200, Janusz Krzysztofik wrote: > > > So since _PAGE_PSE is actually the same value as _PAGE_PAT, you > > > don't > > > actually need to have _PAGE_PSE in _HPAGE_CHG_MA

[Intel-gfx] [PATCH v3] x86/mm: Fix PAT bit missing from page protection modify mask

2023-06-09 Thread Janusz Krzysztofik
/7648 Fixes: 281d4078bec3 ("x86: Make page cache mode a real type") Signed-off-by: Janusz Krzysztofik Tested-by: Marek Marczykowski-Górecki Reviewed-by: Andi Shyti Acked-by: Juergen Gross # v1 Cc: Borislav Petkov Cc: Dave Hansen Cc: "Edgecombe, Rick P" Cc: sta...@vger.kernel.or

[Intel-gfx] [PATCH] drm/i915: Fix premature release of request's reusable memory

2023-07-04 Thread Janusz Krzysztofik
previous active i915 requests from being released while still accessing their memory. Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/8211 Fixes: b1e3177bd1d8 ("drm/i915: Coordinate i915_active with its own mutex") Signed-off-by: Janusz Krzysztofik Cc: # v5.5+ --- drivers/g

[Intel-gfx] [PATCH v3 RESEND] x86/mm: Fix PAT bit missing from page protection modify mask

2023-07-10 Thread Janusz Krzysztofik
/7648 Fixes: 281d4078bec3 ("x86: Make page cache mode a real type") Signed-off-by: Janusz Krzysztofik Tested-by: Marek Marczykowski-Górecki Reviewed-by: Andi Shyti Reviewed-by: Juergen Gross Cc: Borislav Petkov Cc: Dave Hansen Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "E

Re: [Intel-gfx] [PATCH v3 RESEND] x86/mm: Fix PAT bit missing from page protection modify mask

2023-07-17 Thread Janusz Krzysztofik
Hi Borislav, On Monday, 10 July 2023 10:05:19 CEST Borislav Petkov wrote: > On Mon, Jul 10, 2023 at 09:36:14AM +0200, Janusz Krzysztofik wrote: > > Assume that adding _PAGE_PAT to _PAGE_CHG_MASK doesn't break pte_modify() > > and its users, and go for it. Also, add _PAGE_PAT_

Re: [Intel-gfx] [PATCH v3 RESEND] x86/mm: Fix PAT bit missing from page protection modify mask

2023-07-17 Thread Janusz Krzysztofik
Hi Rick, On Monday, 10 July 2023 19:58:07 CEST Edgecombe, Rick P wrote: > On Mon, 2023-07-10 at 09:36 +0200, Janusz Krzysztofik wrote: > > The issue needs to be fixed by including _PAGE_PAT bit into a bitmask > > used > > by pgprot_modify() for selecting bits to be preserved

[Intel-gfx] [PATCH v2] drm/i915: Fix premature release of request's reusable memory

2023-07-17 Thread Janusz Krzysztofik
f85d8582e ("drm/i915: Serialise i915_active_fence_set() with itself") Suggested-by: Chris Wilson Signed-off-by: Janusz Krzysztofik Cc: # v5.6+ --- drivers/gpu/drm/i915/i915_active.c | 99 - drivers/gpu/drm/i915/i915_request.c | 11 2 files changed, 81 insertions(+), 29 d

[Intel-gfx] [PATCH v3] drm/i915: Fix premature release of request's reusable memory

2023-07-20 Thread Janusz Krzysztofik
ts release to call_rcu() (Chris) Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/8211 Fixes: df9f85d8582e ("drm/i915: Serialise i915_active_fence_set() with itself") Suggested-by: Chris Wilson Signed-off-by: Janusz Krzysztofik Cc: # v5.6+ ---

Re: [Intel-gfx] [PATCH i-g-t v2 09/11] lib/kunit: Prepare for KTAP parsing after modprobe completed

2023-10-10 Thread Janusz Krzysztofik
Hi Kamil, Thanks for review. On Monday, 9 October 2023 19:37:31 CEST Kamil Konieczny wrote: > Hi Janusz, > On 2023-10-09 at 14:28:00 +0200, Janusz Krzysztofik wrote: > > We are going to add support for reading a list of kunit test cases > > provided by a kunit test module

Re: [Intel-gfx] [PATCH i-g-t v2 04/11] lib/kunit: Parse KTAP report from the main process thread

2023-10-10 Thread Janusz Krzysztofik
Hi Kamil, Thanks for review. On Tuesday, 10 October 2023 17:59:56 CEST Kamil Konieczny wrote: > Hi Janusz, > On 2023-10-09 at 14:27:55 +0200, Janusz Krzysztofik wrote: > > There was an attempt to parse KTAP reports in the background while a kunit > > test module is loadin

Re: [Intel-gfx] [PATCH i-g-t v2 04/11] lib/kunit: Parse KTAP report from the main process thread

2023-10-10 Thread Janusz Krzysztofik
Hi Mauro, Thanks for review. On Tuesday, 10 October 2023 15:33:57 CEST Mauro Carvalho Chehab wrote: > On Mon, 9 Oct 2023 14:27:55 +0200 > Janusz Krzysztofik wrote: > > > There was an attempt to parse KTAP reports in the background while a kunit > > test module is loa

[Intel-gfx] [PATCH i-g-t v3 01/11] lib/ktap: Improve TODO workaround description

2023-10-11 Thread Janusz Krzysztofik
karound in place. Signed-off-by: Janusz Krzysztofik --- lib/igt_ktap.c | 13 ++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/lib/igt_ktap.c b/lib/igt_ktap.c index 5eac102417..3df4d6950d 100644 --- a/lib/igt_ktap.c +++ b/lib/igt_ktap.c @@ -91,9 +91,16 @@ int igt_ktap_p

[Intel-gfx] [PATCH i-g-t v3 00/11] Kunit fixes and improvements

2023-10-11 Thread Janusz Krzysztofik
always pass last result to next dynamic sub-subtest, fetch first result right after loading the kunit test module for execution, - still break the loop of test cases on unexpected return codes from kunit_kmsg_get_result(), - fix typos (Kamil), - update commit descriptions. Janusz Kr

[Intel-gfx] [PATCH i-g-t v3 02/11] lib/kunit: Fix handling of potential errors from F_GETFL

2023-10-11 Thread Janusz Krzysztofik
Function fcntl(..., F_GETFL, ...) that returns file status flags may also return a negative error code. Handle that error instead of blindly using the returned value as flags. Signed-off-by: Janusz Krzysztofik Reviewed-by: Kamil Konieczny --- lib/igt_kmod.c | 4 ++-- 1 file changed, 2

[Intel-gfx] [PATCH i-g-t v3 03/11] lib/kunit: Be more verbose on errors

2023-10-11 Thread Janusz Krzysztofik
Use a more verbose variant of igt_fail() when failing a dynamic sub- subtest on kernel taint. Also, print a debug message on string duplication failure. Signed-off-by: Janusz Krzysztofik Reviewed-by: Kamil Konieczny --- lib/igt_kmod.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions

[Intel-gfx] [PATCH i-g-t v3 04/11] lib/kunit: Fix misplaced igt_kunit() doc

2023-10-11 Thread Janusz Krzysztofik
: Janusz Krzysztofik Reviewed-by: Kamil Konieczny --- lib/igt_kmod.c | 17 - 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c index df0e650d49..426ae5b26f 100644 --- a/lib/igt_kmod.c +++ b/lib/igt_kmod.c @@ -762,15 +762,6 @@ static void

[Intel-gfx] [PATCH i-g-t v3 05/11] lib/kunit: Parse KTAP report from the main process thread

2023-10-11 Thread Janusz Krzysztofik
be failure. Signed-off-by: Janusz Krzysztofik Acked-by: Mauro Carvalho Chehab # v2 --- lib/igt_kmod.c | 264 +++ lib/igt_ktap.c | 569 + lib/igt_ktap.h | 22 -- 3 files changed, 226 insertions(+), 629 deletions(-) diff -

[Intel-gfx] [PATCH i-g-t v3 09/11] lib/kunit: Prepare for KTAP parsing after modprobe completed

2023-10-11 Thread Janusz Krzysztofik
performed after the kunit test module is successfully loaded and there will be no concurrently running modprobe thread, we need to make synchronization of reads from /dev/kmsg with potential errors modprobe thread optional. Signed-off-by: Janusz Krzysztofik Reviewed-by: Kamil Konieczny --- lib

[Intel-gfx] [PATCH i-g-t v3 06/11] lib/kunit: Omit suite name prefix if the same as subtest name

2023-10-11 Thread Janusz Krzysztofik
is into account when composing names for IGT dynamic sub-subtest names and drop the leading test suite name component when it is the same as subtest name. Signed-off-by: Janusz Krzysztofik Reviewed-by: Mauro Carvalho Chehab --- lib/igt_kmod.c | 11 --- 1 file changed, 8 insertions(+),

[Intel-gfx] [PATCH i-g-t v3 08/11] lib/kunit: Provide all results cleanup helper

2023-10-11 Thread Janusz Krzysztofik
Planned changes require a couple of loops around kunit_result_free(). Since we already have such loop, move it into a helper in preparation for future uses. Signed-off-by: Janusz Krzysztofik Reviewed-by: Kamil Konieczny --- lib/igt_kmod.c | 20 ++-- 1 file changed, 14

[Intel-gfx] [PATCH i-g-t v3 07/11] tests/kms_selftest: Let subtest names match suite names

2023-10-11 Thread Janusz Krzysztofik
to kunit and the i915 module potentially providing three test suites: mock, live and perf). [1] https://docs.kernel.org/dev-tools/kunit/style.html Signed-off-by: Janusz Krzysztofik Reviewed-by: Kamil Konieczny --- tests/kms_selftest.c | 37 - 1 file changed

[Intel-gfx] [PATCH i-g-t v3 11/11] lib/kunit: Execute kunit test cases only when needed

2023-10-11 Thread Janusz Krzysztofik
of dynamic sub-subtests, and do that only from the first actually executed dynamic sub-subtest. v2: Always pass last result to next dynamic sub-subtest, fetch first result right after loading the kunit test module for execution. Signed-off-by: Janusz Krzysztofik Reviewed-by: Kamil Konieczny

[Intel-gfx] [PATCH i-g-t v3 10/11] lib/kunit: Fetch a list of test cases in advance

2023-10-11 Thread Janusz Krzysztofik
xpected return codes from kunit_kmsg_get_result(), - use kunit_results_free() helper, - fix typos (Kamil), - update commit description. Signed-off-by: Janusz Krzysztofik Acked-by: Kamil Konieczny # v2 --- lib/igt_kmod.c | 277 ++--- 1 file chang

[Intel-gfx] [PATCH] drm/i915/vma: Fix potential UAF on multi-tile platforms

2023-11-06 Thread Janusz Krzysztofik
lled from atomic contexts, use async variant of wakeref put. Closes: https://gitlab.freedesktop.org/drm/intel/issues/8875 Fixes: Fixes: 213c43676beb ("drm/i915/mtl: Remove the 'force_probe' requirement for Meteor Lake") Signed-off-by: Janusz Krzysztofik --- drivers/gpu/d

Re: [Intel-gfx] [PATCH] drm/i915/vma: Fix potential UAF on multi-tile platforms

2023-11-06 Thread Janusz Krzysztofik
Hi Jani, Thanks for looking at this. On Monday, 6 November 2023 10:53:11 CET Jani Nikula wrote: > On Mon, 06 Nov 2023, Janusz Krzysztofik wrote: > > Object debugging tools were sporadically reporting illegal attempts to > > free a still active i915 VMA object from when par

Re: [Intel-gfx] ✗ Fi.CI.BAT: failure for drm/i915/vma: Fix potential UAF on multi-tile platforms

2023-11-07 Thread Janusz Krzysztofik
On Monday, 6 November 2023 20:16:20 CET Patchwork wrote: > == Series Details == > > Series: drm/i915/vma: Fix potential UAF on multi-tile platforms > URL : https://patchwork.freedesktop.org/series/126012/ > State : failure > > == Summary == > > CI Bug Log - changes from CI_DRM_13844 -> Patchwo

[Intel-gfx] [PATCH v2] drm/i915/vma: Fix potential UAF on multi-tile platforms

2023-11-08 Thread Janusz Krzysztofik
first supported multi-tile platform -- Meteor Lake. v2: Get the wakeref before vm mutex to avoid circular locking dependency, - drop questionable Fixes: tag. Closes: https://gitlab.freedesktop.org/drm/intel/issues/8875 Signed-off-by: Janusz Krzysztofik --- drivers/gpu/drm/i915/i915_vma.c |

Re: [Intel-gfx] ✗ Fi.CI.BAT: failure for drm/i915/vma: Fix potential UAF on multi-tile platforms (rev2)

2023-11-09 Thread Janusz Krzysztofik
On Wednesday, 8 November 2023 22:27:35 CET Patchwork wrote: > == Series Details == > > Series: drm/i915/vma: Fix potential UAF on multi-tile platforms (rev2) > URL : https://patchwork.freedesktop.org/series/126012/ > State : failure > > == Summary == > > CI Bug Log - changes from CI_DRM_13852

Re: [Intel-gfx] ✗ Fi.CI.BAT: failure for drm/i915/vma: Fix potential UAF on multi-tile platforms (rev3)

2023-11-09 Thread Janusz Krzysztofik
On Thursday, 9 November 2023 22:44:02 CET Patchwork wrote: > == Series Details == > > Series: drm/i915/vma: Fix potential UAF on multi-tile platforms (rev3) > URL : https://patchwork.freedesktop.org/series/126012/ > State : failure > > == Summary == > > CI Bug Log - changes from CI_DRM_13856 -

Re: [Intel-gfx] [PATCH] drm/i915/gt: add missing new-line to GT_TRACE

2023-11-15 Thread Janusz Krzysztofik
On Wednesday, 15 November 2023 13:10:33 CET Andrzej Hajda wrote: > Trace requires new-line at the end of message (in opposition to printk), > otherwise trace dump becomes messy. > > Signed-off-by: Andrzej Hajda Acked-by: Janusz Krzysztofik > --- > drivers/gpu/drm/i915/gt

[Intel-gfx] [PATCH v3] drm/i915/vma: Fix VMA UAF on destroy against deactivate race

2023-11-16 Thread Janusz Krzysztofik
ommit to blame, - identify and drop former workarounds, - update commit message and description. v2: Get the wakeref before VM mutex to avoid circular locking dependency, - drop questionable Fixes: tag. Fixes: d93939730347 ("drm/i915: Remove the vma refcount") Closes: https://gitlab.

Re: [Intel-gfx] ✗ Fi.CI.IGT: failure for drm/i915/vma: Fix VMA UAF on destroy against deactivate race (rev2)

2023-11-17 Thread Janusz Krzysztofik
On Friday, 17 November 2023 17:48:03 CET Patchwork wrote: > == Series Details == > > Series: drm/i915/vma: Fix VMA UAF on destroy against deactivate race (rev2) > URL : https://patchwork.freedesktop.org/series/126530/ > State : failure > > == Summary == > > CI Bug Log - changes from CI_DRM_138

[PATCH v4 1/3] drm/i915/vma: Fix UAF on destroy against retire race

2024-01-22 Thread Janusz Krzysztofik
p former workarounds, - update commit message and description. v2: Get the wakeref before VM mutex to avoid circular locking dependency, - drop questionable Fixes: tag. Fixes: d93939730347 ("drm/i915: Remove the vma refcount") Closes: https://gitlab.freedesktop.org/drm/i

[PATCH v4 2/3] Manually revert "drm/i915: Fix a VMA UAF for multi-gt platform"

2024-01-22 Thread Janusz Krzysztofik
This reverts changes introduced by commit f56fe3e91787, obsoleted by "drm/i915/vma: Fix UAF on destroy against retire race". Signed-off-by: Janusz Krzysztofik --- drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c | 17 - 1 file changed, 17 deletions(-) diff --git a/drive

[PATCH v4 3/3] Revert "drm/i915: Wait for active retire before i915_active_fini()"

2024-01-22 Thread Janusz Krzysztofik
This reverts commit 7a2280e8dcd2f1f436db9631287c0b21cf6a92b0, obsoleted by "drm/i915/vma: Fix UAF on destroy against retire race". Signed-off-by: Janusz Krzysztofik --- drivers/gpu/drm/i915/i915_vma.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/gpu/drm/i915/i915_vma.c

Re: [PATCH v4 1/3] drm/i915/vma: Fix UAF on destroy against retire race

2024-01-23 Thread Janusz Krzysztofik
Hi Rodrigo, Thank you for review. On Monday, 22 January 2024 22:09:38 CET Rodrigo Vivi wrote: > On Mon, Jan 22, 2024 at 03:04:42PM +0100, Janusz Krzysztofik wrote: > > Object debugging tools were sporadically reporting illegal attempts to > > free a still active i915 VMA object

[PATCH v5 0/3] drm/i915: Fix VMA UAF on destroy against deactivate race

2024-01-24 Thread Janusz Krzysztofik
of former insufficient workarounds and move them to separate patches (Nirmoy). - clean up commit message and description a bit. v3: Identify root cause more precisely, and a commit to blame, - identify and drop former workarounds, - update commit message and description. v2: Get the wakeref

[PATCH v5 1/3] drm/i915/vma: Fix UAF on destroy against retire race

2024-01-24 Thread Janusz Krzysztofik
dentify root cause more precisely, and a commit to blame, - identify and drop former workarounds, - update commit message and description. v2: Get the wakeref before VM mutex to avoid circular locking dependency, - drop questionable Fixes: tag. Fixes: d93939730347 ("drm/i915: Remov

[PATCH v5 2/3] drm/i915: Remove extra multi-gt pm-references

2024-01-24 Thread Janusz Krzysztofik
the word "revert" in commit message (Rodrigo), - update commit description reusing relevant chunks dropped from the description of the proper fix (Rodrigo). Signed-off-by: Janusz Krzysztofik Cc: Nirmoy Das Cc: Rodrigo Vivi --- drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c

[PATCH v5 3/3] Revert "drm/i915: Wait for active retire before i915_active_fini()"

2024-01-24 Thread Janusz Krzysztofik
This reverts commit 7a2280e8dcd2f1f436db9631287c0b21cf6a92b0, obsoleted by "drm/i915/vma: Fix UAF on destroy against retire race". Signed-off-by: Janusz Krzysztofik Cc: Nirmoy Das --- drivers/gpu/drm/i915/i915_vma.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/gp

Re: [PATCH v5 0/3] drm/i915: Fix VMA UAF on destroy against deactivate race

2024-01-29 Thread Janusz Krzysztofik
o our __active_retire() used in DMA fence callbacks. I'm going to verify some ideas for a fix on trybot. Thanks, Janusz > > > Regards, > > Nirmoy > > On 1/24/2024 6:13 PM, Janusz Krzysztofik wrote: > > Object debugging tools were sporadically reporting illegal attempts

Re: [PATCH] drm/i915/gt: Restart the heartbeat timer when forcing a pulse

2024-01-31 Thread Janusz Krzysztofik
Hi John, On Wednesday, 10 January 2024 22:02:16 CET john.c.harri...@intel.com wrote: > From: John Harrison > > The context persistence code does things like send super high priority > heartbeat pulses to ensure any leaked context can still be pre-empted > and thus isn't a total denial of service

[Intel-gfx] [PATCH i-g-t] tests/core_hotunplug: Reduce debug noise on stdout

2021-01-19 Thread Janusz Krzysztofik
started. In order to let that subtest skip on unsuccessful initial health check, not fail, move the decision whether to fail or skip on error from the health check helper to its users. Signed-off-by: Janusz Krzysztofik --- tests/core_hotunplug.c | 86 +++--- 1

Re: [Intel-gfx] [PATCH i-g-t] tests/core_hotunplug: Reduce debug noise on stdout

2021-01-21 Thread Janusz Krzysztofik
On Tue, 2021-01-19 at 12:08 +0100, Marcin Bernatowicz wrote: > On Tue, 2021-01-19 at 09:42 +0100, Janusz Krzysztofik wrote: > > Since igt_fixture sections are processed unconditionally regardless > > of > > which subtest has been requested, they can now emit a lot of &g

Re: [Intel-gfx] [PATCH] drm/i915/gt: Only unwedge if we can reset first

2019-10-01 Thread Janusz Krzysztofik
that seems to be a safe choice. Thanks, Janusz > Reported-by: Janusz Krzysztofik > Signed-off-by: Chris Wilson > Cc: Janusz Krzysztofik > Cc: Daniele Ceraolo Spurio > Cc: Ville Syrjälä > Reviewed-by: Daniele Ceraolo Spurio #v1 > --- > drivers/gpu/drm/i915/gt/intel_re

[Intel-gfx] [PATCH 0/2] Conclude load -> probe naming convention switch

2019-10-01 Thread Janusz Krzysztofik
ause I forgot to add a cover letter required for successful joint testing when I was submitting to igt-dev list, sorry. Thanks, Janusz Janusz Krzysztofik (2): drm/i915: Fix i915_inject_load_error() name to read *_probe_* drm/i915: Rename "inject_load_failure" module parameter .../gp

[Intel-gfx] [PATCH 1/2] drm/i915: Fix i915_inject_load_error() name to read *_probe_*

2019-10-01 Thread Janusz Krzysztofik
Suggested-by: Michał Wajdeczko Signed-off-by: Janusz Krzysztofik Cc: Michał Wajdeczko Cc: Michał Winiarski Cc: Piotr Piórkowski Cc: Tomasz Lis Cc: Joonas Lahtinen --- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 +- drivers/gpu/drm/i915/gt/uc/intel_huc.c| 4 ++-- drivers/gp

[Intel-gfx] [PATCH 2/2] drm/i915: Rename "inject_load_failure" module parameter

2019-10-01 Thread Janusz Krzysztofik
the "probe" nomenclature. May affect custom user applications utilizing the old name. Suggested-by: Joonas Lahtinen Signed-off-by: Janusz Krzysztofik Cc: Michał Wajdeczko Cc: Michał Winiarski Cc: Piotr Piórkowski Cc: Tomasz Lis Cc: Joonas Lahtinen --- drivers/gpu/drm/i915/i91

Re: [Intel-gfx] [PATCH 2/2] drm/i915: Rename "inject_load_failure" module parameter

2019-10-01 Thread Janusz Krzysztofik
Hi Chris, On Tuesday, October 1, 2019 3:57:27 PM CEST Chris Wilson wrote: > Quoting Janusz Krzysztofik (2019-10-01 14:45:34) > > Commit f2db53f14d3d ("drm/i915: Replace "_load" with "_probe" > > consequently") deliberately left the name of the module

[Intel-gfx] [PATCH v2 1/2] drm/i915: Fix i915_inject_load_error() name to read *_probe_*

2019-10-01 Thread Janusz Krzysztofik
Suggested-by: Michał Wajdeczko Signed-off-by: Janusz Krzysztofik Cc: Michał Wajdeczko Cc: Michał Winiarski Cc: Piotr Piórkowski Cc: Tomasz Lis Cc: Joonas Lahtinen Reviewed-by: Chris Wilson --- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 +- drivers/gpu/drm/i915/gt/uc/intel_huc.c

[Intel-gfx] [PATCH v2 2/2] drm/i915: Rename "inject_load_failure" module parameter

2019-10-01 Thread Janusz Krzysztofik
ch to the "probe" nomenclature. Suggested-by: Joonas Lahtinen Signed-off-by: Janusz Krzysztofik Cc: Michał Wajdeczko Cc: Michał Winiarski Cc: Piotr Piórkowski Cc: Tomasz Lis Cc: Joonas Lahtinen Reviewed-by: Chris Wilson --- drivers/gpu/drm/i915/i915_params.c | 2 +- drivers/gpu/drm/i91

[Intel-gfx] [PATCH v2 0/2] drm/i915: Conclude load -> probe naming convention switch

2019-10-01 Thread Janusz Krzysztofik
essage of 2/2, there are no such (Chris), * add R-b (thanks Chris), * use correct message ID of (also rerolled) IGT counterpart to be tested with. Janusz Krzysztofik (2): drm/i915: Fix i915_inject_load_error() name to read *_probe_* drm/i915: Rename "inject_load_failure"

Re: [RFC PATCH] iommu/vt-d: Fix IOMMU field not populated on device hot re-plug

2019-10-01 Thread Janusz Krzysztofik
Hi Baolu, On Tuesday, September 3, 2019 9:41:23 AM CEST Janusz Krzysztofik wrote: > Hi Baolu, > > On Tuesday, September 3, 2019 3:29:40 AM CEST Lu Baolu wrote: > > Hi Janusz, > > > > On 9/2/19 4:37 PM, Janusz Krzysztofik wrote: > > >> I am not saying that

Re: [Intel-gfx] ✗ Fi.CI.BAT: failure for drm/i915: Conclude load -> probe naming convention switch

2019-10-02 Thread Janusz Krzysztofik
On Tuesday, October 1, 2019 9:32:32 PM CEST Patchwork wrote: > == Series Details == > > Series: drm/i915: Conclude load -> probe naming convention switch > URL : https://patchwork.freedesktop.org/series/67454/ > State : failure > > == Summary == > > CI Bug Log - changes from CI_DRM_6986 -> Pat

Re: [Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for drm/i915: Conclude load -> probe naming convention switch

2019-10-02 Thread Janusz Krzysztofik
On Tuesday, October 1, 2019 9:08:48 PM CEST Patchwork wrote: > == Series Details == > > Series: drm/i915: Conclude load -> probe naming convention switch > URL : https://patchwork.freedesktop.org/series/67454/ > State : warning > > == Summary == > > $ dim checkpatch origin/drm-tip > f23ba0cec2

[Intel-gfx] [RFC PATCH] drm/i915/debugfs: Only wedge if we have reset available

2019-10-02 Thread Janusz Krzysztofik
edged, verify if we have GPU reset available and fail with -EBUSY if not. Suggested-by: Petri Latvala Signed-off-by: Janusz Krzysztofik Cc: Michał Wajdeczko Cc: Michał Winiarski Cc: Piotr Piórkowski Cc: Tomasz Lis Cc: Petri Latvala Cc: Tvrtko Ursulin Cc: Martin Peres --- drivers/gpu/drm

Re: [Intel-gfx] [PATCH] drm/i915/gt: Warn CI about an unrecoverable wedge

2019-10-10 Thread Janusz Krzysztofik
Hi Chris, On Wednesday, October 2, 2019 6:00:34 PM CEST Chris Wilson wrote: > If we have a wedged GPU that we need to recover, but fail, add a taint > for CI to pickup and schedule a reboot. As your approach has been chosen by CI, FWIW: Reviewed-by: Janusz Krzysztofik Thanks,

Re: [RFC PATCH] iommu/vt-d: Fix IOMMU field not populated on device hot re-plug

2019-10-11 Thread Janusz Krzysztofik
Hi Baolu, On Friday, October 11, 2019 8:54:09 AM CEST Lu Baolu wrote: > Hi Janusz, > > On 9/3/19 3:41 PM, Janusz Krzysztofik wrote: > > Hi Baolu, > > > > On Tuesday, September 3, 2019 3:29:40 AM CEST Lu Baolu wrote: > >> Hi Janusz, > >> > >>

Re: [Intel-gfx] [PATCH 2/3] drm/i915: Allow userspace to specify ringsize on construction

2019-11-18 Thread Janusz Krzysztofik
W before the caller > + * is blocked due to lack of space for more commands. > + * > + * Only reliably possible to be set prior to first use, i.e. during > + * construction. At any later point, the current execution must be flushed as > + * the ring can only be changed while the context is idle. > + * > + * Only applies to the current set of engine and lost when those engines > + * are replaced by a new mapping (see I915_CONTEXT_PARAM_ENGINES). > + * > + * Must be between 4 - 512 KiB, in intervals of page size [4 KiB]. > + * Default is 16 KiB. > + */ > +#define I915_CONTEXT_PARAM_RINGSIZE 0xc I know it looked like that already before, but having other documented flags separated by blank lines from each other, Is there any reason for not putting another blank line after the last one? > /* Must be kept compact -- no holes and well documented */ > > __u64 value; > None of the above comments are essential so with or without them addressed: Reviewed-by: Janusz Krzysztofik Thanks, Janusz ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [Intel-gfx] [igt-dev] [PATCH i-g-t] tests/prime_vgem: Skip basic-read/write subtests if no mappable GGTT

2019-11-20 Thread Janusz Krzysztofik
(fixing malformed intel-gfx list address in Cc:) On Wednesday, November 20, 2019 4:36:41 PM CET Chris Wilson wrote: > Quoting Janusz Krzysztofik (2019-11-20 15:32:19) > > As we've agreed that using I915_GEM_PREAD/PWRITE IOCTLs on dma-buf > > objects doesn't make much s

Re: [Intel-gfx] [igt-dev] [PATCH i-g-t] tests/prime_vgem: Skip basic-read/write subtests if no mappable GGTT

2019-11-20 Thread Janusz Krzysztofik
On Wednesday, November 20, 2019 4:53:30 PM CET Chris Wilson wrote: > Quoting Janusz Krzysztofik (2019-11-20 15:49:33) > > (fixing malformed intel-gfx list address in Cc:) > > > > On Wednesday, November 20, 2019 4:36:41 PM CET Chris Wilson wrote: > > > Quoting Janusz

[Intel-gfx] [PATCH i-g-t v2] tests/prime_vgem: Skip basic-read/write subtests if no mappable GGTT

2019-11-20 Thread Janusz Krzysztofik
ure. When running on that kind of hardware, just skip subtests which use those IOCTLs. Signed-off-by: Janusz Krzysztofik Cc: Daniel Vetter Cc: Joonas Lahtinen --- lib/ioctl_wrappers.c | 2 +- lib/ioctl_wrappers.h | 1 + tests/prime_vgem.c | 6 ++ 3 files changed, 8 insertions(+), 1 deletio

Re: [Intel-gfx] [igt-dev] [PATCH i-g-t v2] tests/prime_vgem: Skip basic-read/write subtests if no mappable GGTT

2019-11-21 Thread Janusz Krzysztofik
Hi Stuart, On Wednesday, November 20, 2019 11:04:59 PM CET Summers, Stuart wrote: > On Wed, 2019-11-20 at 17:14 +, Summers, Stuart wrote: > > On Wed, 2019-11-20 at 17:44 +0100, Janusz Krzysztofik wrote: > > > As we've agreed that using I915_GEM_PREAD/PWRITE IOCTLs o

[Intel-gfx] [RESUBMIT PATCH v2] tests/prime_vgem: Skip basic-read/write subtests if no mappable GGTT

2019-11-26 Thread Janusz Krzysztofik
ure. When running on that kind of hardware, just skip subtests which use those IOCTLs. v2: Examine pread/pwrite ABI, not mmap ABI (Chris) Signed-off-by: Janusz Krzysztofik Cc: Daniel Vetter Cc: Joonas Lahtinen Cc: Chris Wilson --- Resubmitting with Joonas' email address in Cc: tag correc

[Intel-gfx] [RFC PATCH i-g-t] tests/prime_vgem: Skip MMAP_GTT based subtests if no mappable GGTT

2019-11-26 Thread Janusz Krzysztofik
As we've agreed that using GEM_MMAP* IOCTLs on dma-buf objects without a CPU accessible GART is very questionable, we are probably not going to extend our MMAP_OFFSET handler with an extra processing path required for it to work correctly on dma-buf objects. As a consequence, on future hardware wi

Re: [Intel-gfx] [RFC PATCH i-g-t] tests/prime_vgem: Skip MMAP_GTT based subtests if no mappable GGTT

2019-11-26 Thread Janusz Krzysztofik
Hi Chris, On Tuesday, November 26, 2019 3:34:13 PM CET Chris Wilson wrote: > Quoting Janusz Krzysztofik (2019-11-26 14:25:48) > > As we've agreed that using GEM_MMAP* IOCTLs on dma-buf objects without > > a CPU accessible GART is very questionable, we are probably not goi

[Intel-gfx] [PATCH i-g-t v3] tests/prime_vgem: Skip basic-read/write subtests if not supported

2019-11-26 Thread Janusz Krzysztofik
ure. When running on that kind of hardware, just skip subtests which use those IOCTLs. v2: Examine pread/pwrite ABI, not mmap ABI (Chris) v3: Don't use "no mappable GGTT" wording in commit message (Chris) Signed-off-by: Janusz Krzysztofik Cc: Daniel Vetter Cc: Joonas Lahtinen

Re: [Intel-gfx] [RFC PATCH i-g-t] tests/prime_vgem: Skip MMAP_GTT based subtests if no mappable GGTT

2019-11-28 Thread Janusz Krzysztofik
Hi Chris, On Tuesday, November 26, 2019 3:34:13 PM CET Chris Wilson wrote: > Quoting Janusz Krzysztofik (2019-11-26 14:25:48) > > As we've agreed that using GEM_MMAP* IOCTLs on dma-buf objects without > > a CPU accessible GART is very questionable, we are probably not goi

[Intel-gfx] [RESEND PATCH i-g-t v3] tests/prime_vgem: Skip basic-read/write subtests if not supported

2019-12-01 Thread Janusz Krzysztofik
ure. When running on that kind of hardware, just skip subtests which use those IOCTLs. v2: Examine pread/pwrite ABI, not mmap ABI (Chris) v3: Don't use "no mappable GGTT" wording in commit message (Chris) Signed-off-by: Janusz Krzysztofik Cc: Daniel Vetter Cc: Joonas Lahtinen

Re: [Intel-gfx] [RESEND PATCH i-g-t v3] tests/prime_vgem: Skip basic-read/write subtests if not supported

2019-12-02 Thread Janusz Krzysztofik
Please ignore, I didn't notice it has been already merged. Thanks, Janusz On Monday, December 2, 2019 8:53:51 AM CET Janusz Krzysztofik wrote: > As we've agreed that using I915_GEM_PREAD/PWRITE IOCTLs on dma-buf > objects doesn't make much sense, we are not going to exten

Re: [Intel-gfx] [igt-dev] [PATCH i-g-t 9/9] i915: Exercise I915_CONTEXT_PARAM_RINGSIZE

2019-12-02 Thread Janusz Krzysztofik
Hi Chris, I have a few questions rather than comments. I hope they are worth spending your time. On Wednesday, November 13, 2019 1:52:40 PM CET Chris Wilson wrote: > I915_CONTEXT_PARAM_RINGSIZE specifies how large to create the command > ringbuffer for logical ring contects. This directly affec

Re: [RFC PATCH] iommu/vt-d: Fix IOMMU field not populated on device hot re-plug

2019-08-28 Thread Janusz Krzysztofik
On Wednesday, August 28, 2019 2:56:18 AM CEST Lu Baolu wrote: > Hi Janusz, > > On 8/27/19 5:35 PM, Janusz Krzysztofik wrote: > > Hi Lu, > > > > On Monday, August 26, 2019 10:29:12 AM CEST Lu Baolu wrote: > >> Hi Janusz, > >> > >> On

Re: [Intel-gfx] [RFC PATCH] iommu/vt-d: Fix IOMMU field not populated on device hot re-plug

2019-08-29 Thread Janusz Krzysztofik
Hi Baolu, On Thursday, August 29, 2019 3:43:31 AM CEST Lu Baolu wrote: > Hi Janusz, > > On 8/28/19 10:17 PM, Janusz Krzysztofik wrote: > >> We should avoid kernel panic when a intel_unmap() is called against > >> a non-existent domain. > > Does that mean you

Re: [Intel-gfx] [RFC PATCH] iommu/vt-d: Fix IOMMU field not populated on device hot re-plug

2019-09-02 Thread Janusz Krzysztofik
Hi Baolu, On Thursday, August 29, 2019 11:08:18 AM CEST Lu Baolu wrote: > Hi, > > On 8/29/19 3:58 PM, Janusz Krzysztofik wrote: > > Hi Baolu, > > > > On Thursday, August 29, 2019 3:43:31 AM CEST Lu Baolu wrote: > >> Hi Janusz, > >> > >> On

Re: [Intel-gfx] [RFC PATCH] iommu/vt-d: Fix IOMMU field not populated on device hot re-plug

2019-09-03 Thread Janusz Krzysztofik
Hi Baolu, On Tuesday, September 3, 2019 3:29:40 AM CEST Lu Baolu wrote: > Hi Janusz, > > On 9/2/19 4:37 PM, Janusz Krzysztofik wrote: > >> I am not saying that keeping data is not acceptable. I just want to > >> check whether there are any other solutions. > >

<    1   2   3   4   5   6   7   8   9   >