Re: [ANNOUNCE] Interbench v0.21
On Sat, 16 Jul 2005 06:41 pm, Lee Revell wrote: > On Fri, 2005-07-15 at 14:01 +1000, Con Kolivas wrote: > > Interbech is a an application is designed to benchmark interactivity in > > Linux. > > > > Version 0.21 update > > > > http://ck.kolivas.org/apps/interbench/interbench-0.21.tar.bz2 > > I would suggest using microseconds for both the RT and non RT tests. It > would allow easier comparison of results. I have a pretty slow machine > and the max result would only be ~44000 usecs. I think the significance of usec values from the non-rt tests makes this an inappropriate thing to do. The variation in results will be greater than usec resolution. > Also, if it's run with -r and sched_setscheduler fails, rather than > saying "you must be root for SCHED_FIFO" the error message should > encourage the user to try a 2.6.12+ kernel and add themselves to the > "audio" or "realtime" group, and to file a feature request if their > distro does not support the new realtime rlimit feature. > > We should encourage more applications to take advantage of, and distros > to support, the non-root RT scheduling available in 2.6.12+. I really > think the kernel is good enough at this point that we could achieve > OSX-like multimedia performance on the desktop if more apps like xmms, > xine, and mplayer were to adopt a multithreaded model with the > time-critical rendering threads running RT. XMMS recently adopted such > a model, but I don't think the audio thread runs SCHED_FIFO yet. These > benchmarks imply that it would be a massive improvement. While I agree with you in principal on getting the rlimit feature working and supported, this benchmark is meant to be run in single user mode for most reproducible and valid results. However, clearly there will be people using it cautiously as a normal user first. I originally did not include the information that you need to be root in v.20 and said in the documentation "need rt privileges" but within about 5 minutes of posting it I had someone not understanding what "unable to get SCHED_FIFO" meant. I guess a more verbose message will be required explaining non-root RT as well. Cheers, Con - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [ANNOUNCE] Interbench v0.21
On Sat, 16 Jul 2005 06:41 pm, Lee Revell wrote: On Fri, 2005-07-15 at 14:01 +1000, Con Kolivas wrote: Interbech is a an application is designed to benchmark interactivity in Linux. Version 0.21 update http://ck.kolivas.org/apps/interbench/interbench-0.21.tar.bz2 I would suggest using microseconds for both the RT and non RT tests. It would allow easier comparison of results. I have a pretty slow machine and the max result would only be ~44000 usecs. I think the significance of usec values from the non-rt tests makes this an inappropriate thing to do. The variation in results will be greater than usec resolution. Also, if it's run with -r and sched_setscheduler fails, rather than saying you must be root for SCHED_FIFO the error message should encourage the user to try a 2.6.12+ kernel and add themselves to the audio or realtime group, and to file a feature request if their distro does not support the new realtime rlimit feature. We should encourage more applications to take advantage of, and distros to support, the non-root RT scheduling available in 2.6.12+. I really think the kernel is good enough at this point that we could achieve OSX-like multimedia performance on the desktop if more apps like xmms, xine, and mplayer were to adopt a multithreaded model with the time-critical rendering threads running RT. XMMS recently adopted such a model, but I don't think the audio thread runs SCHED_FIFO yet. These benchmarks imply that it would be a massive improvement. While I agree with you in principal on getting the rlimit feature working and supported, this benchmark is meant to be run in single user mode for most reproducible and valid results. However, clearly there will be people using it cautiously as a normal user first. I originally did not include the information that you need to be root in v.20 and said in the documentation need rt privileges but within about 5 minutes of posting it I had someone not understanding what unable to get SCHED_FIFO meant. I guess a more verbose message will be required explaining non-root RT as well. Cheers, Con - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [ANNOUNCE] Interbench v0.21
On Fri, 2005-07-15 at 14:01 +1000, Con Kolivas wrote: > Interbech is a an application is designed to benchmark interactivity in Linux. > > Version 0.21 update > > http://ck.kolivas.org/apps/interbench/interbench-0.21.tar.bz2 > I would suggest using microseconds for both the RT and non RT tests. It would allow easier comparison of results. I have a pretty slow machine and the max result would only be ~44000 usecs. Also, if it's run with -r and sched_setscheduler fails, rather than saying "you must be root for SCHED_FIFO" the error message should encourage the user to try a 2.6.12+ kernel and add themselves to the "audio" or "realtime" group, and to file a feature request if their distro does not support the new realtime rlimit feature. We should encourage more applications to take advantage of, and distros to support, the non-root RT scheduling available in 2.6.12+. I really think the kernel is good enough at this point that we could achieve OSX-like multimedia performance on the desktop if more apps like xmms, xine, and mplayer were to adopt a multithreaded model with the time-critical rendering threads running RT. XMMS recently adopted such a model, but I don't think the audio thread runs SCHED_FIFO yet. These benchmarks imply that it would be a massive improvement. Lee - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [ANNOUNCE] Interbench v0.21
On Fri, 2005-07-15 at 14:01 +1000, Con Kolivas wrote: > his makes a large difference to the latencies measured under mem_load > particularly when running real time benchmarks on a RT-PREEMPT kernel Here are some results from my 600MHz C3. In realtime mode, the PREEMPT_RT kernel performs as expected, max latencies are around 55 usecs with a very tight distribution. Based on what we already know about the RT kernel I think this validates the benchmark. So the numbers Con posted showing an interactivity regression from HZ=250 should be taken seriously. Lee Realtime mode: [EMAIL PROTECTED]:~/kernel-source/interbench-0.21$ ./interbench -r -t 5 84648 loops_per_ms read from file interbench.loops_per_ms Using 84648 loops per ms, running every load for 10 seconds Benchmarking kernel 2.6.12-RT-V0.7.51-28 at datestamp 200507160215 --- Benchmarking Audio real time in the presence of loads --- Latency +/- SD (us) Max Latency % Desired CPU % Deadlines Met None 35 +/- 3.85 41 100100 Video36 +/- 4.15 51 100100 X37 +/- 4.39 49 100100 Burn 34 +/- 4.19 50 100100 Write40 +/- 4.9 52 100100 Read 38 +/- 3.36 46 100100 Compile 40 +/- 4.11 54 100100 Memload 41 +/- 4.24 51 100100 --- Benchmarking Video real time in the presence of loads --- Latency +/- SD (us) Max Latency % Desired CPU % Deadlines Met None 29 +/- 4.15 42 100100 X28 +/- 3.52 46 100100 Burn 28 +/- 3.37 41 100100 Write41 +/- 3.25 59 100100 Read 37 +/- 3.07 43 100100 Compile 36 +/- 4.99 54 100100 Memload 38 +/- 3.39 48 100100 Non realtime mode: [EMAIL PROTECTED]:~/kernel-source/interbench-0.21$ ./interbench -t 5 84648 loops_per_ms read from file interbench.loops_per_ms Using 84648 loops per ms, running every load for 10 seconds Benchmarking kernel 2.6.12-RT-V0.7.51-28 at datestamp 200507160237 --- Benchmarking Audio in the presence of loads --- Latency +/- SD (ms) Max Latency % Desired CPU % Deadlines Met None 0.042 +/- 0.0234 0.176 100100 Video 0.121 +/- 0.926 13 100100 X 1.35 +/- 2.9319.4 100100 Burn 0.067 +/- 0.215 3.02 100100 Write 0.763 +/- 2.1816.9 100100 Read 0.263 +/- 1.129.01 100100 Compile 0.216 +/- 1.06 9.2 100100 Memload 0.541 +/- 1.8613.1 100100 --- Benchmarking Video in the presence of loads --- Latency +/- SD (ms) Max Latency % Desired CPU % Deadlines Met None 0.428 +/- 1.07 17 100 97.2 X 4.6 +/- 4.23 50 100 68.3 Burn 0.394 +/- 1.1816.8 100 97.8 Write 1.31 +/- 2.0539.4 100 93 Read 0.462 +/- 0.809 18.2 100 96.8 Compile 0.991 +/- 1.6742.3 100 94.2 Memload 0.558 +/- 0.949 18.9 100 97.2 --- Benchmarking X in the presence of loads --- Latency +/- SD (ms) Max Latency % Desired CPU % Deadlines Met None 0.599 +/- 0.599 24 100 95 Video 13.3 +/- 13.4 93 100 64 Burn 0.519 +/- 0.52 15 100 94 Write 1.91 +/- 1.91 77 100 91 Read 0.449 +/- 0.45 20 100 95 Compile1.01 +/- 1.04 30 100 93 Memload1.26 +/- 1.26 30 100 88 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [ANNOUNCE] Interbench v0.21
On Fri, 2005-07-15 at 14:01 +1000, Con Kolivas wrote: his makes a large difference to the latencies measured under mem_load particularly when running real time benchmarks on a RT-PREEMPT kernel Here are some results from my 600MHz C3. In realtime mode, the PREEMPT_RT kernel performs as expected, max latencies are around 55 usecs with a very tight distribution. Based on what we already know about the RT kernel I think this validates the benchmark. So the numbers Con posted showing an interactivity regression from HZ=250 should be taken seriously. Lee Realtime mode: [EMAIL PROTECTED]:~/kernel-source/interbench-0.21$ ./interbench -r -t 5 84648 loops_per_ms read from file interbench.loops_per_ms Using 84648 loops per ms, running every load for 10 seconds Benchmarking kernel 2.6.12-RT-V0.7.51-28 at datestamp 200507160215 --- Benchmarking Audio real time in the presence of loads --- Latency +/- SD (us) Max Latency % Desired CPU % Deadlines Met None 35 +/- 3.85 41 100100 Video36 +/- 4.15 51 100100 X37 +/- 4.39 49 100100 Burn 34 +/- 4.19 50 100100 Write40 +/- 4.9 52 100100 Read 38 +/- 3.36 46 100100 Compile 40 +/- 4.11 54 100100 Memload 41 +/- 4.24 51 100100 --- Benchmarking Video real time in the presence of loads --- Latency +/- SD (us) Max Latency % Desired CPU % Deadlines Met None 29 +/- 4.15 42 100100 X28 +/- 3.52 46 100100 Burn 28 +/- 3.37 41 100100 Write41 +/- 3.25 59 100100 Read 37 +/- 3.07 43 100100 Compile 36 +/- 4.99 54 100100 Memload 38 +/- 3.39 48 100100 Non realtime mode: [EMAIL PROTECTED]:~/kernel-source/interbench-0.21$ ./interbench -t 5 84648 loops_per_ms read from file interbench.loops_per_ms Using 84648 loops per ms, running every load for 10 seconds Benchmarking kernel 2.6.12-RT-V0.7.51-28 at datestamp 200507160237 --- Benchmarking Audio in the presence of loads --- Latency +/- SD (ms) Max Latency % Desired CPU % Deadlines Met None 0.042 +/- 0.0234 0.176 100100 Video 0.121 +/- 0.926 13 100100 X 1.35 +/- 2.9319.4 100100 Burn 0.067 +/- 0.215 3.02 100100 Write 0.763 +/- 2.1816.9 100100 Read 0.263 +/- 1.129.01 100100 Compile 0.216 +/- 1.06 9.2 100100 Memload 0.541 +/- 1.8613.1 100100 --- Benchmarking Video in the presence of loads --- Latency +/- SD (ms) Max Latency % Desired CPU % Deadlines Met None 0.428 +/- 1.07 17 100 97.2 X 4.6 +/- 4.23 50 100 68.3 Burn 0.394 +/- 1.1816.8 100 97.8 Write 1.31 +/- 2.0539.4 100 93 Read 0.462 +/- 0.809 18.2 100 96.8 Compile 0.991 +/- 1.6742.3 100 94.2 Memload 0.558 +/- 0.949 18.9 100 97.2 --- Benchmarking X in the presence of loads --- Latency +/- SD (ms) Max Latency % Desired CPU % Deadlines Met None 0.599 +/- 0.599 24 100 95 Video 13.3 +/- 13.4 93 100 64 Burn 0.519 +/- 0.52 15 100 94 Write 1.91 +/- 1.91 77 100 91 Read 0.449 +/- 0.45 20 100 95 Compile1.01 +/- 1.04 30 100 93 Memload1.26 +/- 1.26 30 100 88 - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [ANNOUNCE] Interbench v0.21
On Fri, 2005-07-15 at 14:01 +1000, Con Kolivas wrote: Interbech is a an application is designed to benchmark interactivity in Linux. Version 0.21 update http://ck.kolivas.org/apps/interbench/interbench-0.21.tar.bz2 I would suggest using microseconds for both the RT and non RT tests. It would allow easier comparison of results. I have a pretty slow machine and the max result would only be ~44000 usecs. Also, if it's run with -r and sched_setscheduler fails, rather than saying you must be root for SCHED_FIFO the error message should encourage the user to try a 2.6.12+ kernel and add themselves to the audio or realtime group, and to file a feature request if their distro does not support the new realtime rlimit feature. We should encourage more applications to take advantage of, and distros to support, the non-root RT scheduling available in 2.6.12+. I really think the kernel is good enough at this point that we could achieve OSX-like multimedia performance on the desktop if more apps like xmms, xine, and mplayer were to adopt a multithreaded model with the time-critical rendering threads running RT. XMMS recently adopted such a model, but I don't think the audio thread runs SCHED_FIFO yet. These benchmarks imply that it would be a massive improvement. Lee - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [ANNOUNCE] Interbench v0.21
Con Kolivas wrote: {
Version 0.21 update
Changed the design to run the benchmarked and background loads as separate
processes that spawn their own threads instead of everything running as a
thread of the same process.
}
Great!
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
RE: [ANNOUNCE] Interbench v0.21
Con Kolivas wrote: {
Version 0.21 update
Changed the design to run the benchmarked and background loads as separate
processes that spawn their own threads instead of everything running as a
thread of the same process.
}
Great!
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[ANNOUNCE] Interbench v0.21
Interbech is a an application is designed to benchmark interactivity in Linux. Version 0.21 update http://ck.kolivas.org/apps/interbench/interbench-0.21.tar.bz2 Changes: Changed the design to run the benchmarked and background loads as separate processes that spawn their own threads instead of everything running as a thread of the same process. This was suggested to me originally by Ingo Molnar who noticed significant slowdown due to conflict over ->mm->mmap_sem, invalidating the benchmark results when run in real time mode. This makes a large difference to the latencies measured under mem_load particularly when running real time benchmarks on a RT-PREEMPT kernel. Accounting changes to max_latency to only measure the largest latency of a single scheduling frame - this makes max_latency much smaller (and probably more realistic). Often you may see max latency exactly one frame wide now (consistent with one dropped frame) such as 16.7ms on video. Minor cleanups. Cheers, Con pgpIMsbekm6kw.pgp Description: PGP signature
[ANNOUNCE] Interbench v0.21
Interbech is a an application is designed to benchmark interactivity in Linux. Version 0.21 update http://ck.kolivas.org/apps/interbench/interbench-0.21.tar.bz2 Changes: Changed the design to run the benchmarked and background loads as separate processes that spawn their own threads instead of everything running as a thread of the same process. This was suggested to me originally by Ingo Molnar who noticed significant slowdown due to conflict over -mm-mmap_sem, invalidating the benchmark results when run in real time mode. This makes a large difference to the latencies measured under mem_load particularly when running real time benchmarks on a RT-PREEMPT kernel. Accounting changes to max_latency to only measure the largest latency of a single scheduling frame - this makes max_latency much smaller (and probably more realistic). Often you may see max latency exactly one frame wide now (consistent with one dropped frame) such as 16.7ms on video. Minor cleanups. Cheers, Con pgpIMsbekm6kw.pgp Description: PGP signature
