Re: [RFT][patch] Scheduling for HTT and not only
On 04/10/12 21:46, Arnaud Lacombe wrote: On Tue, Apr 10, 2012 at 1:53 PM, Alexander Motin wrote: On 04/10/12 20:18, Alexander Motin wrote: On 04/10/12 19:58, Arnaud Lacombe wrote: 2012/4/9 Alexander Motin: I have strong feeling that while this test may be interesting for profiling, it's own results in first place depend not from how fast scheduler is, but from the pipes capacity and other alike things. Can somebody hint me what except pipe capacity and context switch to unblocked receiver prevents sender from sending all data in batch and then receiver from receiving them all in batch? If different OSes have different policies there, I think results could be incomparable. Let me disagree on your conclusion. If OS A does a task in X seconds, and OS B does the same task in Y seconds, if Y> X, then OS B is just not performing good enough. Internal implementation's difference for the task can not be waived as an excuse for result's comparability. Sure, numbers are always numbers, but the question is what are they showing? Understanding of the test results is even more important for purely synthetic tests like this. Especially when one test run gives 25 seconds, while another gives 50. This test is not completely clear to me and that is what I've told. Small illustration to my point. Simple scheduler tuning affects thread preemption policy and changes this test results in three times: mav@test:/test/hackbench# ./hackbench 30 process 1000 Running with 30*40 (== 1200) tasks. Time: 9.568 mav@test:/test/hackbench# sysctl kern.sched.interact=0 kern.sched.interact: 30 -> 0 mav@test:/test/hackbench# ./hackbench 30 process 1000 Running with 30*40 (== 1200) tasks. Time: 5.163 mav@test:/test/hackbench# sysctl kern.sched.interact=100 kern.sched.interact: 0 -> 100 mav@test:/test/hackbench# ./hackbench 30 process 1000 Running with 30*40 (== 1200) tasks. Time: 3.190 I think it affects balance between pipe latency and bandwidth, while test measures only the last. It is clear that conclusion from these numbers depends on what do we want to have. I don't really care on this point, I'm only testing default values, or more precisely, whatever developers though default values would be good. Btw, you are testing 3 differents configuration. Different results are expected. What worries me more is the rather the huge instability on the *same* configuration, say on a pipe/thread/70 groups/600 iterations run, where results range from 2.7s[0] to 7.4s, or a socket/thread/20 groups/1400 iterations run, where results range from 2.4s to 4.5s. Due to reason I've pointed in my first message this test is _extremely_ sensitive to context switch interval. The more aggressive scheduler switches threads, the smaller will be pipe latency, but the smaller will be also bandwidth. During test run scheduler all the time recalculates interactivity index for each thread, trying to balance between latency and switching overhead. With hundreds of threads running simultaneously and interfering with each other it is quite unpredictable process. -- Alexander Motin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [RFT][patch] Scheduling for HTT and not only
Hi, On Tue, Apr 10, 2012 at 1:53 PM, Alexander Motin wrote: > On 04/10/12 20:18, Alexander Motin wrote: >> >> On 04/10/12 19:58, Arnaud Lacombe wrote: >>> >>> 2012/4/9 Alexander Motin: [...] I have strong feeling that while this test may be interesting for profiling, it's own results in first place depend not from how fast scheduler is, but from the pipes capacity and other alike things. Can somebody hint me what except pipe capacity and context switch to unblocked receiver prevents sender from sending all data in batch and then receiver from receiving them all in batch? If different OSes have different policies there, I think results could be incomparable. >>> Let me disagree on your conclusion. If OS A does a task in X seconds, >>> and OS B does the same task in Y seconds, if Y> X, then OS B is just >>> not performing good enough. Internal implementation's difference for >>> the task can not be waived as an excuse for result's comparability. >> >> >> Sure, numbers are always numbers, but the question is what are they >> showing? Understanding of the test results is even more important for >> purely synthetic tests like this. Especially when one test run gives 25 >> seconds, while another gives 50. This test is not completely clear to me >> and that is what I've told. > > Small illustration to my point. Simple scheduler tuning affects thread > preemption policy and changes this test results in three times: > > mav@test:/test/hackbench# ./hackbench 30 process 1000 > Running with 30*40 (== 1200) tasks. > Time: 9.568 > > mav@test:/test/hackbench# sysctl kern.sched.interact=0 > kern.sched.interact: 30 -> 0 > mav@test:/test/hackbench# ./hackbench 30 process 1000 > Running with 30*40 (== 1200) tasks. > Time: 5.163 > > mav@test:/test/hackbench# sysctl kern.sched.interact=100 > kern.sched.interact: 0 -> 100 > mav@test:/test/hackbench# ./hackbench 30 process 1000 > Running with 30*40 (== 1200) tasks. > Time: 3.190 > > I think it affects balance between pipe latency and bandwidth, while test > measures only the last. It is clear that conclusion from these numbers > depends on what do we want to have. > I don't really care on this point, I'm only testing default values, or more precisely, whatever developers though default values would be good. Btw, you are testing 3 differents configuration. Different results are expected. What worries me more is the rather the huge instability on the *same* configuration, say on a pipe/thread/70 groups/600 iterations run, where results range from 2.7s[0] to 7.4s, or a socket/thread/20 groups/1400 iterations run, where results range from 2.4s to 4.5s. - Arnaud [0]: numbers extracted from a recent run of 9.0-RELEASE on a Xeon E5-1650 platform. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [RFT][patch] Scheduling for HTT and not only
On 04/10/12 20:18, Alexander Motin wrote: On 04/10/12 19:58, Arnaud Lacombe wrote: 2012/4/9 Alexander Motin: [...] I have strong feeling that while this test may be interesting for profiling, it's own results in first place depend not from how fast scheduler is, but from the pipes capacity and other alike things. Can somebody hint me what except pipe capacity and context switch to unblocked receiver prevents sender from sending all data in batch and then receiver from receiving them all in batch? If different OSes have different policies there, I think results could be incomparable. Let me disagree on your conclusion. If OS A does a task in X seconds, and OS B does the same task in Y seconds, if Y> X, then OS B is just not performing good enough. Internal implementation's difference for the task can not be waived as an excuse for result's comparability. Sure, numbers are always numbers, but the question is what are they showing? Understanding of the test results is even more important for purely synthetic tests like this. Especially when one test run gives 25 seconds, while another gives 50. This test is not completely clear to me and that is what I've told. Small illustration to my point. Simple scheduler tuning affects thread preemption policy and changes this test results in three times: mav@test:/test/hackbench# ./hackbench 30 process 1000 Running with 30*40 (== 1200) tasks. Time: 9.568 mav@test:/test/hackbench# sysctl kern.sched.interact=0 kern.sched.interact: 30 -> 0 mav@test:/test/hackbench# ./hackbench 30 process 1000 Running with 30*40 (== 1200) tasks. Time: 5.163 mav@test:/test/hackbench# sysctl kern.sched.interact=100 kern.sched.interact: 0 -> 100 mav@test:/test/hackbench# ./hackbench 30 process 1000 Running with 30*40 (== 1200) tasks. Time: 3.190 I think it affects balance between pipe latency and bandwidth, while test measures only the last. It is clear that conclusion from these numbers depends on what do we want to have. -- Alexander Motin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [RFT][patch] Scheduling for HTT and not only
On 04/10/12 19:58, Arnaud Lacombe wrote: 2012/4/9 Alexander Motin: [...] I have strong feeling that while this test may be interesting for profiling, it's own results in first place depend not from how fast scheduler is, but from the pipes capacity and other alike things. Can somebody hint me what except pipe capacity and context switch to unblocked receiver prevents sender from sending all data in batch and then receiver from receiving them all in batch? If different OSes have different policies there, I think results could be incomparable. Let me disagree on your conclusion. If OS A does a task in X seconds, and OS B does the same task in Y seconds, if Y> X, then OS B is just not performing good enough. Internal implementation's difference for the task can not be waived as an excuse for result's comparability. Sure, numbers are always numbers, but the question is what are they showing? Understanding of the test results is even more important for purely synthetic tests like this. Especially when one test run gives 25 seconds, while another gives 50. This test is not completely clear to me and that is what I've told. -- Alexander Motin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [RFT][patch] Scheduling for HTT and not only
Hi, 2012/4/9 Alexander Motin : > [...] > > I have strong feeling that while this test may be interesting for profiling, > it's own results in first place depend not from how fast scheduler is, but > from the pipes capacity and other alike things. Can somebody hint me what > except pipe capacity and context switch to unblocked receiver prevents > sender from sending all data in batch and then receiver from receiving them > all in batch? If different OSes have different policies there, I think > results could be incomparable. > Let me disagree on your conclusion. If OS A does a task in X seconds, and OS B does the same task in Y seconds, if Y > X, then OS B is just not performing good enough. Internal implementation's difference for the task can not be waived as an excuse for result's comparability. - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [RFT][patch] Scheduling for HTT and not only
On 04/05/12 21:45, Alexander Motin wrote: On 05.04.2012 21:12, Arnaud Lacombe wrote: Hi, [Sorry for the delay, I got a bit sidetrack'ed...] 2012/2/17 Alexander Motin: On 17.02.2012 18:53, Arnaud Lacombe wrote: On Fri, Feb 17, 2012 at 11:29 AM, Alexander Motin wrote: On 02/15/12 21:54, Jeff Roberson wrote: On Wed, 15 Feb 2012, Alexander Motin wrote: I've decided to stop those cache black magic practices and focus on things that really exist in this world -- SMT and CPU load. I've dropped most of cache related things from the patch and made the rest of things more strict and predictable: http://people.freebsd.org/~mav/sched.htt34.patch This looks great. I think there is value in considering the other approach further but I would like to do this part first. It would be nice to also add priority as a greater influence in the load balancing as well. I haven't got good idea yet about balancing priorities, but I've rewritten balancer itself. As soon as sched_lowest() / sched_highest() are more intelligent now, they allowed to remove topology traversing from the balancer itself. That should fix double-swapping problem, allow to keep some affinity while moving threads and make balancing more fair. I did number of tests running 4, 8, 9 and 16 CPU-bound threads on 8 CPUs. With 4, 8 and 16 threads everything is stationary as it should. With 9 threads I see regular and random load move between all 8 CPUs. Measurements on 5 minutes run show deviation of only about 5 seconds. It is the same deviation as I see caused by only scheduling of 16 threads on 8 cores without any balancing needed at all. So I believe this code works as it should. Here is the patch: http://people.freebsd.org/~mav/sched.htt40.patch I plan this to be a final patch of this series (more to come :)) and if there will be no problems or objections, I am going to commit it (except some debugging KTRs) in about ten days. So now it's a good time for reviews and testing. :) is there a place where all the patches are available ? All my scheduler patches are cumulative, so all you need is only the last mentioned here sched.htt40.patch. You may want to have a look to the result I collected in the `runs/freebsd-experiments' branch of: https://github.com/lacombar/hackbench/ and compare them with vanilla FreeBSD 9.0 and -CURRENT results available in `runs/freebsd'. On the dual package platform, your patch is not a definite win. But in some cases, especially for multi-socket systems, to let it show its best, you may want to apply additional patch from avg@ to better detect CPU topology: https://gitorious.org/~avg/freebsd/avgbsd/commit/6bca4a2e4854ea3fc275946a023db65c483cb9dd test I conducted specifically for this patch did not showed much improvement... If I understand right, this test runs thousands of threads sending and receiving data over the pipes. It is quite likely that all CPUs will be always busy and so load balancing is not really important in this test, What looks good is that more complicated new code is not slower then old one. While this test seems very scheduler-intensive, it may depend on many other factors, such as syscall performance, context switch, etc. I'll try to play more with it. My profiling on 8-core Core i7 system shows that code from sched_ule.c staying on first places consumes still only 13% of kernel CPU time, while doing million of context switches per second. cpu_search(), affected by this patch, even less -- only 8%. The rest of time is spread between many small other functions. I did some optimizations at r234066 to reduce cpu_search(0 time to 6%, but looking on how unstable results of this test are, hardly any difference there can be really measured by it. I have strong feeling that while this test may be interesting for profiling, it's own results in first place depend not from how fast scheduler is, but from the pipes capacity and other alike things. Can somebody hint me what except pipe capacity and context switch to unblocked receiver prevents sender from sending all data in batch and then receiver from receiving them all in batch? If different OSes have different policies there, I think results could be incomparable. -- Alexander Motin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [RFT][patch] Scheduling for HTT and not only
On 04/06/12 17:30, Attilio Rao wrote: Il 06 aprile 2012 15:27, Alexander Motin ha scritto: On 04/06/12 17:13, Attilio Rao wrote: Il 05 aprile 2012 19:12, Arnaud Lacombeha scritto: Hi, [Sorry for the delay, I got a bit sidetrack'ed...] 2012/2/17 Alexander Motin: On 17.02.2012 18:53, Arnaud Lacombe wrote: On Fri, Feb 17, 2012 at 11:29 AM, Alexander Motin wrote: On 02/15/12 21:54, Jeff Roberson wrote: On Wed, 15 Feb 2012, Alexander Motin wrote: I've decided to stop those cache black magic practices and focus on things that really exist in this world -- SMT and CPU load. I've dropped most of cache related things from the patch and made the rest of things more strict and predictable: http://people.freebsd.org/~mav/sched.htt34.patch This looks great. I think there is value in considering the other approach further but I would like to do this part first. It would be nice to also add priority as a greater influence in the load balancing as well. I haven't got good idea yet about balancing priorities, but I've rewritten balancer itself. As soon as sched_lowest() / sched_highest() are more intelligent now, they allowed to remove topology traversing from the balancer itself. That should fix double-swapping problem, allow to keep some affinity while moving threads and make balancing more fair. I did number of tests running 4, 8, 9 and 16 CPU-bound threads on 8 CPUs. With 4, 8 and 16 threads everything is stationary as it should. With 9 threads I see regular and random load move between all 8 CPUs. Measurements on 5 minutes run show deviation of only about 5 seconds. It is the same deviation as I see caused by only scheduling of 16 threads on 8 cores without any balancing needed at all. So I believe this code works as it should. Here is the patch: http://people.freebsd.org/~mav/sched.htt40.patch I plan this to be a final patch of this series (more to come :)) and if there will be no problems or objections, I am going to commit it (except some debugging KTRs) in about ten days. So now it's a good time for reviews and testing. :) is there a place where all the patches are available ? All my scheduler patches are cumulative, so all you need is only the last mentioned here sched.htt40.patch. You may want to have a look to the result I collected in the `runs/freebsd-experiments' branch of: https://github.com/lacombar/hackbench/ and compare them with vanilla FreeBSD 9.0 and -CURRENT results available in `runs/freebsd'. On the dual package platform, your patch is not a definite win. But in some cases, especially for multi-socket systems, to let it show its best, you may want to apply additional patch from avg@ to better detect CPU topology: https://gitorious.org/~avg/freebsd/avgbsd/commit/6bca4a2e4854ea3fc275946a023db65c483cb9dd test I conducted specifically for this patch did not showed much improvement... Can you please clarify on this point? The test you did included cases where the topology was detected badly against cases where the topology was detected correctly as a patched kernel (and you still didn't see a performance improvement), in terms of cache line sharing? At this moment SCHED_ULE does almost nothing in terms of cache line sharing affinity (though it probably worth some further experiments). What this patch may improve is opposite case -- reduce cache sharing pressure for cache-hungry applications. For example, proper cache topology detection (such as lack of global L3 cache, but shared L2 per pairs of cores on Core2Quad class CPUs) increases pbzip2 performance when number of threads is less then number of CPUs (i.e. when there is place for optimization). My asking is not referred to your patch really. I just wanted to know if he correctly benchmark a case where the topology was screwed up and then correctly recognized by avg's patch in terms of cache level aggregation (it wasn't referred to your patch btw). I understand. I've just described test case when properly detected topology could give benefit. What the test really does is indeed a good question. -- Alexander Motin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [RFT][patch] Scheduling for HTT and not only
Il 06 aprile 2012 15:27, Alexander Motin ha scritto: > On 04/06/12 17:13, Attilio Rao wrote: >> >> Il 05 aprile 2012 19:12, Arnaud Lacombe ha scritto: >>> >>> Hi, >>> >>> [Sorry for the delay, I got a bit sidetrack'ed...] >>> >>> 2012/2/17 Alexander Motin: >>>> >>>> On 17.02.2012 18:53, Arnaud Lacombe wrote: >>>>> >>>>> >>>>> On Fri, Feb 17, 2012 at 11:29 AM, Alexander Motin >>>>> wrote: >>>>>> >>>>>> >>>>>> On 02/15/12 21:54, Jeff Roberson wrote: >>>>>>> >>>>>>> >>>>>>> On Wed, 15 Feb 2012, Alexander Motin wrote: >>>>>>>> >>>>>>>> >>>>>>>> I've decided to stop those cache black magic practices and focus on >>>>>>>> things that really exist in this world -- SMT and CPU load. I've >>>>>>>> dropped most of cache related things from the patch and made the >>>>>>>> rest >>>>>>>> of things more strict and predictable: >>>>>>>> http://people.freebsd.org/~mav/sched.htt34.patch >>>>>>> >>>>>>> >>>>>>> >>>>>>> This looks great. I think there is value in considering the other >>>>>>> approach further but I would like to do this part first. It would be >>>>>>> nice to also add priority as a greater influence in the load >>>>>>> balancing >>>>>>> as well. >>>>>> >>>>>> >>>>>> >>>>>> I haven't got good idea yet about balancing priorities, but I've >>>>>> rewritten >>>>>> balancer itself. As soon as sched_lowest() / sched_highest() are more >>>>>> intelligent now, they allowed to remove topology traversing from the >>>>>> balancer itself. That should fix double-swapping problem, allow to >>>>>> keep >>>>>> some >>>>>> affinity while moving threads and make balancing more fair. I did >>>>>> number >>>>>> of >>>>>> tests running 4, 8, 9 and 16 CPU-bound threads on 8 CPUs. With 4, 8 >>>>>> and >>>>>> 16 >>>>>> threads everything is stationary as it should. With 9 threads I see >>>>>> regular >>>>>> and random load move between all 8 CPUs. Measurements on 5 minutes run >>>>>> show >>>>>> deviation of only about 5 seconds. It is the same deviation as I see >>>>>> caused >>>>>> by only scheduling of 16 threads on 8 cores without any balancing >>>>>> needed >>>>>> at >>>>>> all. So I believe this code works as it should. >>>>>> >>>>>> Here is the patch: http://people.freebsd.org/~mav/sched.htt40.patch >>>>>> >>>>>> I plan this to be a final patch of this series (more to come :)) and >>>>>> if >>>>>> there will be no problems or objections, I am going to commit it >>>>>> (except >>>>>> some debugging KTRs) in about ten days. So now it's a good time for >>>>>> reviews >>>>>> and testing. :) >>>>>> >>>>> is there a place where all the patches are available ? >>>> >>>> >>>> >>>> All my scheduler patches are cumulative, so all you need is only the >>>> last >>>> mentioned here sched.htt40.patch. >>>> >>> You may want to have a look to the result I collected in the >>> `runs/freebsd-experiments' branch of: >>> >>> https://github.com/lacombar/hackbench/ >>> >>> and compare them with vanilla FreeBSD 9.0 and -CURRENT results >>> available in `runs/freebsd'. On the dual package platform, your patch >>> is not a definite win. >>> >>>> But in some cases, especially for multi-socket systems, to let it show >>>> its >>>> best, you may want to apply additional patch from avg@ to better detect >>>> CPU >>>> topology: >>>> >>>> https://gitorious.org/~avg/freebsd/avgbsd/commit/6bca4a2e4854ea3fc275946a023db65c483cb9dd >>>> >>> test I conducted specifically for this patch did not showed much >>> improvement... >> >> >> Can you please clarify on this point? >> The test you did included cases where the topology was detected badly >> against cases where the topology was detected correctly as a patched >> kernel (and you still didn't see a performance improvement), in terms >> of cache line sharing? > > > At this moment SCHED_ULE does almost nothing in terms of cache line sharing > affinity (though it probably worth some further experiments). What this > patch may improve is opposite case -- reduce cache sharing pressure for > cache-hungry applications. For example, proper cache topology detection > (such as lack of global L3 cache, but shared L2 per pairs of cores on > Core2Quad class CPUs) increases pbzip2 performance when number of threads is > less then number of CPUs (i.e. when there is place for optimization). My asking is not referred to your patch really. I just wanted to know if he correctly benchmark a case where the topology was screwed up and then correctly recognized by avg's patch in terms of cache level aggregation (it wasn't referred to your patch btw). Attilio -- Peace can only be achieved by understanding - A. Einstein ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [RFT][patch] Scheduling for HTT and not only
On 04/06/12 17:13, Attilio Rao wrote: Il 05 aprile 2012 19:12, Arnaud Lacombe ha scritto: Hi, [Sorry for the delay, I got a bit sidetrack'ed...] 2012/2/17 Alexander Motin: On 17.02.2012 18:53, Arnaud Lacombe wrote: On Fri, Feb 17, 2012 at 11:29 AM, Alexander Motinwrote: On 02/15/12 21:54, Jeff Roberson wrote: On Wed, 15 Feb 2012, Alexander Motin wrote: I've decided to stop those cache black magic practices and focus on things that really exist in this world -- SMT and CPU load. I've dropped most of cache related things from the patch and made the rest of things more strict and predictable: http://people.freebsd.org/~mav/sched.htt34.patch This looks great. I think there is value in considering the other approach further but I would like to do this part first. It would be nice to also add priority as a greater influence in the load balancing as well. I haven't got good idea yet about balancing priorities, but I've rewritten balancer itself. As soon as sched_lowest() / sched_highest() are more intelligent now, they allowed to remove topology traversing from the balancer itself. That should fix double-swapping problem, allow to keep some affinity while moving threads and make balancing more fair. I did number of tests running 4, 8, 9 and 16 CPU-bound threads on 8 CPUs. With 4, 8 and 16 threads everything is stationary as it should. With 9 threads I see regular and random load move between all 8 CPUs. Measurements on 5 minutes run show deviation of only about 5 seconds. It is the same deviation as I see caused by only scheduling of 16 threads on 8 cores without any balancing needed at all. So I believe this code works as it should. Here is the patch: http://people.freebsd.org/~mav/sched.htt40.patch I plan this to be a final patch of this series (more to come :)) and if there will be no problems or objections, I am going to commit it (except some debugging KTRs) in about ten days. So now it's a good time for reviews and testing. :) is there a place where all the patches are available ? All my scheduler patches are cumulative, so all you need is only the last mentioned here sched.htt40.patch. You may want to have a look to the result I collected in the `runs/freebsd-experiments' branch of: https://github.com/lacombar/hackbench/ and compare them with vanilla FreeBSD 9.0 and -CURRENT results available in `runs/freebsd'. On the dual package platform, your patch is not a definite win. But in some cases, especially for multi-socket systems, to let it show its best, you may want to apply additional patch from avg@ to better detect CPU topology: https://gitorious.org/~avg/freebsd/avgbsd/commit/6bca4a2e4854ea3fc275946a023db65c483cb9dd test I conducted specifically for this patch did not showed much improvement... Can you please clarify on this point? The test you did included cases where the topology was detected badly against cases where the topology was detected correctly as a patched kernel (and you still didn't see a performance improvement), in terms of cache line sharing? At this moment SCHED_ULE does almost nothing in terms of cache line sharing affinity (though it probably worth some further experiments). What this patch may improve is opposite case -- reduce cache sharing pressure for cache-hungry applications. For example, proper cache topology detection (such as lack of global L3 cache, but shared L2 per pairs of cores on Core2Quad class CPUs) increases pbzip2 performance when number of threads is less then number of CPUs (i.e. when there is place for optimization). -- Alexander Motin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [RFT][patch] Scheduling for HTT and not only
Il 05 aprile 2012 19:12, Arnaud Lacombe ha scritto: > Hi, > > [Sorry for the delay, I got a bit sidetrack'ed...] > > 2012/2/17 Alexander Motin : >> On 17.02.2012 18:53, Arnaud Lacombe wrote: >>> >>> On Fri, Feb 17, 2012 at 11:29 AM, Alexander Motin wrote: >>>> >>>> On 02/15/12 21:54, Jeff Roberson wrote: >>>>> >>>>> On Wed, 15 Feb 2012, Alexander Motin wrote: >>>>>> >>>>>> I've decided to stop those cache black magic practices and focus on >>>>>> things that really exist in this world -- SMT and CPU load. I've >>>>>> dropped most of cache related things from the patch and made the rest >>>>>> of things more strict and predictable: >>>>>> http://people.freebsd.org/~mav/sched.htt34.patch >>>>> >>>>> >>>>> This looks great. I think there is value in considering the other >>>>> approach further but I would like to do this part first. It would be >>>>> nice to also add priority as a greater influence in the load balancing >>>>> as well. >>>> >>>> >>>> I haven't got good idea yet about balancing priorities, but I've >>>> rewritten >>>> balancer itself. As soon as sched_lowest() / sched_highest() are more >>>> intelligent now, they allowed to remove topology traversing from the >>>> balancer itself. That should fix double-swapping problem, allow to keep >>>> some >>>> affinity while moving threads and make balancing more fair. I did number >>>> of >>>> tests running 4, 8, 9 and 16 CPU-bound threads on 8 CPUs. With 4, 8 and >>>> 16 >>>> threads everything is stationary as it should. With 9 threads I see >>>> regular >>>> and random load move between all 8 CPUs. Measurements on 5 minutes run >>>> show >>>> deviation of only about 5 seconds. It is the same deviation as I see >>>> caused >>>> by only scheduling of 16 threads on 8 cores without any balancing needed >>>> at >>>> all. So I believe this code works as it should. >>>> >>>> Here is the patch: http://people.freebsd.org/~mav/sched.htt40.patch >>>> >>>> I plan this to be a final patch of this series (more to come :)) and if >>>> there will be no problems or objections, I am going to commit it (except >>>> some debugging KTRs) in about ten days. So now it's a good time for >>>> reviews >>>> and testing. :) >>>> >>> is there a place where all the patches are available ? >> >> >> All my scheduler patches are cumulative, so all you need is only the last >> mentioned here sched.htt40.patch. >> > You may want to have a look to the result I collected in the > `runs/freebsd-experiments' branch of: > > https://github.com/lacombar/hackbench/ > > and compare them with vanilla FreeBSD 9.0 and -CURRENT results > available in `runs/freebsd'. On the dual package platform, your patch > is not a definite win. > >> But in some cases, especially for multi-socket systems, to let it show its >> best, you may want to apply additional patch from avg@ to better detect CPU >> topology: >> https://gitorious.org/~avg/freebsd/avgbsd/commit/6bca4a2e4854ea3fc275946a023db65c483cb9dd >> > test I conducted specifically for this patch did not showed much > improvement... Can you please clarify on this point? The test you did included cases where the topology was detected badly against cases where the topology was detected correctly as a patched kernel (and you still didn't see a performance improvement), in terms of cache line sharing? Attilio -- Peace can only be achieved by understanding - A. Einstein ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [RFT][patch] Scheduling for HTT and not only
On 05.04.2012 21:12, Arnaud Lacombe wrote: Hi, [Sorry for the delay, I got a bit sidetrack'ed...] 2012/2/17 Alexander Motin: On 17.02.2012 18:53, Arnaud Lacombe wrote: On Fri, Feb 17, 2012 at 11:29 AM, Alexander Motinwrote: On 02/15/12 21:54, Jeff Roberson wrote: On Wed, 15 Feb 2012, Alexander Motin wrote: I've decided to stop those cache black magic practices and focus on things that really exist in this world -- SMT and CPU load. I've dropped most of cache related things from the patch and made the rest of things more strict and predictable: http://people.freebsd.org/~mav/sched.htt34.patch This looks great. I think there is value in considering the other approach further but I would like to do this part first. It would be nice to also add priority as a greater influence in the load balancing as well. I haven't got good idea yet about balancing priorities, but I've rewritten balancer itself. As soon as sched_lowest() / sched_highest() are more intelligent now, they allowed to remove topology traversing from the balancer itself. That should fix double-swapping problem, allow to keep some affinity while moving threads and make balancing more fair. I did number of tests running 4, 8, 9 and 16 CPU-bound threads on 8 CPUs. With 4, 8 and 16 threads everything is stationary as it should. With 9 threads I see regular and random load move between all 8 CPUs. Measurements on 5 minutes run show deviation of only about 5 seconds. It is the same deviation as I see caused by only scheduling of 16 threads on 8 cores without any balancing needed at all. So I believe this code works as it should. Here is the patch: http://people.freebsd.org/~mav/sched.htt40.patch I plan this to be a final patch of this series (more to come :)) and if there will be no problems or objections, I am going to commit it (except some debugging KTRs) in about ten days. So now it's a good time for reviews and testing. :) is there a place where all the patches are available ? All my scheduler patches are cumulative, so all you need is only the last mentioned here sched.htt40.patch. You may want to have a look to the result I collected in the `runs/freebsd-experiments' branch of: https://github.com/lacombar/hackbench/ and compare them with vanilla FreeBSD 9.0 and -CURRENT results available in `runs/freebsd'. On the dual package platform, your patch is not a definite win. But in some cases, especially for multi-socket systems, to let it show its best, you may want to apply additional patch from avg@ to better detect CPU topology: https://gitorious.org/~avg/freebsd/avgbsd/commit/6bca4a2e4854ea3fc275946a023db65c483cb9dd test I conducted specifically for this patch did not showed much improvement... If I understand right, this test runs thousands of threads sending and receiving data over the pipes. It is quite likely that all CPUs will be always busy and so load balancing is not really important in this test, What looks good is that more complicated new code is not slower then old one. While this test seems very scheduler-intensive, it may depend on many other factors, such as syscall performance, context switch, etc. I'll try to play more with it. -- Alexander Motin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [RFT][patch] Scheduling for HTT and not only
Hi, [Sorry for the delay, I got a bit sidetrack'ed...] 2012/2/17 Alexander Motin : > On 17.02.2012 18:53, Arnaud Lacombe wrote: >> >> On Fri, Feb 17, 2012 at 11:29 AM, Alexander Motin wrote: >>> >>> On 02/15/12 21:54, Jeff Roberson wrote: >>>> >>>> On Wed, 15 Feb 2012, Alexander Motin wrote: >>>>> >>>>> I've decided to stop those cache black magic practices and focus on >>>>> things that really exist in this world -- SMT and CPU load. I've >>>>> dropped most of cache related things from the patch and made the rest >>>>> of things more strict and predictable: >>>>> http://people.freebsd.org/~mav/sched.htt34.patch >>>> >>>> >>>> This looks great. I think there is value in considering the other >>>> approach further but I would like to do this part first. It would be >>>> nice to also add priority as a greater influence in the load balancing >>>> as well. >>> >>> >>> I haven't got good idea yet about balancing priorities, but I've >>> rewritten >>> balancer itself. As soon as sched_lowest() / sched_highest() are more >>> intelligent now, they allowed to remove topology traversing from the >>> balancer itself. That should fix double-swapping problem, allow to keep >>> some >>> affinity while moving threads and make balancing more fair. I did number >>> of >>> tests running 4, 8, 9 and 16 CPU-bound threads on 8 CPUs. With 4, 8 and >>> 16 >>> threads everything is stationary as it should. With 9 threads I see >>> regular >>> and random load move between all 8 CPUs. Measurements on 5 minutes run >>> show >>> deviation of only about 5 seconds. It is the same deviation as I see >>> caused >>> by only scheduling of 16 threads on 8 cores without any balancing needed >>> at >>> all. So I believe this code works as it should. >>> >>> Here is the patch: http://people.freebsd.org/~mav/sched.htt40.patch >>> >>> I plan this to be a final patch of this series (more to come :)) and if >>> there will be no problems or objections, I am going to commit it (except >>> some debugging KTRs) in about ten days. So now it's a good time for >>> reviews >>> and testing. :) >>> >> is there a place where all the patches are available ? > > > All my scheduler patches are cumulative, so all you need is only the last > mentioned here sched.htt40.patch. > You may want to have a look to the result I collected in the `runs/freebsd-experiments' branch of: https://github.com/lacombar/hackbench/ and compare them with vanilla FreeBSD 9.0 and -CURRENT results available in `runs/freebsd'. On the dual package platform, your patch is not a definite win. > But in some cases, especially for multi-socket systems, to let it show its > best, you may want to apply additional patch from avg@ to better detect CPU > topology: > https://gitorious.org/~avg/freebsd/avgbsd/commit/6bca4a2e4854ea3fc275946a023db65c483cb9dd > test I conducted specifically for this patch did not showed much improvement... - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [RFT][patch] Scheduling for HTT and not only
Hi, On Fri, Feb 17, 2012 at 11:29 AM, Alexander Motin wrote: > On 02/15/12 21:54, Jeff Roberson wrote: >> >> On Wed, 15 Feb 2012, Alexander Motin wrote: >>> >>> I've decided to stop those cache black magic practices and focus on >>> things that really exist in this world -- SMT and CPU load. I've >>> dropped most of cache related things from the patch and made the rest >>> of things more strict and predictable: >>> http://people.freebsd.org/~mav/sched.htt34.patch >> >> >> This looks great. I think there is value in considering the other >> approach further but I would like to do this part first. It would be >> nice to also add priority as a greater influence in the load balancing >> as well. > > > I haven't got good idea yet about balancing priorities, but I've rewritten > balancer itself. As soon as sched_lowest() / sched_highest() are more > intelligent now, they allowed to remove topology traversing from the > balancer itself. That should fix double-swapping problem, allow to keep some > affinity while moving threads and make balancing more fair. I did number of > tests running 4, 8, 9 and 16 CPU-bound threads on 8 CPUs. With 4, 8 and 16 > threads everything is stationary as it should. With 9 threads I see regular > and random load move between all 8 CPUs. Measurements on 5 minutes run show > deviation of only about 5 seconds. It is the same deviation as I see caused > by only scheduling of 16 threads on 8 cores without any balancing needed at > all. So I believe this code works as it should. > > Here is the patch: http://people.freebsd.org/~mav/sched.htt40.patch > > I plan this to be a final patch of this series (more to come :)) and if > there will be no problems or objections, I am going to commit it (except > some debugging KTRs) in about ten days. So now it's a good time for reviews > and testing. :) > is there a place where all the patches are available ? I intend to run some tests on a 1x2x2 (atom D510), 1x4x1 (core-2 quad), and eventually a 2x8x2 platforms, against r231573. Results should hopefully be available by the end of the week-end/middle of next week[0]. - Arnaud [0]: the D510 will likely be testing a couple of Linux kernel over the week-end, and a FreeBSD run takes about 2.5 days to complete. > > -- > Alexander Motin > ___ > freebsd-hack...@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [RFT][patch] Scheduling for HTT and not only
On 17.02.2012 18:53, Arnaud Lacombe wrote: On Fri, Feb 17, 2012 at 11:29 AM, Alexander Motin wrote: On 02/15/12 21:54, Jeff Roberson wrote: On Wed, 15 Feb 2012, Alexander Motin wrote: I've decided to stop those cache black magic practices and focus on things that really exist in this world -- SMT and CPU load. I've dropped most of cache related things from the patch and made the rest of things more strict and predictable: http://people.freebsd.org/~mav/sched.htt34.patch This looks great. I think there is value in considering the other approach further but I would like to do this part first. It would be nice to also add priority as a greater influence in the load balancing as well. I haven't got good idea yet about balancing priorities, but I've rewritten balancer itself. As soon as sched_lowest() / sched_highest() are more intelligent now, they allowed to remove topology traversing from the balancer itself. That should fix double-swapping problem, allow to keep some affinity while moving threads and make balancing more fair. I did number of tests running 4, 8, 9 and 16 CPU-bound threads on 8 CPUs. With 4, 8 and 16 threads everything is stationary as it should. With 9 threads I see regular and random load move between all 8 CPUs. Measurements on 5 minutes run show deviation of only about 5 seconds. It is the same deviation as I see caused by only scheduling of 16 threads on 8 cores without any balancing needed at all. So I believe this code works as it should. Here is the patch: http://people.freebsd.org/~mav/sched.htt40.patch I plan this to be a final patch of this series (more to come :)) and if there will be no problems or objections, I am going to commit it (except some debugging KTRs) in about ten days. So now it's a good time for reviews and testing. :) is there a place where all the patches are available ? All my scheduler patches are cumulative, so all you need is only the last mentioned here sched.htt40.patch. But in some cases, especially for multi-socket systems, to let it show its best, you may want to apply additional patch from avg@ to better detect CPU topology: https://gitorious.org/~avg/freebsd/avgbsd/commit/6bca4a2e4854ea3fc275946a023db65c483cb9dd I intend to run some tests on a 1x2x2 (atom D510), 1x4x1 (core-2 quad), and eventually a 2x8x2 platforms, against r231573. Results should hopefully be available by the end of the week-end/middle of next week[0]. [0]: the D510 will likely be testing a couple of Linux kernel over the week-end, and a FreeBSD run takes about 2.5 days to complete. -- Alexander Motin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [RFT][patch] Scheduling for HTT and not only
On 02/15/12 21:54, Jeff Roberson wrote: On Wed, 15 Feb 2012, Alexander Motin wrote: I've decided to stop those cache black magic practices and focus on things that really exist in this world -- SMT and CPU load. I've dropped most of cache related things from the patch and made the rest of things more strict and predictable: http://people.freebsd.org/~mav/sched.htt34.patch This looks great. I think there is value in considering the other approach further but I would like to do this part first. It would be nice to also add priority as a greater influence in the load balancing as well. I haven't got good idea yet about balancing priorities, but I've rewritten balancer itself. As soon as sched_lowest() / sched_highest() are more intelligent now, they allowed to remove topology traversing from the balancer itself. That should fix double-swapping problem, allow to keep some affinity while moving threads and make balancing more fair. I did number of tests running 4, 8, 9 and 16 CPU-bound threads on 8 CPUs. With 4, 8 and 16 threads everything is stationary as it should. With 9 threads I see regular and random load move between all 8 CPUs. Measurements on 5 minutes run show deviation of only about 5 seconds. It is the same deviation as I see caused by only scheduling of 16 threads on 8 cores without any balancing needed at all. So I believe this code works as it should. Here is the patch: http://people.freebsd.org/~mav/sched.htt40.patch I plan this to be a final patch of this series (more to come :)) and if there will be no problems or objections, I am going to commit it (except some debugging KTRs) in about ten days. So now it's a good time for reviews and testing. :) -- Alexander Motin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: HEADSUP: Realtime thread scheduling changed
2011/1/14 John Baldwin : > Note that as a result of these > changes, rtprio threads will no longer share priorities with interactive > timeshare threads. Instead, rtprio threads are now always more important than > non-rt threads. Great! I was thinking about the split of timesharing threads and realtime threads for years, and now you've actually implemented that! Alexander Churanov ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: HEADSUP: Realtime thread scheduling changed
On Friday, January 14, 2011 12:22:18 pm Daniel Eischen wrote: > On Fri, 14 Jan 2011, John Baldwin wrote: > > > This is just a heads up that I've committed some changes to how the > > scheduler > > handles realtime thread priorities. Please let me know of any issues you > > encounter with nice, rtprio, or idprio. Note that as a result of these > > changes, rtprio threads will no longer share priorities with interactive > > timeshare threads. Instead, rtprio threads are now always more important > > than > > non-rt threads. > > Cool - thanks for doing this! Is this something that could > be MFC'able to 8? That's a harder question. This changes the effective value of the P* priority constants passed to *sleep(). That would be an ABI change for kernel modules. We could either 1) decide that it is an ABI change worth making (probably doubtful since it is mostly a new feature rather than a major bug fix) or 2) maybe MFC it but make the different priority ranges be subject to some global kernel config option. 2) isn't super ideal since you have to really make sure kernel modules are compiled with the same setting for that option to avoid weird behavior. I will MFC all the other changes I've made prior to this in which case this change would be the only local patch someone would need to have this. -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: HEADSUP: Realtime thread scheduling changed
On Fri, 14 Jan 2011, John Baldwin wrote: This is just a heads up that I've committed some changes to how the scheduler handles realtime thread priorities. Please let me know of any issues you encounter with nice, rtprio, or idprio. Note that as a result of these changes, rtprio threads will no longer share priorities with interactive timeshare threads. Instead, rtprio threads are now always more important than non-rt threads. Cool - thanks for doing this! Is this something that could be MFC'able to 8? -- DE ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
HEADSUP: Realtime thread scheduling changed
This is just a heads up that I've committed some changes to how the scheduler
handles realtime thread priorities. Please let me know of any issues you
encounter with nice, rtprio, or idprio. Note that as a result of these
changes, rtprio threads will no longer share priorities with interactive
timeshare threads. Instead, rtprio threads are now always more important than
non-rt threads.
--
John Baldwin
--- Begin Message ---
Author: jhb
Date: Fri Jan 14 17:06:54 2011
New Revision: 217410
URL: http://svn.freebsd.org/changeset/base/217410
Log:
Rework realtime priority support:
- Move the realtime priority range up above kernel sleep priorities and
just below interrupt thread priorities.
- Contract the interrupt and kernel sleep priority ranges a bit so that
the timesharing priority band can be increased. The new timeshare range
is now slightly larger than the old realtime + timeshare ranges.
- Change the ULE scheduler to no longer use realtime priorities for
interactive threads. Instead, the larger timeshare range is now split
into separate subranges for interactive and non-interactive ("batch")
threads. The end result is that interactive threads and non-interactive
threads still use the same priority ranges as before, but realtime
threads now have a separate, dedicated priority range.
- Do not modify the priority of non-timeshare threads in sched_sleep()
or via cv_broadcastpri(). Realtime and idle priority threads will
no longer have their priorities affected by sleeping in the kernel.
Reviewed by: jeff
Modified:
head/sys/kern/sched_4bsd.c
head/sys/kern/sched_ule.c
head/sys/kern/subr_sleepqueue.c
head/sys/sys/priority.h
Modified: head/sys/kern/sched_4bsd.c
==
--- head/sys/kern/sched_4bsd.c Fri Jan 14 16:42:13 2011(r217409)
+++ head/sys/kern/sched_4bsd.c Fri Jan 14 17:06:54 2011(r217410)
@@ -908,7 +908,7 @@ sched_sleep(struct thread *td, int pri)
THREAD_LOCK_ASSERT(td, MA_OWNED);
td->td_slptick = ticks;
td->td_sched->ts_slptime = 0;
- if (pri)
+ if (pri != 0 && PRI_BASE(td->td_pri_class) == PRI_TIMESHARE)
sched_prio(td, pri);
if (TD_IS_SUSPENDED(td) || pri >= PSOCK)
td->td_flags |= TDF_CANSWAP;
Modified: head/sys/kern/sched_ule.c
==
--- head/sys/kern/sched_ule.c Fri Jan 14 16:42:13 2011(r217409)
+++ head/sys/kern/sched_ule.c Fri Jan 14 17:06:54 2011(r217410)
@@ -118,11 +118,17 @@ static struct td_sched td_sched0;
/*
* Priority ranges used for interactive and non-interactive timeshare
- * threads. Interactive threads use realtime priorities.
- */
-#definePRI_MIN_INTERACTPRI_MIN_REALTIME
-#definePRI_MAX_INTERACTPRI_MAX_REALTIME
-#definePRI_MIN_BATCH PRI_MIN_TIMESHARE
+ * threads. The timeshare priorities are split up into four ranges.
+ * The first range handles interactive threads. The last three ranges
+ * (NHALF, x, and NHALF) handle non-interactive threads with the outer
+ * ranges supporting nice values.
+ */
+#definePRI_TIMESHARE_RANGE (PRI_MAX_TIMESHARE - PRI_MIN_TIMESHARE
+ 1)
+#definePRI_INTERACT_RANGE ((PRI_TIMESHARE_RANGE -
SCHED_PRI_NRESV) / 2)
+
+#definePRI_MIN_INTERACTPRI_MIN_TIMESHARE
+#definePRI_MAX_INTERACT(PRI_MIN_TIMESHARE + PRI_INTERACT_RANGE
- 1)
+#definePRI_MIN_BATCH (PRI_MIN_TIMESHARE + PRI_INTERACT_RANGE)
#definePRI_MAX_BATCH PRI_MAX_TIMESHARE
/*
@@ -1893,6 +1899,8 @@ sched_sleep(struct thread *td, int prio)
td->td_slptick = ticks;
if (TD_IS_SUSPENDED(td) || prio >= PSOCK)
td->td_flags |= TDF_CANSWAP;
+ if (PRI_BASE(td->td_pri_class) != PRI_TIMESHARE)
+ return;
if (static_boost == 1 && prio)
sched_prio(td, prio);
else if (static_boost && td->td_priority > static_boost)
Modified: head/sys/kern/subr_sleepqueue.c
==
--- head/sys/kern/subr_sleepqueue.c Fri Jan 14 16:42:13 2011
(r217409)
+++ head/sys/kern/subr_sleepqueue.c Fri Jan 14 17:06:54 2011
(r217410)
@@ -745,7 +745,8 @@ sleepq_resume_thread(struct sleepqueue *
/* Adjust priority if requested. */
MPASS(pri == 0 || (pri >= PRI_MIN && pri <= PRI_MAX));
- if (pri != 0 && td->td_priority > pri)
+ if (pri != 0 && td->td_priority > pri &&
+ PRI_BASE(td->td_pri_class) == PRI_TIMESHARE)
sched_prio(td, pri);
/*
Modified: head/sys/sys/priority.h
==
--- head/sys/sys/priority.h Fri Jan 14 16:42:13 2011(r217409)
+++ head
Re: TTY task group scheduling
On Fri, Nov 19, 2010 at 5:27 PM, Oliver Pinter wrote: > My desktop running 7-STABLE with 100Hz and NOPREEMPT (it's a 4core SMP > system), > I tested 8-STABLE, but that is not too responsive, the solution is: > 100Hz NOPREEMPT + kern.sched.preempt_thresh=224 > After this setting, the system is likely responsive as 7-STABLE. > > On 11/19/10, Garrett Cooper wrote: >> On Fri, Nov 19, 2010 at 1:10 PM, Oliver Pinter >> wrote: >>> http://lkml.org/lkml/2010/11/16/392 >>> >>> On 11/18/10, O. Hartmann wrote: On 11/18/10 02:30, grarpamp wrote: > Just documenting regarding interactive performance things. > This one's from Linux. > > http://www.phoronix.com/scan.php?page=article&item=linux_2637_video&num=1 Well, it would be nice to have those improvements in FreeBSD, but I doubt this will make it in due time to FreeBSD's kernel. >> >> And my one line fix: >> >> renice 10 `pidof firefox-bin` >> >> Instantly my system is snappier (and in fact my system got really >> laggy after applying the preempt sysctl that everyone recommended >> before)... Performance issue with firefox maybe :P? I don't see the >> point of adding an additional layer to complicate the system (and >> essentially slow it down) if all you're trying to do is better >> describe the nice'ing problem, unless this logic is what you want to >> do strictly for desktop users in PCBSD, etc who may not have the >> technical wherewithal to accomplish this task. >> >> Besides, the Linux kernel has different compile time profiles for >> different workloads, so maybe it just works better for them because >> they already have a means for describing that functionality, whereas >> FreeBSD is more generic. >> >> It would be nice to describe this in a document though so people could >> also decide how to tune the system for themselves and not deal with a >> patch like what's noted above by the penguin crowd because it will >> invariably fail under some workloads or conditions (I have yet to see >> a one-size-fits-all solution in this area). >> >> >> SCHED_ULE improvements though should be looked into if possible, >> because there are some potential items that could be done to cluster >> processes together better, maybe. >> Ugh. Looks like this was the only machine that I setup recently where I didn't set kern.hz... Ok, will retry after building and installing a whole new world *queue lame Disney music here* and kernel. Thanks for the obvious reminder... -Garrett ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
My desktop running 7-STABLE with 100Hz and NOPREEMPT (it's a 4core SMP system), I tested 8-STABLE, but that is not too responsive, the solution is: 100Hz NOPREEMPT + kern.sched.preempt_thresh=224 After this setting, the system is likely responsive as 7-STABLE. On 11/19/10, Garrett Cooper wrote: > On Fri, Nov 19, 2010 at 1:10 PM, Oliver Pinter > wrote: >> http://lkml.org/lkml/2010/11/16/392 >> >> On 11/18/10, O. Hartmann wrote: >>> On 11/18/10 02:30, grarpamp wrote: Just documenting regarding interactive performance things. This one's from Linux. http://www.phoronix.com/scan.php?page=article&item=linux_2637_video&num=1 >>> >>> Well, >>> it would be nice to have those improvements in FreeBSD, but I doubt this >>> will make it in due time to FreeBSD's kernel. > > And my one line fix: > > renice 10 `pidof firefox-bin` > > Instantly my system is snappier (and in fact my system got really > laggy after applying the preempt sysctl that everyone recommended > before)... Performance issue with firefox maybe :P? I don't see the > point of adding an additional layer to complicate the system (and > essentially slow it down) if all you're trying to do is better > describe the nice'ing problem, unless this logic is what you want to > do strictly for desktop users in PCBSD, etc who may not have the > technical wherewithal to accomplish this task. > > Besides, the Linux kernel has different compile time profiles for > different workloads, so maybe it just works better for them because > they already have a means for describing that functionality, whereas > FreeBSD is more generic. > > It would be nice to describe this in a document though so people could > also decide how to tune the system for themselves and not deal with a > patch like what's noted above by the penguin crowd because it will > invariably fail under some workloads or conditions (I have yet to see > a one-size-fits-all solution in this area). > > > SCHED_ULE improvements though should be looked into if possible, > because there are some potential items that could be done to cluster > processes together better, maybe. > > > Thanks, > -Garrett > ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On 11/19/10 16:49, Taku YAMAMOTO wrote: I have a dumb local hack to grant ts_slice proportional to the duration the waking thread slept rather than unconditionally reset to sched_slice. --- sys/kern/sched_ule.c.orig +++ sys/kern/sched_ule.c @@ -1928,12 +1928,16 @@ sched_wakeup(struct thread *td) u_int hzticks; hzticks = (ticks - slptick)<< SCHED_TICK_SHIFT; + if (hzticks> SCHED_SLP_RUN_MAX) + hzticks = SCHED_SLP_RUN_MAX; ts->ts_slptime += hzticks; + /* Grant additional slices after we sleep. */ + ts->ts_slice += hzticks / tickincr; + if (ts->ts_slice > sched_slice) + ts->ts_slice = sched_slice; If I read it correctly, now instead of the slice given to the thread being always sched_slice, now it is reduced to a value smaller than sched_slice based on how long did the thread sleep? How does that help? sched_interact_update(td); sched_pctcpu_update(ts); } - /* Reset the slice value after we sleep. */ - ts->ts_slice = sched_slice; sched_add(td, SRQ_BORING); } ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
In the last episode (Nov 19), Alexander Leidinger said: > Quoting Alexander Best (from Fri, 19 Nov 2010 00:17:10 > +): > > 17:51 @ Genesys : Luigi Rizzo had a plugabble scheduler back in 4.* or > > thereabouts > > 17:51 @ Genesys : you could kldload new ones and switch to them on the fly > > 17:52 @ arundel : wow. that sounds cool. too bad it didn't make it > > into src tree. by now it's probably outdated and needs to be reworked quite > > a bit. > > > > > > does anybody know something about this? > > I'm aware of the I/O scheduling code (which is now available at least > in -current), but I do not remember CPU scheduling code from Luigi. > Are you sure Genesys didn't mix up something by accident? I am rarely mixed up :) A quick search didn't bring up a direct reference, but here's a mention of it from Luigi: http://lists.freebsd.org/pipermail/freebsd-hackers/2004-November/008891.html -- Dan Nelson dnel...@allantgroup.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On Fri, Nov 19, 2010 at 1:10 PM, Oliver Pinter wrote: > http://lkml.org/lkml/2010/11/16/392 > > On 11/18/10, O. Hartmann wrote: >> On 11/18/10 02:30, grarpamp wrote: >>> Just documenting regarding interactive performance things. >>> This one's from Linux. >>> >>> http://www.phoronix.com/scan.php?page=article&item=linux_2637_video&num=1 >> >> Well, >> it would be nice to have those improvements in FreeBSD, but I doubt this >> will make it in due time to FreeBSD's kernel. And my one line fix: renice 10 `pidof firefox-bin` Instantly my system is snappier (and in fact my system got really laggy after applying the preempt sysctl that everyone recommended before)... Performance issue with firefox maybe :P? I don't see the point of adding an additional layer to complicate the system (and essentially slow it down) if all you're trying to do is better describe the nice'ing problem, unless this logic is what you want to do strictly for desktop users in PCBSD, etc who may not have the technical wherewithal to accomplish this task. Besides, the Linux kernel has different compile time profiles for different workloads, so maybe it just works better for them because they already have a means for describing that functionality, whereas FreeBSD is more generic. It would be nice to describe this in a document though so people could also decide how to tune the system for themselves and not deal with a patch like what's noted above by the penguin crowd because it will invariably fail under some workloads or conditions (I have yet to see a one-size-fits-all solution in this area). SCHED_ULE improvements though should be looked into if possible, because there are some potential items that could be done to cluster processes together better, maybe. Thanks, -Garrett ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
http://lkml.org/lkml/2010/11/16/392 On 11/18/10, O. Hartmann wrote: > On 11/18/10 02:30, grarpamp wrote: >> Just documenting regarding interactive performance things. >> This one's from Linux. >> >> http://www.phoronix.com/scan.php?page=article&item=linux_2637_video&num=1 > > Well, > it would be nice to have those improvements in FreeBSD, but I doubt this > will make it in due time to FreeBSD's kernel. > ___ > freebsd-sta...@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org" > ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On Thu, 18 Nov 2010 21:30:16 +0100 "O. Hartmann" wrote: > On 11/18/10 19:55, Lucius Windschuh wrote: > > 2010/11/18 Andriy Gapon: > >> [Grouping of processes into TTY groups] > >> > >> Well, I think that those improvements apply only to a very specific usage > >> pattern > >> and are greatly over-hyped. > > > > But there are serious issue if you use FreeBSD as a desktop OS with > > SMP and SCHED_ULE, or? > > Because currently, my machine is barely usable if a compile job with > > parallelism is running. Movies stutter, Firefox hangs. And even nice > > -n 20 doesn't do the job in every case, as +20 seems not to be the > > idle priority anymore?!? > > And using "idprio 1 $cmd" as a workaround is, well, a kludge. > > I am not sure if TTY grouping is the right solution, if you look at > > potentially CPU-intensive GUI applications that all run on the same > > TTY (or no TTY at all? Same problem). > > Maybe, we could simply enhance the algorithm that decides if a task is > > interactive? That would also improve the described situation. > > > > Regards, > > > > Lucius > > Stuttering Response, being stuck for over 20 seconds also happens when I > start updating the OS' sources via svn. This happens on all boxes, some > of them do have 8 cores (ob two CPUs) and plenty of RAM. Heavy disk I/O, > doesn't matter on UFS2 or ZFS, also brings boxes to stutter, those > phenomena are most seen when you interact with the machine via X11 > clients. I think it's hard to realize if a server only does console I/O, > but console also seems to be stuck sometimes. It would be worth checking > this with some 'benchmark'. X11 in its kind of oldish incarnation on > FreeBSD seems to contribute most to those slowdowns, what so ever. I guess schedulers can hardly distinguish heavy disk I/Os from nanosleep()s and user-interactions; schedulers think both as voluntary sleep. To make the matters worse, the current implementation of SCHED_ULE reassigns ts_slice on sched_wakeup() no matter how short the sleep was. I have a dumb local hack to grant ts_slice proportional to the duration the waking thread slept rather than unconditionally reset to sched_slice. --- sys/kern/sched_ule.c.orig +++ sys/kern/sched_ule.c @@ -1928,12 +1928,16 @@ sched_wakeup(struct thread *td) u_int hzticks; hzticks = (ticks - slptick) << SCHED_TICK_SHIFT; + if (hzticks > SCHED_SLP_RUN_MAX) + hzticks = SCHED_SLP_RUN_MAX; ts->ts_slptime += hzticks; + /* Grant additional slices after we sleep. */ + ts->ts_slice += hzticks / tickincr; + if (ts->ts_slice > sched_slice) + ts->ts_slice = sched_slice; sched_interact_update(td); sched_pctcpu_update(ts); } - /* Reset the slice value after we sleep. */ - ts->ts_slice = sched_slice; sched_add(td, SRQ_BORING); } -- -|-__ YAMAMOTO, Taku | __ < - A chicken is an egg's way of producing more eggs. - ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On Fri, Nov 19, 2010 at 02:18:52PM +, Vincent Hoffman wrote: > On 19/11/2010 12:42, Eric Masson wrote: > > Bruce Cran writes: > > > > Hello, > > > >> Google suggests that the work was a GSoC project in 2005 on a pluggable > >> disk scheduler. > > It seems that something similar has found its way in DFlyBSD, dsched. > And indeed to FreeBSD, man gsched. Added sometime round April > http://svn.freebsd.org/viewvc/base/head/sys/geom/sched/README?view=log It's been pointed out on the list a couple times, and I've sent mail to the authors about this, that gsched breaks (very, very badly) things like sysinstall, and does other strange things like leaves trailing periods at the end of its ".sched." labels. This appears to be by design, but I'm still left thinking "?!" It's hard to discern technical innards/workings of GEOM since the documentation is so poor (and reading the code doesn't help, especially with regards to libgeom). IMHO, the gsched "stuff", as a "layer", should probably be moved into the I/O framework by default, with the functionality *disabled* by default and tunables to adjust it. That's just how I feel about it. -- | Jeremy Chadwick j...@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On 19/11/2010 12:42, Eric Masson wrote: > Bruce Cran writes: > > Hello, > >> Google suggests that the work was a GSoC project in 2005 on a pluggable >> disk scheduler. > It seems that something similar has found its way in DFlyBSD, dsched. And indeed to FreeBSD, man gsched. Added sometime round April http://svn.freebsd.org/viewvc/base/head/sys/geom/sched/README?view=log Vince > Éric Masson > ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
Bruce Cran writes: Hello, > Google suggests that the work was a GSoC project in 2005 on a pluggable > disk scheduler. It seems that something similar has found its way in DFlyBSD, dsched. Éric Masson -- manquerait plus que les groupes soient pollués. c'est beaucoup plus grave que des plages bretonnes à deux francs qui étaient déjà polluées par ces salopards de volatiles. dieu merci, il n'y en aura bientôt plus -+- tilt in http://www.le-gnu.net : Les oiseaux sont des cons. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
on 18/11/2010 22:20 Julian Elischer said the following: > tty grouping is a variant of what we used to have at one stage which is > a "kernel schedulable entity group".. KSEG Or rather, I think, a concrete application of a variant of that. > the idea is that all items in a group share some characteristic and some > amount > of resources. > > We stripped the KSEG out of the picture because it really complicated the > picture. Yes, unfortunately. One can think about a number of applications for hierarchical schedulable resources. Even one-level group scheduling could be a very useful subcase. BTW, http://www.mjmwired.net/kernel/Documentation/cgroups.txt -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
Quoting Alexander Best (from Fri, 19 Nov 2010 00:17:10 +): 17:51 @ Genesys : Luigi Rizzo had a plugabble scheduler back in 4.* or \ thereabouts 17:51 @ Genesys : you could kldload new ones and switch to them on the fly 17:52 @ arundel : wow. that sounds cool. too bad it didn't make it into src \ tree. by now it's probably outdated and needs to be reworked quite a bit. does anybody know something about this? I'm aware of the I/O scheduling code (which is now available at least in -current), but I do not remember CPU scheduling code from Luigi. Are you sure Genesys didn't mix up something by accident? Bye, Alexander. -- Ferengi Rule of Acquisition #123: Even a blind man can recognize the glow of latinum. -- ST: Legends of the Ferengi http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On Fri, 19 Nov 2010 00:17:10 + Alexander Best wrote: > 17:51 @ Genesys : Luigi Rizzo had a plugabble scheduler back in 4.* > or \ thereabouts > 17:51 @ Genesys : you could kldload new ones and switch to them on > the fly 17:52 @ arundel : wow. that sounds cool. too bad it didn't > make it into src \ tree. by now it's probably outdated and needs to > be reworked quite a bit. > > does anybody know something about this? Google suggests that the work was a GSoC project in 2005 on a pluggable disk scheduler. -- Bruce Cran ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
on 19/11/2010 00:55 Daniel Nebdal said the following: > On Fri, Nov 19, 2010 at 12:06 AM, Alexander Kabaev wrote: >> On Thu, 18 Nov 2010 18:56:35 + >> Alexander Best wrote: >>> well...i tried playing back a 1080p vide files while doing >>> `make -j64 buildkernel` and FreeBSD's interactivity seems far from >>> perfect. >> >> One thing that just begs to be asked: since when decoding 1080p became >> an interactive task? >> > > Strictly speaking it isn't - but displaying it is a timing-sensitive > task that isn't CPU- or I/O-bound, and scheduling-wise that probably Well, I am not sure if I can agree about CPU-bound-ness. Depends on the exact video file, of course, but certain high-quality 1080p are very CPU intensive unless decoding is offloaded from the CPU. Depends on decoder code too. I had some videos that were CPU-bound on my Athlon II X2 250 with then-version of mplayer from ports. YMMV. -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
on 18/11/2010 20:56 Alexander Best said the following: > On Thu Nov 18 10, Matthew D. Fuller wrote: >> On Thu, Nov 18, 2010 at 06:23:24PM + I heard the voice of >> Alexander Best, and lo! it spake thus: >>> >>> judging from the videos the changes are having a huge impact imo. >> >> Well, my (admittedly limited, and certainly anecdotal) experience is >> that Linux's interactive response when under heavy load was always >> much worse than FreeBSD's. So maybe that's just them catching up to >> where we already are ;) > > well...i tried playing back a 1080p vide files while doing > `make -j64 buildkernel` and FreeBSD's interactivity seems far from perfect. > > it might be possible that linux'es interactivity was worse than freebsd's, > but still this patch should be evaluated for freebsd. in this particular case > it seems linux now does better than freebsd. You do realize that there are many more variables for such a test than just "1080p video" and "make -j64 buildkernel"? Let's not forget about hardware, video drivers, player capabilities, exact kind of the video (you know, 1080p alone doesn't tell much). Besides, someone might be interested in running -j64 on his 1,2,4-core desktop system, but it's definitely not me. I prefer to be reasonable. I am not saying that our scheduler (ULE) is perfect. I don't even say that it's better (in whatever comparison system) than Linux scheduler X. I say that I wouldn't spend my time improving system behavior in a scenario like this. I compile stuff very frequently (kernels, ports, world) while browsing web, reading email, doing various "desktop stuff", sometimes watching videos from the web (like e.g. trailers). On this machine/hardware I have never personally felt a need for improvements in the scheduler. And I run KDE4 with all bells and whistles enabled. YMMV. P.S. I don't discourage anyone from improving our scheduler, I even do encourage that. -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On Thu, Nov 18, 2010 at 06:23:24PM +, Alexander Best wrote: > you think so? judging from the videos the changes are having a huge impact > imo. On Linux. Have you ever seen those sorts of UI problems on FreeBSD? I don't watch much video on my systems, but I haven't seen that. FreeBSD has always been good at keeping user-interactive processes responsive while compiles or what-not are going on in the background. Cheers, -- Andrew ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On Nov 18, 2010, at 18:43, Julian Elischer wrote: we are part of the way there.. at least we did abstract the scheduler to the point where we have two completely different ones. you are welcome to develop a 'framework as you describe and plug it into the abstraction we already have. It may be something to suggest for the next round of Google's Summer of Code. Or perhaps part of a school project in operating systems work (master's level? eager-beaver bachelor's thesis?). Having a bit more flexibility in being able to make different components "pluggable" would help encourage the use of BSD in more research projects. And more people learning and hacking on BSD can't be a bad thing. :) ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On Thu Nov 18 10, Julian Elischer wrote: > On 11/18/10 3:37 PM, Alexander Best wrote: > >On Fri Nov 19 10, Daniel Nebdal wrote: > >>On Fri, Nov 19, 2010 at 12:06 AM, Alexander Kabaev > >>wrote: > >>>On Thu, 18 Nov 2010 18:56:35 + > >>>Alexander Best wrote: > >>> > >>>>On Thu Nov 18 10, Matthew D. Fuller wrote: > >>>>>On Thu, Nov 18, 2010 at 06:23:24PM + I heard the voice of > >>>>>Alexander Best, and lo! it spake thus: > >>>>>>judging from the videos the changes are having a huge impact imo. > >>>>>Well, my (admittedly limited, and certainly anecdotal) experience is > >>>>>that Linux's interactive response when under heavy load was always > >>>>>much worse than FreeBSD's. So maybe that's just them catching up to > >>>>>where we already are ;) > >>>>well...i tried playing back a 1080p vide files while doing > >>>>`make -j64 buildkernel` and FreeBSD's interactivity seems far from > >>>>perfect. > >>>One thing that just begs to be asked: since when decoding 1080p became > >>>an interactive task? > >>> > >>Strictly speaking it isn't - but displaying it is a timing-sensitive > >>task that isn't CPU- or I/O-bound, and scheduling-wise that probably > >>makes it more like the "fast response when woken up" interactive tasks > >>than a CPU-bound non-interactive process. > >>Decoding it into another file on the disk is in the latter category, > >>of course - but I don't think that's what he meant. :) > >> > >>More on topic - while this was a tiny patch for Linux, it seems like > >>it would take more work for us, since I don't believe either of the > >>schedulers handles task groups in the required way. The linux patch > >>was just "create task groups automatically", since they already had > >>some suitable logic for scheduling based on task groups in their CFS > >>scheduler. We would have to (re-)add that first, which is non-trivial. > >personally i think freebsd would hugely benefit from a scheduler framework > >such as geom/gsched, where it's easy to switch between various algorithms. > > > >that way it be much easier to try out new concepts without having to write > >a > >completely new scheduler. > > we are part of the way there.. > > at least we did abstract the scheduler to the point where > we have two completely different ones. > you are welcome to develop a 'framework as you describe and plug it into > the abstraction we already have. 17:49 @ arundel : also looking at the svn log shows that still a lot of \ commits happen to sched_4bsd. so it's defenately not being abbandoned. in \ fact there might be situations where it performs better than sched_ule. 17:50 @ arundel : i'm looking forward to a scheduler which looks sorta like \ geom and enables you to plugin addition plugins with different scheduling \ algorithms. :) 17:51 @ Genesys : Luigi Rizzo had a plugabble scheduler back in 4.* or \ thereabouts 17:51 @ Genesys : you could kldload new ones and switch to them on the fly 17:52 @ arundel : wow. that sounds cool. too bad it didn't make it into src \ tree. by now it's probably outdated and needs to be reworked quite a bit. does anybody know something about this? i'm sorry. i'd really love to contribute some code, but my programing skills are pretty scrappy. ;) it would probably take me 20 years to figure out the current sched code. cheers. alex > > >cheers. > >alex > > > >> > >>-- > >>Daniel Nebdal -- a13x ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On 11/18/10 3:37 PM, Alexander Best wrote: On Fri Nov 19 10, Daniel Nebdal wrote: On Fri, Nov 19, 2010 at 12:06 AM, Alexander Kabaev wrote: On Thu, 18 Nov 2010 18:56:35 + Alexander Best wrote: On Thu Nov 18 10, Matthew D. Fuller wrote: On Thu, Nov 18, 2010 at 06:23:24PM + I heard the voice of Alexander Best, and lo! it spake thus: judging from the videos the changes are having a huge impact imo. Well, my (admittedly limited, and certainly anecdotal) experience is that Linux's interactive response when under heavy load was always much worse than FreeBSD's. So maybe that's just them catching up to where we already are ;) well...i tried playing back a 1080p vide files while doing `make -j64 buildkernel` and FreeBSD's interactivity seems far from perfect. One thing that just begs to be asked: since when decoding 1080p became an interactive task? Strictly speaking it isn't - but displaying it is a timing-sensitive task that isn't CPU- or I/O-bound, and scheduling-wise that probably makes it more like the "fast response when woken up" interactive tasks than a CPU-bound non-interactive process. Decoding it into another file on the disk is in the latter category, of course - but I don't think that's what he meant. :) More on topic - while this was a tiny patch for Linux, it seems like it would take more work for us, since I don't believe either of the schedulers handles task groups in the required way. The linux patch was just "create task groups automatically", since they already had some suitable logic for scheduling based on task groups in their CFS scheduler. We would have to (re-)add that first, which is non-trivial. personally i think freebsd would hugely benefit from a scheduler framework such as geom/gsched, where it's easy to switch between various algorithms. that way it be much easier to try out new concepts without having to write a completely new scheduler. we are part of the way there.. at least we did abstract the scheduler to the point where we have two completely different ones. you are welcome to develop a 'framework as you describe and plug it into the abstraction we already have. cheers. alex -- Daniel Nebdal ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On Fri Nov 19 10, Daniel Nebdal wrote: > On Fri, Nov 19, 2010 at 12:06 AM, Alexander Kabaev wrote: > > On Thu, 18 Nov 2010 18:56:35 + > > Alexander Best wrote: > > > >> On Thu Nov 18 10, Matthew D. Fuller wrote: > >> > On Thu, Nov 18, 2010 at 06:23:24PM + I heard the voice of > >> > Alexander Best, and lo! it spake thus: > >> > > > >> > > judging from the videos the changes are having a huge impact imo. > >> > > >> > Well, my (admittedly limited, and certainly anecdotal) experience is > >> > that Linux's interactive response when under heavy load was always > >> > much worse than FreeBSD's. So maybe that's just them catching up to > >> > where we already are ;) > >> > >> well...i tried playing back a 1080p vide files while doing > >> `make -j64 buildkernel` and FreeBSD's interactivity seems far from > >> perfect. > > > > One thing that just begs to be asked: since when decoding 1080p became > > an interactive task? > > > > Strictly speaking it isn't - but displaying it is a timing-sensitive > task that isn't CPU- or I/O-bound, and scheduling-wise that probably > makes it more like the "fast response when woken up" interactive tasks > than a CPU-bound non-interactive process. > Decoding it into another file on the disk is in the latter category, > of course - but I don't think that's what he meant. :) > > More on topic - while this was a tiny patch for Linux, it seems like > it would take more work for us, since I don't believe either of the > schedulers handles task groups in the required way. The linux patch > was just "create task groups automatically", since they already had > some suitable logic for scheduling based on task groups in their CFS > scheduler. We would have to (re-)add that first, which is non-trivial. personally i think freebsd would hugely benefit from a scheduler framework such as geom/gsched, where it's easy to switch between various algorithms. that way it be much easier to try out new concepts without having to write a completely new scheduler. cheers. alex > > > -- > Daniel Nebdal -- a13x ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On Thu, Nov 18, 2010 at 3:12 PM, Steve Kargl wrote: > On Thu, Nov 18, 2010 at 10:59:43PM +, Alexander Best wrote: >> >> well i did exactly what they did in the video. watch a 1080p video and move >> the output window around while compiling the kernel. >> > > It is trivial to bring ULE to its knees. If you > have N cores then all you need is N+1 cpu intensive > task. The issue has been known for a few years. I/O intensive tasks don't help either though. Things tend to get choppy whenever I'm doing something like update from svn and doing something a bit more interactive like use firefox (and scroll... for instance gmail seems to be a pig in this area -- heh), vlc is a little less painful, etc. I wonder if the issue isn't necessarily tasks but more or less locking. firefox uses a lot of threads and file based mutexes according to what I've seen with top and ps (please correct me if I'm wrong). firefox also has a tendency with the nvidia-driver (I know... blobs are bad) to produce funky artifacts on the screen (again, when scrolling on gmail, dealing with flash, etc) or certain bits don't redraw properly (text when scrolling ends up ghosting on multiple lines until I force a redraw with the cursor or by scrolling back over that section of text). I'm sure there are a lot of performance issues within FreeBSD and opensource desktop software that needs to be properly worked out on a case by case basis, so I think that writing everything off as awesome and working better with one magic patch is unlikely. Besides, Linux has a more complicated scheduler than FreeBSD does. Thanks, -Garrett ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On Thu, Nov 18, 2010 at 10:59:43PM +, Alexander Best wrote: > > well i did exactly what they did in the video. watch a 1080p video and move > the output window around while compiling the kernel. > It is trivial to bring ULE to its knees. If you have N cores then all you need is N+1 cpu intensive task. The issue has been known for a few years. http://freebsd.monkey.org/freebsd-current/200807/msg00278.html http://www.mail-archive.com/freebsd-hack...@freebsd.org/msg65839.html -- Steve ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On Thu Nov 18 10, Alexander Kabaev wrote: > On Thu, 18 Nov 2010 18:56:35 + > Alexander Best wrote: > > > On Thu Nov 18 10, Matthew D. Fuller wrote: > > > On Thu, Nov 18, 2010 at 06:23:24PM + I heard the voice of > > > Alexander Best, and lo! it spake thus: > > > > > > > > judging from the videos the changes are having a huge impact imo. > > > > > > Well, my (admittedly limited, and certainly anecdotal) experience is > > > that Linux's interactive response when under heavy load was always > > > much worse than FreeBSD's. So maybe that's just them catching up to > > > where we already are ;) > > > > well...i tried playing back a 1080p vide files while doing > > `make -j64 buildkernel` and FreeBSD's interactivity seems far from > > perfect. > > One thing that just begs to be asked: since when decoding 1080p became > an interactive task? well i did exactly what they did in the video. watch a 1080p video and move the output window around while compiling the kernel. cheers. alex > > -- > Alexander Kabaev -- a13x ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On Fri, Nov 19, 2010 at 12:06 AM, Alexander Kabaev wrote: > On Thu, 18 Nov 2010 18:56:35 + > Alexander Best wrote: > >> On Thu Nov 18 10, Matthew D. Fuller wrote: >> > On Thu, Nov 18, 2010 at 06:23:24PM + I heard the voice of >> > Alexander Best, and lo! it spake thus: >> > > >> > > judging from the videos the changes are having a huge impact imo. >> > >> > Well, my (admittedly limited, and certainly anecdotal) experience is >> > that Linux's interactive response when under heavy load was always >> > much worse than FreeBSD's. So maybe that's just them catching up to >> > where we already are ;) >> >> well...i tried playing back a 1080p vide files while doing >> `make -j64 buildkernel` and FreeBSD's interactivity seems far from >> perfect. > > One thing that just begs to be asked: since when decoding 1080p became > an interactive task? > Strictly speaking it isn't - but displaying it is a timing-sensitive task that isn't CPU- or I/O-bound, and scheduling-wise that probably makes it more like the "fast response when woken up" interactive tasks than a CPU-bound non-interactive process. Decoding it into another file on the disk is in the latter category, of course - but I don't think that's what he meant. :) More on topic - while this was a tiny patch for Linux, it seems like it would take more work for us, since I don't believe either of the schedulers handles task groups in the required way. The linux patch was just "create task groups automatically", since they already had some suitable logic for scheduling based on task groups in their CFS scheduler. We would have to (re-)add that first, which is non-trivial. -- Daniel Nebdal ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On Thu, 18 Nov 2010 18:56:35 + Alexander Best wrote: > On Thu Nov 18 10, Matthew D. Fuller wrote: > > On Thu, Nov 18, 2010 at 06:23:24PM + I heard the voice of > > Alexander Best, and lo! it spake thus: > > > > > > judging from the videos the changes are having a huge impact imo. > > > > Well, my (admittedly limited, and certainly anecdotal) experience is > > that Linux's interactive response when under heavy load was always > > much worse than FreeBSD's. So maybe that's just them catching up to > > where we already are ;) > > well...i tried playing back a 1080p vide files while doing > `make -j64 buildkernel` and FreeBSD's interactivity seems far from > perfect. One thing that just begs to be asked: since when decoding 1080p became an interactive task? -- Alexander Kabaev signature.asc Description: PGP signature
Re: TTY task group scheduling
On 11/18/10 19:55, Lucius Windschuh wrote: 2010/11/18 Andriy Gapon: [Grouping of processes into TTY groups] Well, I think that those improvements apply only to a very specific usage pattern and are greatly over-hyped. But there are serious issue if you use FreeBSD as a desktop OS with SMP and SCHED_ULE, or? Because currently, my machine is barely usable if a compile job with parallelism is running. Movies stutter, Firefox hangs. And even nice -n 20 doesn't do the job in every case, as +20 seems not to be the idle priority anymore?!? And using "idprio 1 $cmd" as a workaround is, well, a kludge. I am not sure if TTY grouping is the right solution, if you look at potentially CPU-intensive GUI applications that all run on the same TTY (or no TTY at all? Same problem). Maybe, we could simply enhance the algorithm that decides if a task is interactive? That would also improve the described situation. Regards, Lucius Stuttering Response, being stuck for over 20 seconds also happens when I start updating the OS' sources via svn. This happens on all boxes, some of them do have 8 cores (ob two CPUs) and plenty of RAM. Heavy disk I/O, doesn't matter on UFS2 or ZFS, also brings boxes to stutter, those phenomena are most seen when you interact with the machine via X11 clients. I think it's hard to realize if a server only does console I/O, but console also seems to be stuck sometimes. It would be worth checking this with some 'benchmark'. X11 in its kind of oldish incarnation on FreeBSD seems to contribute most to those slowdowns, what so ever. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On 11/18/10 19:28, Matthew D. Fuller wrote: On Thu, Nov 18, 2010 at 06:23:24PM + I heard the voice of Alexander Best, and lo! it spake thus: judging from the videos the changes are having a huge impact imo. Well, my (admittedly limited, and certainly anecdotal) experience is that Linux's interactive response when under heavy load was always much worse than FreeBSD's. So maybe that's just them catching up to where we already are ;) Wishful thinking? ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On 11/18/10 10:55 AM, Lucius Windschuh wrote: 2010/11/18 Andriy Gapon: [Grouping of processes into TTY groups] Well, I think that those improvements apply only to a very specific usage pattern and are greatly over-hyped. But there are serious issue if you use FreeBSD as a desktop OS with SMP and SCHED_ULE, or? Because currently, my machine is barely usable if a compile job with parallelism is running. Movies stutter, Firefox hangs. And even nice -n 20 doesn't do the job in every case, as +20 seems not to be the idle priority anymore?!? And using "idprio 1 $cmd" as a workaround is, well, a kludge. I am not sure if TTY grouping is the right solution, if you look at potentially CPU-intensive GUI applications that all run on the same TTY (or no TTY at all? Same problem). Maybe, we could simply enhance the algorithm that decides if a task is interactive? That would also improve the described situation. tty grouping is a variant of what we used to have at one stage which is a "kernel schedulable entity group".. KSEG the idea is that all items in a group share some characteristic and some amount of resources. We stripped the KSEG out of the picture because it really complicated the picture. Regards, Lucius ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On Thu, Nov 18, 2010 at 10:56 AM, Alexander Best wrote: > On Thu Nov 18 10, Matthew D. Fuller wrote: >> On Thu, Nov 18, 2010 at 06:23:24PM + I heard the voice of >> Alexander Best, and lo! it spake thus: >> > >> > judging from the videos the changes are having a huge impact imo. >> >> Well, my (admittedly limited, and certainly anecdotal) experience is >> that Linux's interactive response when under heavy load was always >> much worse than FreeBSD's. So maybe that's just them catching up to >> where we already are ;) > > well...i tried playing back a 1080p vide files while doing > `make -j64 buildkernel` and FreeBSD's interactivity seems far from perfect. > > it might be possible that linux'es interactivity was worse than freebsd's, > but still this patch should be evaluated for freebsd. in this particular case > it seems linux now does better than freebsd. Maybe I'm just a lowly user and don't fully understand the issue, but isn't this the whole reason for having /usr/bin/nice installed? As in, if you don't want your make job to hog resources, then use nice to run it in the background. How does the work on the geom_sched (for I/O scheduling) play into this? Am I missing something fundamental to the issue in question? Also, does this really need to be cross-posted to -current, -hackers, and -performance? -- Freddie Cash fjwc...@gmail.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On Nov 18, 2010, at 10:23 AM, Alexander Best wrote: > On Thu Nov 18 10, Andriy Gapon wrote: >> on 18/11/2010 13:04 O. Hartmann said the following: >>> On 11/18/10 02:30, grarpamp wrote: Just documenting regarding interactive performance things. This one's from Linux. http://www.phoronix.com/scan.php?page=article&item=linux_2637_video&num=1 >>> >>> Well, >>> it would be nice to have those improvements in FreeBSD, but I doubt this >>> will make >>> it in due time to FreeBSD's kernel. >> >> Well, I think that those improvements apply only to a very specific usage >> pattern >> and are greatly over-hyped. > > you think so? judging from the videos the changes are having a huge impact > imo. > > cheers. > alex > > btw: i posted a similar thread on freebsd-backers@, but nobody seemed > interested in it. subject line is "sched_autogroup_enabled". I attempted to find reliable benchmarks or even an idea as to what they thought they were measuring, because a sentence like the following: Tests done by Mike show the maximum latency dropping by over ten times and the average latency of the desktop by about 60 times. ...isn't mathematically possible, I'm pretty sure. Frankly, I'm also turned off by the attempt to popup a full page ad in addition to the rest of the advertising content which surrounds what is nominally supposed to be the real content. That doesn't mean there is anything wrong with the patch or the notion of adjusting the scheduler, but I don't see any value added from these phoronix.com links. Regards, -- -Chuck ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
2010/11/18 Andriy Gapon : > [Grouping of processes into TTY groups] > > Well, I think that those improvements apply only to a very specific usage > pattern > and are greatly over-hyped. But there are serious issue if you use FreeBSD as a desktop OS with SMP and SCHED_ULE, or? Because currently, my machine is barely usable if a compile job with parallelism is running. Movies stutter, Firefox hangs. And even nice -n 20 doesn't do the job in every case, as +20 seems not to be the idle priority anymore?!? And using "idprio 1 $cmd" as a workaround is, well, a kludge. I am not sure if TTY grouping is the right solution, if you look at potentially CPU-intensive GUI applications that all run on the same TTY (or no TTY at all? Same problem). Maybe, we could simply enhance the algorithm that decides if a task is interactive? That would also improve the described situation. Regards, Lucius ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On Thu Nov 18 10, Rob Farmer wrote: > On Thu, Nov 18, 2010 at 10:39, Chuck Swiger wrote: > > Frankly, I'm also turned off by the attempt to popup a full page ad in > > addition to the rest of the advertising content which surrounds what is > > nominally supposed to be the real content. That doesn't mean there is > > anything wrong with the patch or the notion of adjusting the scheduler, but > > I don't see any value added from these phoronix.com links. > > Most stuff on Phoronix is of dubious value, and they have outright > lied about stuff in the past, in order to drum up advertising business > (such as Steam on Linux, which Value has consistently said isn't > happening). so we need a trusted source to check whether the impact of the ~200 line patch as claimed by phoronix remains valid. can anybody test this? or provide links to independant benchmark results? cheers. alex > > -- > Rob Farmer -- a13x ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On Thu Nov 18 10, Matthew D. Fuller wrote: > On Thu, Nov 18, 2010 at 06:23:24PM + I heard the voice of > Alexander Best, and lo! it spake thus: > > > > judging from the videos the changes are having a huge impact imo. > > Well, my (admittedly limited, and certainly anecdotal) experience is > that Linux's interactive response when under heavy load was always > much worse than FreeBSD's. So maybe that's just them catching up to > where we already are ;) well...i tried playing back a 1080p vide files while doing `make -j64 buildkernel` and FreeBSD's interactivity seems far from perfect. it might be possible that linux'es interactivity was worse than freebsd's, but still this patch should be evaluated for freebsd. in this particular case it seems linux now does better than freebsd. cheers. alex > > > -- > Matthew Fuller (MF4839) | fulle...@over-yonder.net > Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ >On the Internet, nobody can hear you scream. -- a13x ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On Thu, Nov 18, 2010 at 06:23:24PM + I heard the voice of Alexander Best, and lo! it spake thus: > > judging from the videos the changes are having a huge impact imo. Well, my (admittedly limited, and certainly anecdotal) experience is that Linux's interactive response when under heavy load was always much worse than FreeBSD's. So maybe that's just them catching up to where we already are ;) -- Matthew Fuller (MF4839) | fulle...@over-yonder.net Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ On the Internet, nobody can hear you scream. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On Thu, Nov 18, 2010 at 10:39, Chuck Swiger wrote: > Frankly, I'm also turned off by the attempt to popup a full page ad in > addition to the rest of the advertising content which surrounds what is > nominally supposed to be the real content. That doesn't mean there is > anything wrong with the patch or the notion of adjusting the scheduler, but I > don't see any value added from these phoronix.com links. Most stuff on Phoronix is of dubious value, and they have outright lied about stuff in the past, in order to drum up advertising business (such as Steam on Linux, which Value has consistently said isn't happening). -- Rob Farmer ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On Thu Nov 18 10, Andriy Gapon wrote: > on 18/11/2010 13:04 O. Hartmann said the following: > > On 11/18/10 02:30, grarpamp wrote: > >> Just documenting regarding interactive performance things. > >> This one's from Linux. > >> > >> http://www.phoronix.com/scan.php?page=article&item=linux_2637_video&num=1 > > > > Well, > > it would be nice to have those improvements in FreeBSD, but I doubt this > > will make > > it in due time to FreeBSD's kernel. > > Well, I think that those improvements apply only to a very specific usage > pattern > and are greatly over-hyped. you think so? judging from the videos the changes are having a huge impact imo. cheers. alex btw: i posted a similar thread on freebsd-backers@, but nobody seemed interested in it. subject line is "sched_autogroup_enabled". > > P.S. What is the due time, BTW? > > -- > Andriy Gapon -- a13x ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
on 18/11/2010 13:04 O. Hartmann said the following: > On 11/18/10 02:30, grarpamp wrote: >> Just documenting regarding interactive performance things. >> This one's from Linux. >> >> http://www.phoronix.com/scan.php?page=article&item=linux_2637_video&num=1 > > Well, > it would be nice to have those improvements in FreeBSD, but I doubt this will > make > it in due time to FreeBSD's kernel. Well, I think that those improvements apply only to a very specific usage pattern and are greatly over-hyped. P.S. What is the due time, BTW? -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TTY task group scheduling
On 11/18/10 02:30, grarpamp wrote: Just documenting regarding interactive performance things. This one's from Linux. http://www.phoronix.com/scan.php?page=article&item=linux_2637_video&num=1 Well, it would be nice to have those improvements in FreeBSD, but I doubt this will make it in due time to FreeBSD's kernel. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Event based scheduling and USB.
Alexander Motin wrote: > Takanori Watanabe wrote: >> I updated my FreeBSD tree on laptop, to the current >> as of 18 Oct.2010, it works fine with CPU C3 state enabled, >> >> I think this is your achievement of event time scheduler, >> thanks! >> >> But when USB driver is enabled, the load average is considerablly >> high (0.6 to 1.0) if sysctl oid kern.eventtimer.periodic is set to 0. >> Then kern.eventtimer.periodic is set to 1, the load average goes >> to 0 quickly as before, but almost never transit to C3. >> >> Is this behavior expected, or something wrong? >> I noticed one of usb host controller device shares HPET irq. >> When I implement interrupt filter in uhci driver, the load average >> goes to 0 as before. >> >> >> % vmstat -i >> interrupt total rate >> irq1: atkbd0 398 2 >> irq9: acpi0 408 2 >> irq12: psm03 0 >> irq19: ehci1 37 0 >> irq20: hpet0 uhci0 35970230 >> irq22: ehci0 2 0 >> irq256: em04 0 >> irq257: ahci0 1692 10 >> Total 38514246 >> === > > I haven't noticed that issue and it is surely not expected for me. I > will try to reproduce it. I've easily reproduced the problem. Scheduler tracing shows that problem is the result of aliasing between "swi4: clock" thread on one CPU (measuring load average) and "irq21: hpet0 uhci1" thread on another. Those two events are aliased by definition due to shared interrupt source. Not sure what to do with it. Either we should change algorithm of load average calculation or exclude timer's interrupt threads from load average accounting. Adding interrupt filter for USB also reasonably helps, but it is only a partial solution for this specific sharing case. -- Alexander Motin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Event based scheduling and USB.
On Wednesday 27 October 2010 10:14:18 Alexander Motin wrote: > As I understand, if respective USB port is not used, USB stack should > put it into power_save mode not poll so often to deny entering C3 state. USB will stop the hardware from polling RAM, but still a 4 second root HUB software timer/watchdog will be running. --HPS ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Event based scheduling and USB.
Quoting Alexander Motin (from Tue, 26 Oct 2010 22:57:59 +0300): Takanori Watanabe wrote: It's time to implement powertop for freebsd, isn't it? Surely it is. I was even thinking about possibility to port one from OpenSolaris, but other work distracted me. You may take it, it you wish. For the benefit of the people which didn't see my message with the URL (I don't know if I was sending the URL to mav@ personally or if it appeared also in the lists): http://hub.opensolaris.org/bin/view/Project+tesla/Powertop is a DTrace-ified version of PowerTop (at the end of the page is a description how to get the source). And for those which like plots of the values: http://hub.opensolaris.org/bin/view/Project+tesla/ptop-gtk Bye, Alexander. -- A RACF protected dataset is inaccessible. http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Event based scheduling and USB.
Nate Lawson wrote: > On 10/26/2010 12:57 PM, Alexander Motin wrote: >> Takanori Watanabe wrote: >>> I updated my FreeBSD tree on laptop, to the current >>> as of 18 Oct.2010, it works fine with CPU C3 state enabled, >>> >>> I think this is your achievement of event time scheduler, >>> thanks! > > Ah, so mav@ implemented a tickless-scheduler? That is nice. Not exactly. I've only made system to delay empty ticks when idle and execute them later on wakeup in a batch. Scheduler work is still wanted. >>> But when USB driver is enabled, the load average is considerablly >>> high (0.6 to 1.0) if sysctl oid kern.eventtimer.periodic is set to 0. >>> Then kern.eventtimer.periodic is set to 1, the load average goes >>> to 0 quickly as before, but almost never transit to C3. >>> >>> Is this behavior expected, or something wrong? > > The USB controller often keeps the bus mastering bit set. This keeps the > system out of C3. The way to fix this is to implement global suspend. > Put a device in suspend mode and then turn off power to the USB port it > is on. Then the USB controller will stop polling the bus. As I understand, if respective USB port is not used, USB stack should put it into power_save mode not poll so often to deny entering C3 state. >>> I noticed one of usb host controller device shares HPET irq. >>> When I implement interrupt filter in uhci driver, the load average >>> goes to 0 as before. >>> >>> >>> >>> % vmstat -i >>> interrupt total rate >>> irq1: atkbd0 398 2 >>> irq9: acpi0 408 2 >>> irq12: psm03 0 >>> irq19: ehci1 37 0 >>> irq20: hpet0 uhci0 35970230 >>> irq22: ehci0 2 0 >>> irq256: em04 0 >>> irq257: ahci0 1692 10 >>> Total 38514246 >>> === >> I haven't noticed that issue and it is surely not expected for me. I >> will try to reproduce it. >> >> Most likely you should be able to avoid interrupt sharing using some >> additional HPET options, described at hpet(4). > > This seems silly. The whole point of APIC is to avoid clustering on a > single interrupt but the BIOS put the timer on the USB controller irq? HPET timer is not a regular ISA or PCI device. It allows several different interrupt configurations. In most cases I remember, BIOS setups interrupts 0 and 8, like for legacy_route mode. But this mode is not really suitable as default in our case ATM due to conflict with atrtc and attimer drivers. -- Alexander Motin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Event based scheduling and USB.
By default USB devices are not suspended. You can use "usbconfig power_save" to enable automatic power save for all devices. --HPS ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Event based scheduling and USB.
On 10/26/2010 12:57 PM, Alexander Motin wrote: > Takanori Watanabe wrote: >> I updated my FreeBSD tree on laptop, to the current >> as of 18 Oct.2010, it works fine with CPU C3 state enabled, >> >> I think this is your achievement of event time scheduler, >> thanks! Ah, so mav@ implemented a tickless-scheduler? That is nice. >> But when USB driver is enabled, the load average is considerablly >> high (0.6 to 1.0) if sysctl oid kern.eventtimer.periodic is set to 0. >> Then kern.eventtimer.periodic is set to 1, the load average goes >> to 0 quickly as before, but almost never transit to C3. >> >> Is this behavior expected, or something wrong? The USB controller often keeps the bus mastering bit set. This keeps the system out of C3. The way to fix this is to implement global suspend. Put a device in suspend mode and then turn off power to the USB port it is on. Then the USB controller will stop polling the bus. >> I noticed one of usb host controller device shares HPET irq. >> When I implement interrupt filter in uhci driver, the load average >> goes to 0 as before. >> >> >> >> % vmstat -i >> interrupt total rate >> irq1: atkbd0 398 2 >> irq9: acpi0 408 2 >> irq12: psm03 0 >> irq19: ehci1 37 0 >> irq20: hpet0 uhci0 35970230 >> irq22: ehci0 2 0 >> irq256: em04 0 >> irq257: ahci0 1692 10 >> Total 38514246 >> === > > I haven't noticed that issue and it is surely not expected for me. I > will try to reproduce it. > > Most likely you should be able to avoid interrupt sharing using some > additional HPET options, described at hpet(4). This seems silly. The whole point of APIC is to avoid clustering on a single interrupt but the BIOS put the timer on the USB controller irq? >> It's time to implement powertop for freebsd, isn't it? > > Surely it is. I was even thinking about possibility to port one from > OpenSolaris, but other work distracted me. You may take it, it you wish. It seems worth doing the internals new, but maybe outputting information in a compatible format for reporting tools. -- Nate ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Event based scheduling and USB.
Takanori Watanabe wrote: > I updated my FreeBSD tree on laptop, to the current > as of 18 Oct.2010, it works fine with CPU C3 state enabled, > > I think this is your achievement of event time scheduler, > thanks! > > But when USB driver is enabled, the load average is considerablly > high (0.6 to 1.0) if sysctl oid kern.eventtimer.periodic is set to 0. > Then kern.eventtimer.periodic is set to 1, the load average goes > to 0 quickly as before, but almost never transit to C3. > > Is this behavior expected, or something wrong? > I noticed one of usb host controller device shares HPET irq. > When I implement interrupt filter in uhci driver, the load average > goes to 0 as before. > > > > % vmstat -i > interrupt total rate > irq1: atkbd0 398 2 > irq9: acpi0 408 2 > irq12: psm03 0 > irq19: ehci1 37 0 > irq20: hpet0 uhci0 35970230 > irq22: ehci0 2 0 > irq256: em04 0 > irq257: ahci0 1692 10 > Total 38514246 > === I haven't noticed that issue and it is surely not expected for me. I will try to reproduce it. Most likely you should be able to avoid interrupt sharing using some additional HPET options, described at hpet(4). > BTW, when USB port is enabled C3 transition rate gets lower. > I think it is likely to occur. But how can I supress power > consumption? I can't say about USB, but you may try this patch to optimize some other subsystems: http://people.freebsd.org/~mav/tm6292_idle.patch > It's time to implement powertop for freebsd, isn't it? Surely it is. I was even thinking about possibility to port one from OpenSolaris, but other work distracted me. You may take it, it you wish. -- Alexander Motin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Event based scheduling and USB.
I updated my FreeBSD tree on laptop, to the current as of 18 Oct.2010, it works fine with CPU C3 state enabled, I think this is your achievement of event time scheduler, thanks! But when USB driver is enabled, the load average is considerablly high (0.6 to 1.0) if sysctl oid kern.eventtimer.periodic is set to 0. Then kern.eventtimer.periodic is set to 1, the load average goes to 0 quickly as before, but almost never transit to C3. Is this behavior expected, or something wrong? I noticed one of usb host controller device shares HPET irq. When I implement interrupt filter in uhci driver, the load average goes to 0 as before. % vmstat -i interrupt total rate irq1: atkbd0 398 2 irq9: acpi0 408 2 irq12: psm03 0 irq19: ehci1 37 0 irq20: hpet0 uhci0 35970230 irq22: ehci0 2 0 irq256: em04 0 irq257: ahci0 1692 10 Total 38514246 === BTW, when USB port is enabled C3 transition rate gets lower. I think it is likely to occur. But how can I supress power consumption? It's time to implement powertop for freebsd, isn't it? ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: scheduling
-current?
libkse?
libthr?
SCHED_ULE?
SCHED_4BSD?
On Thu, 4 Sep 2003, Andy Farkas wrote:
>
> Not to flog a dead horse, but scheduling seems to be very broken this month.
>
> I am subjectively watching my smp box do a:
>
> 'cd /usr/ports/www/apache2 ; make all' in one window, and
>
> 'cd cd /usr/ports/databases/mysql40-server/ ; make all' in another window,
>
> and most disturbingly a 'top -S' in a 3rd window reporting 42.63% idle on
> cpu0, 39.50% idle on cpu1.
>
> It just doesn't seem right to me.
>
> --
>
> :{ [EMAIL PROTECTED]
>
> Andy Farkas
> System Administrator
>Speednet Communications
> http://www.speednet.com.au/
>
>
>
> ___
> [EMAIL PROTECTED] mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "[EMAIL PROTECTED]"
>
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: scheduling
On Thu, Sep 04, 2003 at 03:24:13AM +1000, Andy Farkas wrote: > > Not to flog a dead horse, but scheduling seems to be very broken this month. > > I am subjectively watching my smp box do a: > > 'cd /usr/ports/www/apache2 ; make all' in one window, and > > 'cd cd /usr/ports/databases/mysql40-server/ ; make all' in another window, > > and most disturbingly a 'top -S' in a 3rd window reporting 42.63% idle on > cpu0, 39.50% idle on cpu1. > > It just doesn't seem right to me. > You failed to mention which scheduler you are using. -- Steve ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "[EMAIL PROTECTED]"
scheduling
Not to flog a dead horse, but scheduling seems to be very broken this month.
I am subjectively watching my smp box do a:
'cd /usr/ports/www/apache2 ; make all' in one window, and
'cd cd /usr/ports/databases/mysql40-server/ ; make all' in another window,
and most disturbingly a 'top -S' in a 3rd window reporting 42.63% idle on
cpu0, 39.50% idle on cpu1.
It just doesn't seem right to me.
--
:{ [EMAIL PROTECTED]
Andy Farkas
System Administrator
Speednet Communications
http://www.speednet.com.au/
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
CIBRES Article: The New Paradigm In Project Scheduling
Title: CIBRES.COM Remove Subscription Contact CIBRES CIBRES.COM NEWS CIBRES Announces New E-business Certification Program (CIEB) CIBRES Guarantees CIERP Certification CIBRES "ERP For Dummies" is now shipping EVENTS CIBRES CIERP Seminar 8 openings left (Reno, NV Feb 7-9) CIBRES CIERP Exam (World Wide Feb 13) CIBRES CIERP Seminar (San Fran., CA, Apr 4-6) The New Paradigm In Project Scheduling by Marc Weinberg, CFPIM, CIRM, PMP, CIERP Released January 15, 2002 Most people agree that traditional project scheduling needs improvement. One only has to look at some of the major projects that never finished on time and at a high cost, such as the Denver, CO Airport or the Chunnel connecting England and France. With project managers under increased pressure to bring their projects in on time, the traditional project scheduling approach is giving way to a new approach based on the Critical Chain. Using the Critical Chain approach, more projects are completed more quickly and with greater scheduling reliability. Details 35 New Forums!- Check out the CIBRES forums for information needs. Searchable indexes and easy no registration access make finding information easy! CIBRES offers forums for your networking, e
USB - bulk data scheduling in UHCI
The current UHCI driver constructs the bulk transfer queue as a simple
list with a 'terminate' marker on the end. This means that the bulk queue
runs only once per frame period. This is OK for devices with large input
buffers, but in the case of a large transfer to a device with a small
input buffer, it limits the throughput to 1 buffer per frame time (nominal
1ms). In the case of the hardware I am using, the buffer is 128 bytes, so
I only get 128000 bytes/sec throughput with the UHCI driver, compared to
over 20 bytes/sec with OHCI.
If the UHCI driver arranges the bulk transfer queue as a circular list,
transfers will be retried repeatedly in what would otherwise be wasted
time at the end of the frame; this is similar to what OHCI does. In fact
in my application the patched UHCI driver comes out slightly better than
OHCI (though this may be other factors like CPU speed).
The patch to do this appears to be very simple (this diff is against
-stable as my -current machine is OHCI, but the code is identical in
-current).
Index: uhci.c
===
RCS file: /repository/src/sys/dev/usb/uhci.c,v
retrieving revision 1.40.2.7
diff -c -r1.40.2.7 uhci.c
*** uhci.c 31 Oct 2000 23:23:29 - 1.40.2.7
--- uhci.c 15 Dec 2001 23:19:17 -
***
*** 371,377
bsqh = uhci_alloc_sqh(sc);
if (bsqh == NULL)
return (USBD_NOMEM);
! bsqh->qh.qh_hlink = LE(UHCI_PTR_T); /* end of QH chain */
bsqh->qh.qh_elink = LE(UHCI_PTR_T);
sc->sc_bulk_start = sc->sc_bulk_end = bsqh;
--- 371,378
bsqh = uhci_alloc_sqh(sc);
if (bsqh == NULL)
return (USBD_NOMEM);
! bsqh->hlink = bsqh; /* Circular QH chain */
! bsqh->qh.qh_hlink = LE(bsqh->physaddr | UHCI_PTR_Q);
bsqh->qh.qh_elink = LE(UHCI_PTR_T);
sc->sc_bulk_start = sc->sc_bulk_end = bsqh;
***
*** 890,896
DPRINTFN(10, ("uhci_remove_bulk: sqh=%p\n", sqh));
for (pqh = sc->sc_bulk_start; pqh->hlink != sqh; pqh = pqh->hlink)
#if defined(DIAGNOSTIC) || defined(UHCI_DEBUG)
! if (LE(pqh->qh.qh_hlink) & UHCI_PTR_T) {
printf("uhci_remove_bulk: QH not found\n");
return;
}
--- 891,897
DPRINTFN(10, ("uhci_remove_bulk: sqh=%p\n", sqh));
for (pqh = sc->sc_bulk_start; pqh->hlink != sqh; pqh = pqh->hlink)
#if defined(DIAGNOSTIC) || defined(UHCI_DEBUG)
! if (pqh == sc->sc_bulk_end) {
printf("uhci_remove_bulk: QH not found\n");
return;
}
