Re: CFS scheduler unfairly prefers pinned tasks

2015-10-11 Thread paul . szabo
then things seem fair and my users are happy. Thanks, Paul Paul Szabo p...@maths.usyd.edu.au http://www.maths.usyd.edu.au/u/psz/ School of Mathematics and Statistics University of SydneyAustralia -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

Re: [patch] sched: disable task group re-weighting on the desktop

2015-10-11 Thread paul . szabo
Dear Mike, Did you check whether setting min_- and max_interval e.g. as per https://lkml.org/lkml/2015/10/11/34 would help with your issue (instead of your "horrible gs destroying" patch)? Cheers, Paul Paul Szabo p...@maths.usyd.edu.au http://www.maths.usyd.edu.au/u/psz/

Re: CFS scheduler unfairly prefers pinned tasks

2015-10-11 Thread paul . szabo
or, or pointers to documentation, would be appreciated.) --- Thanks for the insightful discussion. (Scary, isn't it?) Thanks, Paul Paul Szabo p...@maths.usyd.edu.au http://www.maths.usyd.edu.au/u/psz/ School of Mathematics and Statistics University of SydneyAustralia -- To unsub

Re: [patch] sched: disable task group re-weighting on the desktop

2015-10-11 Thread paul . szabo
d reproduce, and anyway I believe I have now fixed my problem. (Solution in that "other" email thread.) Cheers, Paul Paul Szabo p...@maths.usyd.edu.au http://www.maths.usyd.edu.au/u/psz/ School of Mathematics and Statistics University of SydneyAustralia -- To unsubscribe from thi

Re: [patch] sched: disable task group re-weighting on the desktop

2015-10-10 Thread paul . szabo
o maybe this is all un-related. Thanks, Paul Paul Szabo p...@maths.usyd.edu.au http://www.maths.usyd.edu.au/u/psz/ School of Mathematics and Statistics University of SydneyAustralia -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a m

Re: CFS scheduler unfairly prefers pinned tasks

2015-10-08 Thread paul . szabo
o get 700%. > IFF ... massively parallel and synchronized ... You would be making the assumption that you had the machine to yourself: might be the wrong thing to assume. >> Good to see that you agree ... > Weeell, we've disagreed on pretty much everything ... Sorry I disagree: we

Re: CFS scheduler unfairly prefers pinned tasks

2015-10-08 Thread paul . szabo
hat happens when you un-pin pert: does it get 100%? What if you run two perts? Have you reproduced my observations? --- Good to see that you agree on the fairness issue... it MUST be fixed! CFS might be wrong or wasteful, but never unfair. Cheers, Paul Paul Szabo p...@maths.usyd.edu.au h

Re: CFS scheduler unfairly prefers pinned tasks

2015-10-06 Thread paul . szabo
are fair with just one un-pinned and un-fair with 2 already. > What you're seeing is not a bug. No task can occupy more than one CPU > at a time, making space reservation on multiple CPUs a very bad idea. I agree that pinning may be bad... should not the kernel penalize the ba

Re: CFS scheduler unfairly prefers pinned tasks

2015-10-06 Thread paul . szabo
icks them off: could not that be done just a little earlier? And further... the CFS is meant to be fair, using things like vruntime to preempt, and throttling. Why are those pinned tasks not preempted or throttled? Thanks, Paul Paul Szabo p...@maths.usyd.edu.au http://www.maths.usyd.edu.au/u/psz/ S

CFS scheduler unfairly prefers pinned tasks

2015-10-05 Thread paul . szabo
irect to me as I am not subscribed to the linux-kernel mailing list (but will try to watch the archives). This bug is also reported to Debian, please see http://bugs.debian.org/800945 I use Debian with the 3.16 kernel, have not yet tried 4.* kernels. Thanks, Paul Paul Szabo p...@maths.usyd.

Re: [RFC] Reproducible OOM with just a few sleeps

2013-02-24 Thread paul . szabo
set # CONFIG_SPARSE_RCU_POINTER is not set Is that sufficient for sparse memory, or should I try something else? Or maybe, you meant that some kernel source patches might be possible in the sparse memory code? Thanks, Paul Paul Szabo p...@maths.usyd.edu.au http://www.maths.usyd.edu.au/u/psz/ School

Re: Bug#695182: [RFC] Reproducible OOM with just a few sleeps

2013-01-31 Thread paul . szabo
ot;sleep test": n=0; while [ $n -lt 33000 ]; do sleep 600 & ((n=n+1)); done and somewhere also said that non-PAE passes. Does not that prove that PAE is broken? Cheers, Paul Paul Szabo p...@maths.usyd.edu.au http://www.maths.usyd.edu.au/u/psz/ School of Mathematics and Statistics Univ

Re: Bug#695182: [RFC] Reproducible OOM with just a few sleeps

2013-01-31 Thread paul . szabo
Dear Ben, >> PAE is broken for any amount of RAM. > > No it isn't. Could I please ask you to expand on that? Thanks, Paul Paul Szabo p...@maths.usyd.edu.au http://www.maths.usyd.edu.au/u/psz/ School of Mathematics and Statistics University of SydneyAustralia -- To

Re: Bug#695182: [RFC] Reproducible OOM with just a few sleeps

2013-01-31 Thread paul . szabo
en for any amount of RAM. More precisely, PAE with any RAM fails the "sleep test": n=0; while [ $n -lt 33000 ]; do sleep 600 & ((n=n+1)); done and with >32GB fails the "write test": n=0; while [ $n -lt 99 ]; do dd bs=1M count=1024 if=/dev/zero of=x$n; ((n=n+1)); done

Re: Bug#695182: [RFC] Reproducible OOM with just a few sleeps

2013-01-31 Thread paul . szabo
try PAE on my 512MB laptop. - Though, what do I know, have not yet found the buggy line of code I believe is lurking there... Thanks, Paul Paul Szabo p...@maths.usyd.edu.au http://www.maths.usyd.edu.au/u/psz/ School of Mathematics and Statistics University of SydneyAustralia -- To unsubs

Re: [RFC] Reproducible OOM with just a few sleeps

2013-01-30 Thread paul . szabo
xpected; whereas PAE OOMed then hung (tested with various RAM from 3GB to 64GB). The feeling I get is that amd64 is proposed as a drop-in replacement for PAE, that support and development of PAE is gone, that PAE is dead. Cheers, Paul Paul Szabo p...@maths.usyd.edu.au http://www.maths.u

Re: Bug#695182: [PATCH] Subtract min_free_kbytes from dirtyable memory

2013-01-26 Thread paul . szabo
"solve" all problems, the latest 3.8 kernel still crashes with OOM: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1098961/comments/18 Thanks, Paul Paul Szabo p...@maths.usyd.edu.au http://www.maths.usyd.edu.au/u/psz/ School of Mathematics and Statistics University of Sydney

Re: [PATCH] Negative (setpoint-dirty) in bdi_position_ratio()

2013-01-25 Thread paul . szabo
there is no CONFIG_NUMA nor /proc/sys/vm/zone_reclaim_mode in the Ubuntu non-PAE "plain" HIGHMEM4G kernel, and still it handles the "sleep test" just fine. Where does reclaiming happen (or meant to happen)? Thanks, Paul Paul Szabo p...@maths.usyd.edu.au http://www.maths.us

Re: Bug#695182: [PATCH] Subtract min_free_kbytes from dirtyable memory

2013-01-25 Thread paul . szabo
Dear Ben, > ... the mm maintainers are probably much better placed ... Exactly. Now I wonder: are you one of them? Thanks, Paul Paul Szabo p...@maths.usyd.edu.au http://www.maths.usyd.edu.au/u/psz/ School of Mathematics and Statistics University of SydneyAustralia -- To unsubscr

Re: Bug#695182: [PATCH] Subtract min_free_kbytes from dirtyable memory

2013-01-25 Thread paul . szabo
-- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/

Re: Bug#695182: [PATCH] Subtract min_free_kbytes from dirtyable memory

2013-01-25 Thread paul . szabo
Dear Ben, > If you can identify where it was fixed then ... Sorry I cannot do that. I have no idea where kernel changelogs are kept. I am happy to do some work. Please do not call me lazy. Cheers, Paul Paul Szabo p...@maths.usyd.edu.au http://www.maths.usyd.edu.au/u/psz/ School

Re: [PATCH] Subtract min_free_kbytes from dirtyable memory

2013-01-25 Thread paul . szabo
used in setting limits or threshholds, but not used in decisions where to put dirty things. Is that so, is that as should be? What is the recommended setting of highmem_is_dirtyable? The silence is deafening. I guess highmem_is_dirtyable is an aberration. Thanks, Paul Paul Szabo p...@maths.

Re: [PATCH] Negative (setpoint-dirty) in bdi_position_ratio()

2013-01-24 Thread paul . szabo
t, the drop_caches patch did not protect against the "sleep test". > ... What's your filesystem and the content of /proc/slabinfo? Filesystem is EXT3. See output of slabinfo in Debian bug above or in http://marc.info/?l=linux-mm&m=135796154427544&w=2 Thanks, Paul Paul Sza

Re: [PATCH] Negative (setpoint-dirty) in bdi_position_ratio()

2013-01-24 Thread paul . szabo
quot; (not PAE but HIGHMEM4G) kernel handles the same "sleep test" without any problems. (Thus I now think that the remaining bug is not with writeback.) Cheers, Paul Paul Szabo p...@maths.usyd.edu.au http://www.maths.usyd.edu.au/u/psz/ School of Mathematics and Statistics University of

Re: [PATCH] Negative (setpoint-dirty) in bdi_position_ratio()

2013-01-24 Thread paul . szabo
r 32GB RAM. Oddly the problem does not seem to occur when using mem=32g or lower on the kernel boot line (or on machines with less than 32GB RAM). Cheers, Paul Paul Szabo p...@maths.usyd.edu.au http://www.maths.usyd.edu.au/u/psz/ School of Mathematics and Statistics University of SydneyAustr

Re: [PATCH] Subtract min_free_kbytes from dirtyable memory

2013-01-22 Thread paul . szabo
patch is not needed on those newer kernels. A question: what is the use or significance of vm_highmem_is_dirtyable? It seems odd that it would be used in setting limits or threshholds, but not used in decisions where to put dirty things. Is that so, is that as should be? What is th

[PATCH] Subtract min_free_kbytes from dirtyable memory

2013-01-20 Thread paul . szabo
When calculating amount of dirtyable memory, min_free_kbytes should be subtracted because it is not intended for dirty pages. Using an "extern int" because that is the only interface to some such sysctl values. (This patch does not solve the PAE OOM issue.) Paul Szabo p...@maths.u

[PATCH] MAX_PAUSE to be at least 4

2013-01-20 Thread paul . szabo
Ensure MAX_PAUSE is 4 or larger, so limits in return clamp_val(t, 4, MAX_PAUSE); (the only use of it) are not back-to-front. (This patch does not solve the PAE OOM issue.) Paul Szabo p...@maths.usyd.edu.au http://www.maths.usyd.edu.au/u/psz/ School of Mathematics and Statistics

[PATCH] Negative (setpoint-dirty) in bdi_position_ratio()

2013-01-19 Thread paul . szabo
s64 also prevents overflow with left-shift; though normally these numbers are small and I never observed a 32-bit overflow there. (This patch does not solve the PAE OOM issue.) Paul Szabo p...@maths.usyd.edu.au http://www.maths.usyd.edu.au/u/psz/ School of Mathematics and Statistics

Re: [RFC] Reproducible OOM with just a few sleeps

2013-01-17 Thread paul . szabo
free' shows total Mem 65854128 (up from 64447796 with PAE kernel), and I do not see much change in /proc/iomem output (below). Is that as should be? Thanks, Paul Paul Szabo p...@maths.usyd.edu.au http://www.maths.usyd.edu.au/u/psz/ School of Mathematics and Statistics University of S

Re: [RFC] Reproducible OOM with just a few sleeps

2013-01-14 Thread paul . szabo
y reasonable workarounds ... Only one workaround was proposed: use amd64. PAE is buggy and useless, should be deprecated and removed. Cheers, Paul Paul Szabo p...@maths.usyd.edu.au http://www.maths.usyd.edu.au/u/psz/ School of Mathematics and Statistics University of SydneyAustralia

Re: [RFC] Reproducible OOM with just a few sleeps

2013-01-14 Thread paul . szabo
/proc/iomem will let you > locate your memory holes. Thanks, that might explain it. Output of /proc/iomem below: sorry I do not know how to interpret it. Cheers, Paul Paul Szabo p...@maths.usyd.edu.au http://www.maths.usyd.edu.au/u/psz/ School of Mathematics and Statistics Universit

Re: [RFC] Reproducible OOM with just a few sleeps

2013-01-12 Thread paul . szabo
4671081755304 -/+ buffers/cache: 1615602926060 Swap: 2920 0 2920 and re-trying the "sleep test", it ran into OOM after 18000 or so sleeps and crashed/froze so I had to press the POWER button to recover. Cheers, Paul Paul Szabo p...@maths.usyd.edu.au

[RFC] Reproducible OOM with just a few sleeps

2013-01-11 Thread paul . szabo
0 4068 111328 Low:769044 120232 648812 High: 13740320 315208 13425112 -/+ buffers/cache: 320044 14189320 Swap:134217724 0 134217724 --- Please let me know of any ideas, or if you want me to run some other test or want to see some other out

Re: [RFC] Reproducible OOM with partial workaround

2013-01-11 Thread paul . szabo
v_s64((setpoint - dirty) << RATELIMIT_CALC_SHIFT, + x = div_s64(((s64)setpoint - (s64)dirty) << RATELIMIT_CALC_SHIFT, limit - setpoint + 1); pos_ratio = x; pos_ratio = pos_ratio * x >> RATELIMIT_CALC_SHIFT; ... Cheers, Paul Paul Szabo

Re: [RFC] Reproducible OOM with partial workaround

2013-01-11 Thread paul . szabo
now seems OK with my patch doing drop_caches. Cheers, Paul Paul Szabo p...@maths.usyd.edu.au http://www.maths.usyd.edu.au/u/psz/ School of Mathematics and Statistics University of SydneyAustralia --- root@como:~# free -lm total used free sharedbuff

Re: [RFC] Reproducible OOM with partial workaround

2013-01-10 Thread paul . szabo
0644424892 59646172 -/+ buffers/cache:4232800 60214100 Swap:134217724 0 134217724 psz@como:~$ (though I would not know about violations). But OK, I take your point that I should move with the times. Cheers, Paul Paul Szabo p...@maths.usyd.edu.au http://www.maths.usy

Re: [RFC] Reproducible OOM with partial workaround

2013-01-10 Thread paul . szabo
dual, with OOM occuring much sooner with 64GB than with 34GB. Also, the kernel seems capable of reclaiming lowmem, so I wonder why does that fail just over the 32GB threshhold. (Obviously I have no idea what I am talking about.) --- Thanks, Paul Paul Szabo p...@maths.usyd.edu.au http://www.maths.usy

[RFC] Reproducible OOM with partial workaround

2013-01-10 Thread paul . szabo
://bugs.debian.org/cgi-bin/bugreport.cgi?msg=101;att=1;bug=695182 (Please reply to me directly, as I am not a subscriber to the linux-mm mailing list.) Thanks, Paul Paul Szabo p...@maths.usyd.edu.au http://www.maths.usyd.edu.au/u/psz/ School of Mathematics and Statistics University of Sydney