Re: [commit: ghc] master: Per-thread allocation counters and limits (b0534f7)
I tested my patch on an old 32 bit laptop over night and it didn't work. I think it's best to revert Simon's patch and let him fix it when he gets chance. On 3 May 2014 12:59, William Kenyon g...@abacathoo.org wrote: I think this should fix the problem. It certainly doesn't break anything on a 64 bit machine, and should fix the problem on a 32 bit machine (although I can't test that) ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: [commit: ghc] master: Per-thread allocation counters and limits (b0534f7)
On Sun, 4 May 2014 09:34:16 +0100 William Kenyon g...@abacathoo.org wrote: I tested my patch on an old 32 bit laptop over night and it didn't work. I think it's best to revert Simon's patch and let him fix it when he gets chance. On 3 May 2014 12:59, William Kenyon g...@abacathoo.org wrote: I think this should fix the problem. It certainly doesn't break anything on a 64 bit machine, and should fix the problem on a 32 bit machine (although I can't test that) To be more precise the patch hits codegen limitation: inplace/bin/ghc-stage1 -static -H32m -O -optc-march=i686 -opta-march=i686 -optl-Wl,-O1 -optl-Wl,--as-needed -optl-Wl,--hash-style=gnu -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header -Irts -Irts/dist/build -DCOMPILING_RTS -package-name rts -dcmm-lint -i -irts -irts/dist/build -irts/dist/build/autogen -Irts/dist/build -Irts/dist/build/autogen -O2-c rts/PrimOps.cmm -o rts/dist/build/PrimOps.o ghc-stage1: panic! (the 'impossible' happened) (GHC version 7.9.20140504 for i386-unknown-linux): iselExpr64(i386) I64[_cfk::P32 + 60] - %MO_SS_Conv_W32_W64((Hp + 4) - I32[_cfl::I32]) Does it make sense to have 64-bit alloc_limit on 32-bit box? -- Sergei signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: [commit: ghc] master: Per-thread allocation counters and limits (b0534f7)
On Sun, May 4, 2014 at 5:45 PM, Sergei Trofimovich sly...@gmail.com wrote: Does it make sense to have 64-bit alloc_limit on 32-bit box? I think so. You allocate 2^32 bytes pretty quickly. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: [commit: ghc] master: Per-thread allocation counters and limits (b0534f7)
According to google, 2^32 bytes = 4GB, I think that is more memory than a 32 bit machine can handle anyway? Maybe alloc_limit should be 32 bits on a 32 bit machine, and 64 bit on a 64 bit machine? On 4 May 2014 16:06, Johan Tibell johan.tib...@gmail.com wrote: On Sun, May 4, 2014 at 5:45 PM, Sergei Trofimovich sly...@gmail.comwrote: Does it make sense to have 64-bit alloc_limit on 32-bit box? I think so. You allocate 2^32 bytes pretty quickly. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: [commit: ghc] master: Per-thread allocation counters and limits (b0534f7)
I thought the limit was the total amount allocated, not the amount live. On Sun, May 4, 2014 at 7:50 PM, William Kenyon g...@abacathoo.org wrote: According to google, 2^32 bytes = 4GB, I think that is more memory than a 32 bit machine can handle anyway? Maybe alloc_limit should be 32 bits on a 32 bit machine, and 64 bit on a 64 bit machine? On 4 May 2014 16:06, Johan Tibell johan.tib...@gmail.com wrote: On Sun, May 4, 2014 at 5:45 PM, Sergei Trofimovich sly...@gmail.comwrote: Does it make sense to have 64-bit alloc_limit on 32-bit box? I think so. You allocate 2^32 bytes pretty quickly. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: [commit: ghc] master: Per-thread allocation counters and limits (b0534f7)
On 04/05/14 16:45, Sergei Trofimovich wrote: On Sun, 4 May 2014 09:34:16 +0100 William Kenyon g...@abacathoo.org wrote: I tested my patch on an old 32 bit laptop over night and it didn't work. I think it's best to revert Simon's patch and let him fix it when he gets chance. On 3 May 2014 12:59, William Kenyon g...@abacathoo.org wrote: I think this should fix the problem. It certainly doesn't break anything on a 64 bit machine, and should fix the problem on a 32 bit machine (although I can't test that) To be more precise the patch hits codegen limitation: inplace/bin/ghc-stage1 -static -H32m -O -optc-march=i686 -opta-march=i686 -optl-Wl,-O1 -optl-Wl,--as-needed -optl-Wl,--hash-style=gnu -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header -Irts -Irts/dist/build -DCOMPILING_RTS -package-name rts -dcmm-lint -i -irts -irts/dist/build -irts/dist/build/autogen -Irts/dist/build -Irts/dist/build/autogen -O2-c rts/PrimOps.cmm -o rts/dist/build/PrimOps.o ghc-stage1: panic! (the 'impossible' happened) (GHC version 7.9.20140504 for i386-unknown-linux): iselExpr64(i386) I64[_cfk::P32 + 60] - %MO_SS_Conv_W32_W64((Hp + 4) - I32[_cfl::I32]) Does it make sense to have 64-bit alloc_limit on 32-bit box? Sorry about this, just validating a revert patch to fix the builds while I figure out how to fix the breakage on 32-bit machines. The alloc limit needs to be 64 bits, even on 32-bit machines. Cheers, Simon ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: [commit: ghc] master: Per-thread allocation counters and limits (b0534f7)
On 05/02/2014 03:50 PM, g...@git.haskell.org wrote: Repository : ssh://g...@git.haskell.org/ghc On branch : master Link : http://ghc.haskell.org/trac/ghc/changeset/b0534f78a73f972e279eed4447a5687bd6a8308e/ghc --- commit b0534f78a73f972e279eed4447a5687bd6a8308e Author: Simon Marlow marlo...@gmail.com Date: Mon Apr 28 16:55:47 2014 +0100 Per-thread allocation counters and limits This tracks the amount of memory allocation by each thread in a counter stored in the TSO. Optionally, when the counter drops below zero (it counts down), the thread can be sent an asynchronous exception: AllocationLimitExceeded. When this happens, given a small additional limit so that it can handle the exception. See documentation in GHC.Conc for more details. Allocation limits are similar to timeouts, but - timeouts use real time, not CPU time. Allocation limits do not count anything while the thread is blocked or in foreign code. - timeouts don't re-trigger if the thread catches the exception, allocation limits do. - timeouts can catch non-allocating loops, if you use -fno-omit-yields. This doesn't work for allocation limits. I couldn't measure any impact on benchmarks with these changes, even for nofib/smp. --- b0534f78a73f972e279eed4447a5687bd6a8308e compiler/cmm/CmmLayoutStack.hs |9 +- compiler/codeGen/StgCmmForeign.hs | 268 ++-- includes/CodeGen.Platform.hs |4 +- includes/rts/Constants.h |6 + includes/rts/Flags.h |8 + includes/rts/Threads.h |8 +- includes/rts/storage/TSO.h | 31 ++- libraries/base/Control/Exception.hs|1 + libraries/base/Control/Exception/Base.hs |1 + libraries/base/GHC/Conc.lhs|6 + libraries/base/GHC/Conc/Sync.lhs | 92 ++- libraries/base/GHC/IO/Exception.hs | 21 +- rts/HeapStackCheck.cmm |4 +- rts/Linker.c |4 + rts/Prelude.h |2 + rts/RaiseAsync.c | 54 rts/RaiseAsync.h |4 + rts/RtsFlags.c | 10 + rts/RtsStartup.c |1 + rts/Schedule.c | 19 ++ rts/Threads.c | 77 +++--- rts/package.conf.in|2 + rts/sm/Storage.c |6 + testsuite/tests/concurrent/should_run/all.T|7 + .../tests/concurrent/should_run/allocLimit1.hs |9 + .../tests/concurrent/should_run/allocLimit1.stderr |1 + .../tests/concurrent/should_run/allocLimit2.hs | 17 ++ .../tests/concurrent/should_run/allocLimit3.hs | 15 ++ .../tests/concurrent/should_run/allocLimit3.stderr |1 + .../should_run/allocLimit3.stdout} |1 - .../tests/concurrent/should_run/allocLimit4.hs | 31 +++ utils/deriveConstants/DeriveConstants.hs |1 + 32 files changed, 576 insertions(+), 145 deletions(-) Diff suppressed because of size. To see it, use: git diff-tree --root --patch-with-stat --no-color --find-copies-harder --ignore-space-at-eol --cc b0534f78a73f972e279eed4447a5687bd6a8308e ___ ghc-commits mailing list ghc-comm...@haskell.org http://www.haskell.org/mailman/listinfo/ghc-commits I just tried to compile a snapshot involving this commit and got a compile failure: http://lpaste.net/103540 I was compiling with GHC 7.8.2 on a 32-bit machine. -- Mateusz K. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: [commit: ghc] master: Per-thread allocation counters and limits (b0534f7)
On 2014-05-03 at 08:31:47 +0200, Mateusz Kowalczyk wrote: [...] I just tried to compile a snapshot involving this commit and got a compile failure: http://lpaste.net/103540 I was compiling with GHC 7.8.2 on a 32-bit machine. The nightly 32bit GHC HEAD PPA build failed as well, see https://launchpadlibrarian.net/174440396/buildlog_ubuntu-precise-i386.ghc-head_7.9.20140502-1_FAILEDTOBUILD.txt.gz ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: [commit: ghc] master: Per-thread allocation counters and limits (b0534f7)
On 05/ 3/14 08:35 AM, Herbert Valerio Riedel wrote: On 2014-05-03 at 08:31:47 +0200, Mateusz Kowalczyk wrote: [...] I just tried to compile a snapshot involving this commit and got a compile failure: http://lpaste.net/103540 I was compiling with GHC 7.8.2 on a 32-bit machine. The nightly 32bit GHC HEAD PPA build failed as well, see https://launchpadlibrarian.net/174440396/buildlog_ubuntu-precise-i386.ghc-head_7.9.20140502-1_FAILEDTOBUILD.txt.gz All builders (solaris/freebsd/smartos) fail too. Example: http://haskell.inf.elte.hu/builders/solaris-x86-head/47/10.html build a day ago was ok: http://haskell.inf.elte.hu/builders/solaris-x86-head/46.html Thanks, Karel ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: [commit: ghc] master: Per-thread allocation counters and limits (b0534f7)
I think this should fix the problem. It certainly doesn't break anything on a 64 bit machine, and should fix the problem on a 32 bit machine (although I can't test that) On 3 May 2014 06:31, Mateusz Kowalczyk fuuze...@fuuzetsu.co.uk wrote: On 05/02/2014 03:50 PM, g...@git.haskell.org wrote: Repository : ssh://g...@git.haskell.org/ghc On branch : master Link : http://ghc.haskell.org/trac/ghc/changeset/b0534f78a73f972e279eed4447a5687bd6a8308e/ghc --- commit b0534f78a73f972e279eed4447a5687bd6a8308e Author: Simon Marlow marlo...@gmail.com Date: Mon Apr 28 16:55:47 2014 +0100 Per-thread allocation counters and limits This tracks the amount of memory allocation by each thread in a counter stored in the TSO. Optionally, when the counter drops below zero (it counts down), the thread can be sent an asynchronous exception: AllocationLimitExceeded. When this happens, given a small additional limit so that it can handle the exception. See documentation in GHC.Conc for more details. Allocation limits are similar to timeouts, but - timeouts use real time, not CPU time. Allocation limits do not count anything while the thread is blocked or in foreign code. - timeouts don't re-trigger if the thread catches the exception, allocation limits do. - timeouts can catch non-allocating loops, if you use -fno-omit-yields. This doesn't work for allocation limits. I couldn't measure any impact on benchmarks with these changes, even for nofib/smp. --- b0534f78a73f972e279eed4447a5687bd6a8308e compiler/cmm/CmmLayoutStack.hs |9 +- compiler/codeGen/StgCmmForeign.hs | 268 ++-- includes/CodeGen.Platform.hs |4 +- includes/rts/Constants.h |6 + includes/rts/Flags.h |8 + includes/rts/Threads.h |8 +- includes/rts/storage/TSO.h | 31 ++- libraries/base/Control/Exception.hs|1 + libraries/base/Control/Exception/Base.hs |1 + libraries/base/GHC/Conc.lhs|6 + libraries/base/GHC/Conc/Sync.lhs | 92 ++- libraries/base/GHC/IO/Exception.hs | 21 +- rts/HeapStackCheck.cmm |4 +- rts/Linker.c |4 + rts/Prelude.h |2 + rts/RaiseAsync.c | 54 rts/RaiseAsync.h |4 + rts/RtsFlags.c | 10 + rts/RtsStartup.c |1 + rts/Schedule.c | 19 ++ rts/Threads.c | 77 +++--- rts/package.conf.in|2 + rts/sm/Storage.c |6 + testsuite/tests/concurrent/should_run/all.T|7 + .../tests/concurrent/should_run/allocLimit1.hs |9 + .../tests/concurrent/should_run/allocLimit1.stderr |1 + .../tests/concurrent/should_run/allocLimit2.hs | 17 ++ .../tests/concurrent/should_run/allocLimit3.hs | 15 ++ .../tests/concurrent/should_run/allocLimit3.stderr |1 + .../should_run/allocLimit3.stdout} |1 - .../tests/concurrent/should_run/allocLimit4.hs | 31 +++ utils/deriveConstants/DeriveConstants.hs |1 + 32 files changed, 576 insertions(+), 145 deletions(-) Diff suppressed because of size. To see it, use: git diff-tree --root --patch-with-stat --no-color --find-copies-harder --ignore-space-at-eol --cc b0534f78a73f972e279eed4447a5687bd6a8308e ___ ghc-commits mailing list ghc-comm...@haskell.org http://www.haskell.org/mailman/listinfo/ghc-commits I just tried to compile a snapshot involving this commit and got a compile failure: http://lpaste.net/103540 I was compiling with GHC 7.8.2 on a 32-bit machine. -- Mateusz K. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs From 988ff210fe62d9a62396177891b9dcccd7f63ce0 Mon Sep 17 00:00:00 2001 From: William Kenyon git...@abacathoo.org Date: Sat, 3 May 2014 11:16:54 + Subject: [PATCH][BUILD] Fixed inconsistent word widths for an addition and subtraction This was causing build failures on 32 bit machines Signed-off-by: William Kenyon git...@abacathoo.org --- compiler/codeGen/StgCmmForeign.hs | 8 1 file
