Hi Mark, Hi Ricardo, On Wed, 14 Feb 2018 20:39:12 +0100 Ricardo Wurmus <rek...@elephly.net> wrote:
> Nor do I see this message: > > ghc-pkg: unable to decommit memory: Invalid argument Which Linux kernel version does this run on? > I don’t know what this message means, but the messages about requiring a If there's large address space support [1], ghc 8 does its own allocation in a 1 TB address space. That means it has to tell the kernel when it doesn't need some chunk anymore - otherwise you're gonna run out of memory. It does that using madvise(2). There's two ways it tries to do that: (1) MADV_FREE: Signals that "I don't need that range at all anymore". It usually means Linux will mark those pages free. (2) MADV_DONTNEED: Signals that "I don't need that range in the NEAR FUTURE". It usually means Linux will swap those pages out. MADV_FREE was added in Linux 4.5. Haskell uses a #ifdef to detect it. Newer glibc (such as the one in core-updates) unconditionally define MADV_FREE to prevent programs from depending on a specific Linux kernel in this way [2]. There's a patch to ghc that falls back to (2) if (1) doesn't work: https://git.haskell.org/ghc.git/commitdiff/6576bf83cdf4eac05eb88a24aa934a736c91e3da ... but ghc 8.0.2 which we have on core-updates doesn't use it. It uses either MADV_FREE or MADV_DONTNEED, determined at compile time. So if the Linux kernel is < 4.5 that's gonna end very badly. For the record: https://ghc.haskell.org/trac/ghc/ticket/12865 Also https://github.com/NixOS/nixpkgs/issues/18118 mmap has a flag MAP_HUGETLB which would cause it to use a mounted hugetlbfs (the cgroup of which I advised to remove from GuixSD from the time being). ghc 8 does not use it so we are safe there. [1] use_large_address_space=no if test "$ac_cv_sizeof_void_p" -eq 8 ; then if test "x$EnableLargeAddressSpace" = "xyes" ; then if test "$ghc_host_os" = "darwin" ; then use_large_address_space=yes elif test "$ghc_host_os" = "openbsd" ; then # as of OpenBSD 5.8 (2015), OpenBSD does not support mmap with MAP_NORESERVE. # The flag MAP_NORESERVE is supported for source compatibility reasons, # but is completely ignored by OS mmap use_large_address_space=no else AC_CHECK_DECLS([MAP_NORESERVE, MADV_FREE, MADV_DONTNEED],[],[], [ #include <unistd.h> #include <sys/types.h> #include <sys/mman.h> #include <fcntl.h> ]) if test "$ac_cv_have_decl_MAP_NORESERVE" = "yes" && test "$ac_cv_have_decl_MADV_FREE" = "yes" || test "$ac_cv_have_decl_MADV_DONTNEED" = "yes" ; then use_large_address_space=yes fi fi fi fi if test "$use_large_address_space" = "yes" ; then AC_DEFINE([USE_LARGE_ADDRESS_SPACE], [1], [Enable single heap address space support]) fi madvise: EINVAL addr is not page-aligned or length is negative. EINVAL advice is not a valid. EINVAL advice is MADV_DONTNEED or MADV_REMOVE and the specified address range includes locked, Huge TLB pages, or VM_PFNMAP pages. EINVAL advice is MADV_MERGEABLE or MADV_UNMERGEABLE, but the kernel was not configured with CONFIG_KSM. EINVAL advice is MADV_FREE or MADV_WIPEONFORK but the specified address range includes file, Huge TLB, MAP_SHARED, or VM_PFNMAP ranges. [2] https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=981569c74cbb6bafa2ddcefa6dd9dbdc938ff1c8