Re: Storage layout of integral types

[Stefan Schulze Frielinghaus]

On Tue, Jan 19, 2021 at 05:34:04PM +0100, Stefan Schulze Frielinghaus wrote:
> Hi all,
> 
> I'm wondering what the supposed storage layout of integral types is.  In
> particular for integral types with size less than the size of a word.  For
> example, on a 64bit machine is a 32bit integer supposed to be written as a
> whole word and therefore as 64 bits or just as 32bits in the payload of a
> closure?
> 
> I'm asking because since commit be5d74ca I see differently aligned integers in
> the payload of a closure on a 64bit big-endian machine.  For example, in the
> following code an Int32 object is created which contains the actual integer in
> the high part of the payload (the snippet comes from the add operator
> GHC.Int.$fNumInt32_$c+_entry):
> 
> Hp = Hp + 16;
> ...
> I64[Hp - 8] = GHC.Int.I32#_con_info;
> I32[Hp] = _scz7::I32;
> 
> whereas e.g. in function rts_getInt32 the opposite is assumed and the actual
> integer is expected in the low part of the payload:
> 
> HsInt32
> rts_getInt32 (HaskellObj p)
> {
> // See comment above:
> // ASSERT(p->header.info == I32zh_con_info ||
> //p->header.info == I32zh_static_info);
> return (HsInt32)(HsInt)(UNTAG_CLOSURE(p)->payload[0]);
> }
> 
> The same seems to be the case for the interpreter and foreign calls (case
> bci_CCALL) where integral arguments are passed in the low part of a whole 
> word.
> 
> Currently, my intuition is that the payload of a closure for an integral type
> with size smaller than WordSize is written as a whole word where the subword 
> is
> aligned according to the machines endianness.  Can someone confirm this?  If
> that is indeed true, then rts_getInt32 seems to be correct but not the former.
> Otherwise the converse seems to be the case.

Interestingly it looks like as if 32bit floats are also only accessed as 32bit
values, whereas for characters we have that a 32bit subword is supposed to be
accessed as a 64bit word:

 section ""data" . stg_CHARLIKE_closure" {
 stg_CHARLIKE_closure:
 const ghczmprim_GHCziTypes_Czh_con_info;
 const 0;
 ...

Thus in total what I see is that the payload of a closure for
I{8,16,32}# and F# are read/written according to their natural size,
respectively (ignoring the inconsistency in rts_getInt* for a moment).
In contrast to the payload of a C# closure which is read/written as a
64bit value although the natural size is 32bit only.

Can someone confirm these observations?  What is the general direction:
are subwords supposed to be read/written according to their natural size
only?

Cheers,
Stefan
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Storage layout of integral types

[Stefan Schulze Frielinghaus]

Hi all,

I'm wondering what the supposed storage layout of integral types is.  In
particular for integral types with size less than the size of a word.  For
example, on a 64bit machine is a 32bit integer supposed to be written as a
whole word and therefore as 64 bits or just as 32bits in the payload of a
closure?

I'm asking because since commit be5d74ca I see differently aligned integers in
the payload of a closure on a 64bit big-endian machine.  For example, in the
following code an Int32 object is created which contains the actual integer in
the high part of the payload (the snippet comes from the add operator
GHC.Int.$fNumInt32_$c+_entry):

Hp = Hp + 16;
...
I64[Hp - 8] = GHC.Int.I32#_con_info;
I32[Hp] = _scz7::I32;

whereas e.g. in function rts_getInt32 the opposite is assumed and the actual
integer is expected in the low part of the payload:

HsInt32
rts_getInt32 (HaskellObj p)
{
// See comment above:
// ASSERT(p->header.info == I32zh_con_info ||
//p->header.info == I32zh_static_info);
return (HsInt32)(HsInt)(UNTAG_CLOSURE(p)->payload[0]);
}

The same seems to be the case for the interpreter and foreign calls (case
bci_CCALL) where integral arguments are passed in the low part of a whole word.

Currently, my intuition is that the payload of a closure for an integral type
with size smaller than WordSize is written as a whole word where the subword is
aligned according to the machines endianness.  Can someone confirm this?  If
that is indeed true, then rts_getInt32 seems to be correct but not the former.
Otherwise the converse seems to be the case.

Cheers,
Stefan
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: IP/key change for gitlab.haskell.org?

[Stefan Schulze Frielinghaus]

Just in case someone else runs into a similar problem as mine which is
related to the SSH key change:

Removing the old SSH fingerprint from known_hosts file and then running
git resulted in an error for me and the culprit seems to be SSH:

$ ssh -vvvT g...@gitlab.haskell.org
...
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
Connection reset by 139.178.85.33 port 22

If I manually add

gitlab.haskell.org,139.178.85.33 ssh-ed25519 
C3NzaC1lZDI1NTE5IA7ltOZyaULDgxE3Vw6RgQVp+OPKQi79ssUenbhdWy36

to $HOME/.ssh/known_hosts, then SSH works again for me and consequently
git also.  I retrieved the known_hosts file entry manually via

$ ssh-keyscan gitlab.haskell.org >> $HOME/.ssh/known_hosts

which interestingly works without any problem.  Note, I'm running a
stock Fedora 32 without any SSH configuration changes except adding my
SSH key.  This is still kind of mystic to me and so far I couldn't
figure out why this is the case, but at least it solves my problem.

Hope this helps in case someone else runs into the same problem.

Cheers,
Stefan


On Thu, Jun 25, 2020 at 11:41:12AM +0100, Richard Eisenberg wrote:
> An innocent `git push` yielded this today:
> 
> @@@
> @   WARNING: POSSIBLE DNS SPOOFING DETECTED!  @
> @@@
> The ED25519 host key for gitlab.haskell.org has changed,
> and the key for the corresponding IP address 139.178.85.33
> is unknown. This could either mean that
> DNS SPOOFING is happening or the IP address for the host
> and its host key have changed at the same time.
> @@@
> @WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
> @@@
> IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
> Someone could be eavesdropping on you right now (man-in-the-middle attack)!
> It is also possible that a host key has just been changed.
> The fingerprint for the ED25519 key sent by the remote host is
> SHA256:/dI7zsBRZNPB+0TqskF7rSaZ/LhQw0cF4c5W+4uMlRo.
> Please contact your system administrator.
> Add correct host key in /Users/rae/.ssh/known_hosts to get rid of this 
> message.
> Offending ED25519 key in /Users/rae/.ssh/known_hosts:21
> ED25519 host key for gitlab.haskell.org has changed and you have requested 
> strict checking.
> Host key verification failed.
> 
> I know the server had a rough weekend. Is this a natural consequence, or is 
> something fishy going on?
> 
> Thanks,
> Richard
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

[hadrian] Use LLVM for test and validate target

[Stefan Schulze Frielinghaus]

Hi all,

Is it possible to force Hadrian to use the LLVM code generator for the
test and validate targets?  For example, while using the following
commands it seems to me that only the native code generator (if
available) is used:

$ ./hadrian/build -c -j --flavour=quick-llvm test
$ ./hadrian/build -c -j --flavour=validate

Cheers,
Stefan
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: stage2 build fails

[Stefan Schulze Frielinghaus]

I ran into a similar(?) problem a while ago. "make maintainer-clean"
does not necessarily remove every generated file. See for example

   https://mail.haskell.org/pipermail/ghc-devs/2019-August/018049.html

Since then I only use

  git clean -dfxq && git submodule foreach git clean -dfxq

which worked reliably for me (beware that this removes any untracked
file). Maybe this helps?

Cheers,
Stefan

On Tue, Jan 28, 2020 at 10:41:33AM +, Simon Peyton Jones via ghc-devs wrote:
> I did "sh validate --legacy" which uses "make maintainer-clean".
> 
> I'll try again to double check.  Is there any particular file I should remove?
> 
> Simon
> 
> |  -Original Message-
> |  From: Ben Gamari 
> |  Sent: 27 January 2020 18:59
> |  To: Simon Peyton Jones ; ghc-devs  |  d...@haskell.org>
> |  Subject: RE: stage2 build fails
> |  
> |  Simon Peyton Jones  writes:
> |  
> |  > |  Indeed it would be nice to know which compiler you are using to
> |  > |  bootstrap. I suspect Sylvain is correct that the alternative can be
> |  > |  removed but first I would like to understand why this is arising only
> |  > |  now.
> |  >
> |  > I'm using 8.6.4 as my bootstrap compiler. But this message occurs only
> |  when compiling the *stage2* compiler with the *stage1* compiler. At that
> |  moment the bootstrap compiler is irrelevant.
> |  >
> |  > It's as if the build system doesn't know that the stage1 compiler is >=
> |  8.10
> |  >
> |  Hmm, interesting. Has this tree been cleaned in the last month or two?
> |  I suspect that this may have something to do with the changes to the
> |  GHC_STAGE macro that were merged a while back (perhaps a stale header
> |  file?).
> |  
> |  Cheers,
> |  
> |  - Ben
> 
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: GHC_STAGE

[Stefan Schulze Frielinghaus]

Hi Shayne,

does commit 35cc5eff fix your problem?

Cheers,
Stefan

On Thu, Oct 10, 2019 at 07:41:47AM -0400, Shayne Fletcher via ghc-devs wrote:
> Hi Ben,
> 
> I observed warnings of this kind in a couple of places building head
> yesterday:
> ```
> compiler/utils/FastString.hs:336:5: error:
>  warning: 'GHC_STAGE' is not defined, evaluates to 0 [-Wundef]
> |
> 336 | #if GHC_STAGE < 2
> | ^
> #if GHC_STAGE < 2
> ^
> ```
> 
> ```
> compiler/main/DynFlags.hs:323:5: error:
>  warning: 'GHC_STAGE' is not defined, evaluates to 0 [-Wundef]
> |
> 323 | #if GHC_STAGE >= 2
> | ^
> #if GHC_STAGE >= 2
> ^
> ```
> I was using the `hadrian/build.stack.sh` approach. Figured you'd want to
> know!
> 
> -- 
> *Shayne Fletcher*
> Language Engineer */* +1 917 699 7663
> *Digital Asset* , creators of *DAML
> *
> 
> -- 
> This message, and any attachments, is for the intended recipient(s) only, 
> may contain information that is privileged, confidential and/or proprietary 
> and subject to important terms and conditions available at 
> http://www.digitalasset.com/emaildisclaimer.html 
> . If you are not the 
> intended recipient, please delete this message.

> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Test cases and missing RTS linker

[Stefan Schulze Frielinghaus]

Hi all,

How is a test case supposed to be marked if it makes use of the RTS linker
which itself is not supported on some architectures?

For example, some test cases are marked for a particular architecutre like
PowerPC64 that it fails because of a missing RTS linker.

Style (A)

  test('T10052', [when(arch('powerpc64') or arch('powerpc64le'),
  expect_broken(11259)), req_interp],
   makefile_test, ['T10052'])

Whereas another test case is marked to be skipped if RTS linker is missing.

Style (B)

  test('static-plugins',
   [extra_files(['simple-plugin/']),
unless(config.have_RTS_linker, skip),
expect_broken_for(16803, prof_ways),
extra_run_opts('"' + config.libdir + '"')],
   compile_and_run,
   ['-package ghc -isimple-plugin/'])

If style (A) is the way to go, then for every architecture which does not
support the RTS linker a when-clause must be added.

If style (B) is the way to go, then shouldn't we change all test cases of style
(A) to style (B)?

Personally I prefer style (B) over (A) because it makes it easier to add an
architecture where the RTS linker is not provided. Even if the RTS linker is
provided after some time, then the test case must not be changed. Once
PowerPC64 supports the RTS linker all when-clauses must be removed.

Any thoughts which style to use?

Cheers,
Stefan
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Cabal reports mismatched interface file ways dyn

[Stefan Schulze Frielinghaus]

I'm resurrecting an old thread because I thought that this might also be
helpful for others. The problem I encountered was a result of old build
artifacts which didn't get cleaned up.

Assume you have build GHC before and now you changed something e.g. the
LLVM compiler. Thus the build system won't detect automatically that you
have to recompile and therefore you want to go for a full build by doing
a clean in the first place.

$ make maintainer-clean
$ git status --ignored
...
Ignored files:
libraries/ghc-heap/dist-boot
...

The maintainer-clean target does not remove all build artifacts. Thus it
is more save to perform a `git clean -dfx`. In my case this solved the
problem.

Cheers,
Stefan
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Cabal reports mismatched interface file ways dyn

[Stefan Schulze Frielinghaus]

Yes, it looks so:

$ ls -1 libraries/ghc-prim/dist-install/build/GHC/CString.*
libraries/ghc-prim/dist-install/build/GHC/CString.dyn_hi
libraries/ghc-prim/dist-install/build/GHC/CString.dyn_o
libraries/ghc-prim/dist-install/build/GHC/CString.hi
libraries/ghc-prim/dist-install/build/GHC/CString.o

$ inplace/bin/ghc-stage2 --show-iface 
libraries/ghc-prim/dist-install/build/GHC/CString.hi | grep --after-context=1 
Way
Way: Wanted [d, y, n],
 got[]

On Thu, Jul 11, 2019 at 11:03:16AM +0100, Matthew Pickering wrote:
> Does the quick flavour build the dynamic libraries?
> 
> On Thu, Jul 11, 2019 at 10:07 AM Stefan Schulze Frielinghaus
>  wrote:
> >
> > Hi all,
> >
> > I'm trying to compile GHC 8.6.5 using the LLVM backend and dynamic linking. 
> > My
> > build.mk file looks as follows:
> >
> > include mk/flavours/quick-llvm.mk
> > DYNAMIC_BY_DEFAULT = YES
> >
> > I can build a stage3 compiler. However while executing `make install` the
> > following error shows up:
> >
> > "inplace/bin/ghc-cabal" register libraries/ghc-prim dist-install 
> > "/devel/ghc86/lib/ghc-8.6.5/bin/ghc" 
> > "/devel/ghc86/lib/ghc-8.6.5/bin/ghc-pkg" "/devel/ghc86/lib/ghc-8.6.5" '' 
> > '/devel/ghc86' '/devel/ghc86/lib/ghc-8.6.5' 
> > '/devel/ghc86/share/doc/ghc-8.6.5/html/libraries' NO
> > ghc-cabal: '/devel/ghc86/lib/ghc-8.6.5/bin/ghc' exited with an error:
> > Bad interface file: dist-install/build/GHC/CString.hi
> > mismatched interface file ways (wanted "dyn", got "")
> >
> > I found similar reports [1,2] from 2013 but no solution. Any ideas how to 
> > fix
> > this?
> >
> > Cheers,
> > Stefan
> >
> > [1] https://mail.haskell.org/pipermail/ghc-devs/2013-December/003488.html
> > [2] https://mail.haskell.org/pipermail/ghc-devs/2013-December/003507.html
> > ___
> > ghc-devs mailing list
> > ghc-devs@haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Cabal reports mismatched interface file ways dyn

[Stefan Schulze Frielinghaus]

Hi all,

I'm trying to compile GHC 8.6.5 using the LLVM backend and dynamic linking. My
build.mk file looks as follows:

include mk/flavours/quick-llvm.mk
DYNAMIC_BY_DEFAULT = YES

I can build a stage3 compiler. However while executing `make install` the
following error shows up:

"inplace/bin/ghc-cabal" register libraries/ghc-prim dist-install 
"/devel/ghc86/lib/ghc-8.6.5/bin/ghc" "/devel/ghc86/lib/ghc-8.6.5/bin/ghc-pkg" 
"/devel/ghc86/lib/ghc-8.6.5" '' '/devel/ghc86' '/devel/ghc86/lib/ghc-8.6.5' 
'/devel/ghc86/share/doc/ghc-8.6.5/html/libraries' NO
ghc-cabal: '/devel/ghc86/lib/ghc-8.6.5/bin/ghc' exited with an error:
Bad interface file: dist-install/build/GHC/CString.hi
mismatched interface file ways (wanted "dyn", got "")

I found similar reports [1,2] from 2013 but no solution. Any ideas how to fix
this?

Cheers,
Stefan

[1] https://mail.haskell.org/pipermail/ghc-devs/2013-December/003488.html
[2] https://mail.haskell.org/pipermail/ghc-devs/2013-December/003507.html
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Vector registers assumed to be caller or callee-saved?

[Stefan Schulze Frielinghaus]

But this only includes the NCG. What about the LLVM backend? For LLVM I
only found in compiler/llvmGen/LlvmCodeGen/CodeGen.hs function
definition getTrashRegs which makes use of function callerSaves which is
defined in includes/CodeGen.Platform.hs:

callerSaves :: GlobalReg -> Bool
#if defined(CALLER_SAVES_Base)
callerSaves BaseReg   = True
#endif
...
callerSaves _ = False

There only for general-purpose and floating-point registers function
callerSaves may be defined to True. Thus, for XMMi, YMMi, and ZMMi
arguments the function evaluates to False.

Do I miss something for the LLVM backend? Maybe we just need to extend
the definition of callerSaves in order to respect vector registers, too?

Cheers,
Stefan


On Sun, Jun 30, 2019 at 07:16:15PM +0200, Andreas Klebinger wrote:
> What you want is not the macro but this function:
> https://hackage.haskell.org/package/ghc-8.6.5/docs/src/X86.Regs.html#callClobberedRegs
>
>
> whose results depend on the System ABI.
>
> Cheers,
> Andreas
>
>
>
>
> > Hi all,
> >
> > I'm wondering what GHC assumes about vector registers XMMi, YMMi, and ZMMi 
> > used
> > by the STG machine: are those assumed to be caller or callee-saved? Only for
> > the x86-64 architecture there exist macro definitions like 
> > CALLER_SAVES_XMM1 in
> > includes/stg/MachRegs.h.  However, I cannot find any other place where those
> > macros are used.  AFAIK most C ABIs assume that vector registers are call
> > clobbered. Is this also the case for GHC?
> >
> > Many thanks in advance,
> > Stefan
> >
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Vector registers assumed to be caller or callee-saved?

[Stefan Schulze Frielinghaus]

Hi all,

I'm wondering what GHC assumes about vector registers XMMi, YMMi, and ZMMi used
by the STG machine: are those assumed to be caller or callee-saved? Only for
the x86-64 architecture there exist macro definitions like CALLER_SAVES_XMM1 in
includes/stg/MachRegs.h.  However, I cannot find any other place where those
macros are used.  AFAIK most C ABIs assume that vector registers are call
clobbered. Is this also the case for GHC?

Many thanks in advance,
Stefan
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

StgRun and reserved C stack space

[Stefan Schulze Frielinghaus]

Hi all,

I'm trying to understand how StgRun/StgReturn are supposed to be implemented.
My current understanding is that StgRun is called from C land in order to save
all callee-saved registers, allocate some stack space, and then finally jump to
an STG function.  StgReturn is the counterpart of StgRun and is called from STG
land in order to restore all callee-saved registers, free the previously
allocated stack space, and finally return to C land.

Assuming that my intuition is correct I still have some concerns regarding the
stack space which is allocated by StgRun.  StgRun allocates
RESERVED_C_STACK_BYTES many bytes on the C stack.  However, I cannot see how an
STG function is supposed to access this reserved area since no pointer is
derived or whatsoever.  Does an STG function make use of the C stack pointer
and assume that the reserved area is at the very bottom of the stack?

Lets make an example.  Assume the C ABI demands that every function
allocates a Register Save Area (RSA for short) at the bottom of a stack
frame which may be used by a subsequent C function.  That means a caller
allocates some stack space for its callee.  This area must be below the
local variable and parameter area which means it is at the very bottom
of a stack frame.  A C stack frame then looks as follows:

  . .
  . .
  | previous frame  |
  +-+
  | local variables |
  +-+
  | parameter area  |
  +-+
  | register save area  |
  +-+  <---+ SP

Thus StgRun must also allocate a register save area which is used by subsequent
C functions.  An example stack while calling StgRun could look as follows:

  . .
  . .
  | |
  +-+
RSA1  | register save area  |
  | used by StgRun  |
  +-+ xxx
  | |   xx
  | |xx
  | | xx
  | | xx
  | |  xx RESERVED_C_STACK_BYTES
  | | xx
  | | xx
  | |xx
  | |   xx
  +-+ xxx
RSA2  | register save area  |
  | for further C calls |
  +-+  <---+ SP

RSA1 is allocated by the caller of StgRun and is used by StgRun in order to
save all callee-saved registers.  StgRun itself then allocates
RESERVED_C_STACK_BYTES many bytes and RSA2.  RSA2 will be used by subsequent C
functions which are called from STG land.

Thus coming back to my initial question how is an STG function supposed to know
where the reserved area starts?  One way would be to use SP + len(RSA2) but
this would mean an STG function is aware of the register in which the C stack
pointer is hold and the length of a register save area.  I didn't find anything
which suggests this while skimming over the code.

Many thanks and kind regards,
Stefan
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs