> On Sep 25, 2017, at 09:24, Joe Groff via swift-dev <swift-dev@swift.org> > wrote: > > > >> On Sep 24, 2017, at 10:30 PM, John McCall <rjmcc...@apple.com> wrote: >> >> >>> On Sep 22, 2017, at 8:39 PM, Saleem Abdulrasool <compn...@compnerd.org> >>> wrote: >>> >>> On Thu, Sep 21, 2017 at 10:28 PM, John McCall <rjmcc...@apple.com> wrote: >>> >>>> On Sep 21, 2017, at 10:10 PM, Saleem Abdulrasool <compn...@compnerd.org> >>>> wrote: >>>> >>>> On Thu, Sep 21, 2017 at 5:18 PM, John McCall <rjmcc...@apple.com> wrote: >>>>> On Sep 21, 2017, at 1:26 PM, Saleem Abdulrasool via swift-dev >>>>> <swift-dev@swift.org> wrote: >>>>> On Thu, Sep 21, 2017 at 12:04 PM, Joe Groff <jgr...@apple.com> wrote: >>>>> >>>>> >>>>>> On Sep 21, 2017, at 11:49 AM, Saleem Abdulrasool <compn...@compnerd.org> >>>>>> wrote: >>>>>> >>>>>> On Thu, Sep 21, 2017 at 10:53 AM, Joe Groff <jgr...@apple.com> wrote: >>>>>> >>>>>> >>>>>>> On Sep 21, 2017, at 9:32 AM, Saleem Abdulrasool via swift-dev >>>>>>> <swift-dev@swift.org> wrote: >>>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> The current layout for the swift metadata for structure types, as >>>>>>> emitted, seems to be unrepresentable in PE/COFF (at least for x86_64). >>>>>>> There is a partial listing of the generated code following the message >>>>>>> for reference. >>>>>>> >>>>>>> When building the standard library, LLVM encounters a relocation which >>>>>>> cannot be represented. Tracking down the relocation led to the type >>>>>>> metadata for SwiftNSOperatingSystemVersion. The metadata here is >>>>>>> _T0SC30_SwiftNSOperatingSystemVersionVN. At +32-bytes we find the Kind >>>>>>> (1). So, this is a struct metadata type. Thus at Offset 1 (+40 bytes) >>>>>>> we have the nominal type descriptor reference. This is the relocation >>>>>>> which we fail to represent correctly. If I'm not mistaken, it seems >>>>>>> that the field is supposed to be a relative offset to the nominal type >>>>>>> descriptor. However, currently, the nominal type descriptor is emitted >>>>>>> in a different section (.rodata) as opposed to the type descriptor >>>>>>> (.data). This cross-section relocation cannot be represented in the >>>>>>> file format. >>>>>>> >>>>>>> My understanding is that the type metadata will be adjusted during the >>>>>>> load for the field offsets. Furthermore, my guess is that the relative >>>>>>> offset is used to encode the location to avoid a relocation for the >>>>>>> load address base. In the case of windows, the based relocations are a >>>>>>> given, and I'm not sure if there is a better approach to be taken. >>>>>>> There are a couple of solutions which immediately spring to mind: >>>>>>> moving the nominal type descriptor into the (RW) data segment and the >>>>>>> other is to adjust the ABI to use an absolute relocation which would be >>>>>>> rebased. Given that the type metadata may be adjusted means that we >>>>>>> cannot emit it into the RO data segment. Is there another solution >>>>>>> that I am overlooking which may be simpler or better? >>>>>> >>>>>> IIRC, this came up when someone was trying to port Swift to Windows on >>>>>> ARM as well, and they were able to conditionalize the code so that we >>>>>> used absolute pointers on Windows/ARM, and we may have to do the same on >>>>>> Windows in general. It may be somewhat more complicated on Win64 since >>>>>> we generally assume that relative references can be 32-bit, whereas an >>>>>> absolute reference will be 64-bit, so some formats may have to change >>>>>> layout to make this work too. I believe Windows' executable loader still >>>>>> ultimately maps the final PE image contiguously, so alternatively, you >>>>>> could conceivably build a Swift toolchain that used ELF or Mach-O or >>>>>> some other format with better support for PIC as the intermediate object >>>>>> format and still linked a final PE executable. Using relative references >>>>>> should still be a win on Windows both because of the size benefit of >>>>>> being 32-bit and the fact that they don't need to be slid when running >>>>>> under ASLR or when a DLL needs to be rebased. >>>>>> >>>>>> >>>>>> Yeah, I tracked down the relativePointer thing. There is a nice subtle >>>>>> little warning that it is not fully portable :-). Would you happen to >>>>>> have a pointer to where the adjustment for the absolute pointers on WoA >>>>>> is? >>>>>> >>>>>> You are correct that the image should be contiugously mapped on Windows. >>>>>> The idea of MachO as an intermediatary is rather intriguing. Thinking >>>>>> longer term, maybe we want to use that as a global solution? It would >>>>>> also provide a nicer autolinking mechanism for ELF which is the one >>>>>> target which currently is missing this functionality. However, if Im >>>>>> not mistaken, this would require a MachO linker (and the only current >>>>>> viable MachO linker would be ld64). The MachO binary would then need to >>>>>> be converted into ELF or COFF. This seems like it could take a while to >>>>>> implement though, but would not really break ABI, so pushing that off to >>>>>> later may be wise. >>>>> >>>>> Intriguingly, LLVM does support `*-*-win32-macho` as a target triple >>>>> already, though I don't know what Mach-O to PE linker (if any) that's >>>>> intended to be used with. We implemented relative references using >>>>> current-position-relative offsets for Darwin and Linux both because that >>>>> still allows for a fairly convenient pointer-like C++ API for working >>>>> with relative offsets, and because the established toolchains on those >>>>> platforms already have to support PIC so had most of the relocations we >>>>> needed to make them work already; is there another base we could use for >>>>> relative offsets on Windows that would fit in the set of relocations >>>>> supported by standard COFF linkers? >>>>> >>>>> >>>>> Yes, the `-windows-macho` target is used for UEFI :-). The MachO binary >>>>> is translated later to PE/COFF as required by the UEFI specification. >>>>> >>>>> There are only two relocation types which can be used for relative >>>>> displacements: __ImageBase relative (IMAGE_REL_*_ADDR32NB) and section >>>>> relative (IMAGE_REL_*_SECREL) which are relative to the beginning of the >>>>> section. The latter is why I mentioned that moving them into the same >>>>> section could be a solution as that would allow the relative distance to >>>>> be encoded. Unfortunately, the section relative relocation is relative >>>>> to the section within which the symbol is. >>>> >>>> What's wrong with IMAGE_REL_AMD64_REL32? We'd have to adjust the >>>> relative-pointer logic to store an offset from the end of the relative >>>> pointer instead of the beginning, but it doesn't seem to have a section >>>> requirement. >>>> >>>> Hmm, is it possible to use RIP relative addressing in data? If so, yes, >>>> that could work. >>> >>> There's no inherent reason, but I wouldn't put it past the linker to fall >>> over and die. But it should at least be section-agnostic about the target, >>> since this is likely to be used for all sorts of PC-relative addressing. >>> >>> >>> At least MC doesnt seem to like it. Something like this for example: >>> >>> ``` >>> .data >>> data: >>> .long 0 >>> >>> .section .rodata >>> rodata: >>> .quad data(%rip) >>> ``` >>> >>> Bails out due to the unexpected modifier. Now, theoretically, we could >>> support that modififer, but it does seem pretty odd. >>> >>> Now, as it so happens, both PE and PE+ have limitations on the file size at >>> 4GiB. This means that we are guaranteed that the relative difference is >>> guaranteed to fit within 32-bits. This is where things get really >>> interesting! >>> >>> We cannot generate the relocation because we are emitting the values at >>> pointer width. However, the value that we are emitting is a relative >>> offset, which we just determined to be limited to 32-bits in width. The >>> thing is, the IMAGE_REL_AMD64_REL32 doesn't actually seem to care about the >>> cross-setionness as you pointed out. So, rather than emitting a >>> pointer-width value (`.quad`), we could emit a pad (`.long 0`) and follow >>> that with the relative displacement (`.long <expr>`). This would be >>> representable in the PE/COFF model. >>> >>> If I understand the layout correctly, the type metadata fields are supposed >>> to be pointer sized. I assume that we would like to maintain that across >>> the formats. It may be possible to alter the emission to change the >>> relative pointer emission to emit a pair of longs instead for PE/COFF with >>> a 64-bit pointer value. Basically, we cannot truncate the relocation to a >>> IMAGE_REL_AMD64_REL32 but we could generate the appropriate relocation and >>> pad to the desired width. >>> >>> Are there any pitfalls that I should be aware of trying to adjust the >>> emission to do this? The only downsides that I can see is that the >>> emission would need to be taret dependent (that is check the output object >>> format and the target pointer width). >>> >>> Thanks for the hint John! It seems that was spot on :-). >> >> Honestly, I don't know that there's a great reason for this pointer to be >> relative in the first place. The struct metadata will already have an >> absolute pointer to the value witness table which requires load-time >> relocation, so maybe we should just make this an absolute pointer, too, >> unless we're seriously considering making that a relative pointer before >> allocation. >> >> In practice this will just be a rebase, not a full relocation, so it should >> be relatively cheap. > > At one point we discussed the possibility of also making the value witness > table pointer relative, which would allow concrete value type metadata to be > fully read-only, and since code invoking a value witness is almost certainly > going to have the base type metadata pointer live, probably not an undue > burden on code size. It's a fair question though whether we'll ever get > around to that analysis, and I think the nominal type descriptor reference is > the only place we statically emit a pointer-sized rather than 32-bit relative > offset, which has caused problems for ports to other platforms that only > support 32-bit relative offsets.
That seems at odds with having standard VWTs in the runtime, though, or having a single-element struct share a VWT with its lone member. Jordan
_______________________________________________ swift-dev mailing list swift-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-dev
