Just a question: how would/does allowing the reordering of fields affect the
correctness and performance of the (de)serialization API added in Swift 4?
> On Jul 9, 2017, at 6:21 PM, Jens Persson via swift-evolution
> <swift-evolution@swift.org> wrote:
>
> Thanks for that clarification John McCall.
> My code is using a lot of generic structs (in which memory layout is
> important) though, an example would be:
> struct Vector4<E> : Vector {
> typealias Index = VectorIndex4
> typealias Element = E
> var e0, e1, e2, e3: Element
> …
> }
> And I guess I'm out of luck trying to represent those as C structs?
> So AFAICS it looks like it is currently impossible to write generic low level
> code in Swift, unless I just keep doing what I've been doing (It does
> currently work after all) knowing that it will probably break in some future
> versions of Swift. But in that possible future version of Swift, I could
> probably find a way to make it work again (using some possible explicit tools
> for layout control present in that version of Swift).
> Correct?
> /Jens
>
>
>> On Sun, Jul 9, 2017 at 11:41 PM, John McCall <rjmcc...@apple.com> wrote:
>>
>>> On Jul 9, 2017, at 4:49 PM, Jens Persson via swift-evolution
>>> <swift-evolution@swift.org> wrote:
>>>
>>> Sorry for making so much off topic noise in this thread, but I made a
>>> mistake regarding the Metal tutorial:
>>> Looking more carefully I see now that they rebuild a vertedData: [Float]
>>> from their vertices: [Vertex] using the floatBuffer() method of the Vertex
>>> struct, which returns an Array with the stored properties of Vertex in
>>> correct order.
>>>
>>> While searching the internet about this I saw Joe Groff mentioning on
>>> Twitter that:
>>> "We plan to sort fields in padding order to minimize size, and may also
>>> automatically pack bools and enums in bitfields."
>>>
>>> So AFAICS my current image processing code is making the possibly invalid
>>> assumption that eg
>>> struct S {
>>> var a, b, c, d: Float
>>> }
>>> will have a memory layout of 4*4=16 bytes (stride and size == 16) and an
>>> alignment of 4, and most importantly that a, b, c, d will be in that order.
>>
>> This is currently true, but may not always be. We want to reserve the right
>> to re-order the fields even if it doesn't improve packing — for example, if
>> two fields are frequently accessed together, field reordering could yield
>> substantial locality benefits. We've also discussed reordering fields to
>> put them in a canonical order for resilience, since it's a little
>> counter-intuitive that reordering the fields of a struct should be
>> ABI-breaking. (There are arguments against doing that as well, of course —
>> for example, the programmer may have chosen the current order for their own
>> locality optimizations.)
>>
>>> It looks like I should be defining my structs (the ones for which memory
>>> layout is important) in C and import them.
>>
>> This should always work, yes.
>>
>>> Although I would be surprised if a Swift-struct containing only same-sized
>>> fields (all of the same primitive type) would be reordered, and such
>>> changes to the language would probably include some per-struct way to
>>> express some sort of layout control (since being able to define structs to
>>> be used for low level data manipulation is important in a systems language).
>>
>> Exactly. In the long term, Swift will have some explicit tools for layout
>> control.
>>
>> John.
>>
>>>
>>> /Jens
>>>
>>>
>>>> On Sun, Jul 9, 2017 at 7:01 PM, Jens Persson via swift-evolution
>>>> <swift-evolution@swift.org> wrote:
>>>> I should perhaps add that in my image processing code, I use code like
>>>> this:
>>>>
>>>> func withVImageBuffer<Data, R>(for table: Table<Data>, body:
>>>> (vImage_Buffer) -> R) -> R
>>>> where
>>>> Data.Coordinate.Index == VectorIndex2
>>>> {
>>>> let vib = vImage_Buffer(
>>>> data: table.baseAddress,
>>>> height: vImagePixelCount(table.size.e1),
>>>> width: vImagePixelCount(table.size.e0),
>>>> rowBytes: table.stride.e1
>>>> )
>>>> return withExtendedLifetime(table) { body(vib) }
>>>> }
>>>>
>>>> Here, Table<Data> is the raster image. Data.Coordinate == VectorIndex2
>>>> makes it a 2D table, and a Table's Data also has a type parameter
>>>> Data.Value which can be eg one of the "pixel"-struct I showed before.
>>>> This works without any problems (I've tested and used the some variant of
>>>> this type of code for years) but it would surely break if the memory
>>>> layout of simple structs changed.
>>>>
>>>> I can't see how this usage is much different from the one in the Metal
>>>> tutorial. It too uses pointers to point into a data created using the
>>>> (Swift) struct "Vertex", and the GPU hardware has its expectations on the
>>>> memory layout of that data, so the code would break if the memory layout
>>>> of the Vertex struct changed.
>>>>
>>>> /Jens
>>>>
>>>>
>>>>> On Sun, Jul 9, 2017 at 6:35 PM, Jens Persson <j...@bitcycle.com> wrote:
>>>>
>>>>> I don't think I'm misunderstanding you, but I might be, so I'll add more
>>>>> detail:
>>>>>
>>>>> If you look at the Metal article, you'll see that the (Swift) struct
>>>>> "Vertex" is used to specify the data that is sent to Metal for creating a
>>>>> buffer (using MTLDevice.makeBuffer). The result that the GPU will produce
>>>>> surely depends on the fields of the Vertex struct (x, y, z, r, g, b, a)
>>>>> being in the specified order (ie swapping the red channel with the x
>>>>> coordinate would produce an unexpected result).
>>>>>
>>>>> And regarding the second example, pixel structs used for manipulating
>>>>> raster image data. Manipulating raster image data presumably includes
>>>>> stuff like displaying to screen, loading and saving raster images.
>>>>> I currently use this way of doing this right now without any problems,
>>>>> but if the order of the fields (eg a, r, g, b) should change in the
>>>>> future, then my code would break (the colors of the images would at least
>>>>> not come out as expected).
>>>>>
>>>>> /Jens
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> On Sun, Jul 9, 2017 at 5:53 PM, Chris Lattner <clatt...@nondot.org>
>>>>>> wrote:
>>>>>>
>>>>>>> On Jul 9, 2017, at 12:23 AM, Jens Persson <j...@bitcycle.com> wrote:
>>>>>>>
>>>>>>>
>>>>>>>> On Sat, Jul 8, 2017 at 6:28 PM, Chris Lattner via swift-evolution
>>>>>>>> <swift-evolution@swift.org> wrote:
>>>>>>>> Hi Susan,
>>>>>>>>
>>>>>>>> Swift does not currently specify a layout for Swift structs. You
>>>>>>>> shouldn’t be using them for memory mapped i/o or writing to a file,
>>>>>>>> because their layout can change. When ABI stability for fragile
>>>>>>>> structs lands, you will be able to count on it, but until then
>>>>>>>> something like this is probably a bad idea.
>>>>>>>>
>>>>>>>> -Chris
>>>>>>>
>>>>>>> Does this imply that you should never use Swift structs to eg interact
>>>>>>> with Metal?
>>>>>>
>>>>>> No.
>>>>>>
>>>>>>> This seems to be a very common practice. Here is a typical example
>>>>>>> (from a Metal tutorial at raywenderlich.com):
>>>>>>>
>>>>>>> struct Vertex {
>>>>>>> var x,y,z: Float // position data
>>>>>>> var r,g,b,a: Float // color data
>>>>>>>
>>>>>>> func floatBuffer() -> [Float] {
>>>>>>> return [x,y,z,r,g,b,a]
>>>>>>> }
>>>>>>> }
>>>>>>
>>>>>> This doesn’t appear to expose the layout of the struct.
>>>>>>> Also, does it imply that we cannot use structs (of only primitive
>>>>>>> types) like:
>>>>>>>
>>>>>>> struct RgbaFloatsLinearGamma {
>>>>>>> var r, g, b, a: Float
>>>>>>> …
>>>>>>> }
>>>>>>> struct BgraBytesSrgbGamma {
>>>>>>> var b, g, r, a: UInt8
>>>>>>> }
>>>>>>>
>>>>>>> for manipulating raster image data?
>>>>>>
>>>>>> I don’t see why that would be a problem.
>>>>>>
>>>>>>> I vaguely remember a swift evo discussion where it was concluded that
>>>>>>> such usage was considered OK provided the stored properties of the
>>>>>>> structs was only primitive types, but I can't find it now.
>>>>>>>
>>>>>>> Perhaps it could be considered OK at least when the intended platforms
>>>>>>> are known to be only iOS devices?
>>>>>>
>>>>>> I think you’re misunderstanding what I’m saying. It isn’t correct to
>>>>>> take (e.g.) an unsafepointer to the beginning of a struct, and serialize
>>>>>> that out to disk, and expect that the fields are emitted in some order
>>>>>> with some specific padding between them. None of the uses above try to
>>>>>> do this.
>>>>>>
>>>>>> -Chris
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> swift-evolution mailing list
>>>> swift-evolution@swift.org
>>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>>>
>>>
>>> _______________________________________________
>>> swift-evolution mailing list
>>> swift-evolution@swift.org
>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>
>
> _______________________________________________
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution
