Just a question: how would/does allowing the reordering of fields affect the correctness and performance of the (de)serialization API added in Swift 4?
> On Jul 9, 2017, at 6:21 PM, Jens Persson via swift-evolution > <swift-evolution@swift.org> wrote: > > Thanks for that clarification John McCall. > My code is using a lot of generic structs (in which memory layout is > important) though, an example would be: > struct Vector4<E> : Vector { > typealias Index = VectorIndex4 > typealias Element = E > var e0, e1, e2, e3: Element > … > } > And I guess I'm out of luck trying to represent those as C structs? > So AFAICS it looks like it is currently impossible to write generic low level > code in Swift, unless I just keep doing what I've been doing (It does > currently work after all) knowing that it will probably break in some future > versions of Swift. But in that possible future version of Swift, I could > probably find a way to make it work again (using some possible explicit tools > for layout control present in that version of Swift). > Correct? > /Jens > > >> On Sun, Jul 9, 2017 at 11:41 PM, John McCall <rjmcc...@apple.com> wrote: >> >>> On Jul 9, 2017, at 4:49 PM, Jens Persson via swift-evolution >>> <swift-evolution@swift.org> wrote: >>> >>> Sorry for making so much off topic noise in this thread, but I made a >>> mistake regarding the Metal tutorial: >>> Looking more carefully I see now that they rebuild a vertedData: [Float] >>> from their vertices: [Vertex] using the floatBuffer() method of the Vertex >>> struct, which returns an Array with the stored properties of Vertex in >>> correct order. >>> >>> While searching the internet about this I saw Joe Groff mentioning on >>> Twitter that: >>> "We plan to sort fields in padding order to minimize size, and may also >>> automatically pack bools and enums in bitfields." >>> >>> So AFAICS my current image processing code is making the possibly invalid >>> assumption that eg >>> struct S { >>> var a, b, c, d: Float >>> } >>> will have a memory layout of 4*4=16 bytes (stride and size == 16) and an >>> alignment of 4, and most importantly that a, b, c, d will be in that order. >> >> This is currently true, but may not always be. We want to reserve the right >> to re-order the fields even if it doesn't improve packing — for example, if >> two fields are frequently accessed together, field reordering could yield >> substantial locality benefits. We've also discussed reordering fields to >> put them in a canonical order for resilience, since it's a little >> counter-intuitive that reordering the fields of a struct should be >> ABI-breaking. (There are arguments against doing that as well, of course — >> for example, the programmer may have chosen the current order for their own >> locality optimizations.) >> >>> It looks like I should be defining my structs (the ones for which memory >>> layout is important) in C and import them. >> >> This should always work, yes. >> >>> Although I would be surprised if a Swift-struct containing only same-sized >>> fields (all of the same primitive type) would be reordered, and such >>> changes to the language would probably include some per-struct way to >>> express some sort of layout control (since being able to define structs to >>> be used for low level data manipulation is important in a systems language). >> >> Exactly. In the long term, Swift will have some explicit tools for layout >> control. >> >> John. >> >>> >>> /Jens >>> >>> >>>> On Sun, Jul 9, 2017 at 7:01 PM, Jens Persson via swift-evolution >>>> <swift-evolution@swift.org> wrote: >>>> I should perhaps add that in my image processing code, I use code like >>>> this: >>>> >>>> func withVImageBuffer<Data, R>(for table: Table<Data>, body: >>>> (vImage_Buffer) -> R) -> R >>>> where >>>> Data.Coordinate.Index == VectorIndex2 >>>> { >>>> let vib = vImage_Buffer( >>>> data: table.baseAddress, >>>> height: vImagePixelCount(table.size.e1), >>>> width: vImagePixelCount(table.size.e0), >>>> rowBytes: table.stride.e1 >>>> ) >>>> return withExtendedLifetime(table) { body(vib) } >>>> } >>>> >>>> Here, Table<Data> is the raster image. Data.Coordinate == VectorIndex2 >>>> makes it a 2D table, and a Table's Data also has a type parameter >>>> Data.Value which can be eg one of the "pixel"-struct I showed before. >>>> This works without any problems (I've tested and used the some variant of >>>> this type of code for years) but it would surely break if the memory >>>> layout of simple structs changed. >>>> >>>> I can't see how this usage is much different from the one in the Metal >>>> tutorial. It too uses pointers to point into a data created using the >>>> (Swift) struct "Vertex", and the GPU hardware has its expectations on the >>>> memory layout of that data, so the code would break if the memory layout >>>> of the Vertex struct changed. >>>> >>>> /Jens >>>> >>>> >>>>> On Sun, Jul 9, 2017 at 6:35 PM, Jens Persson <j...@bitcycle.com> wrote: >>>> >>>>> I don't think I'm misunderstanding you, but I might be, so I'll add more >>>>> detail: >>>>> >>>>> If you look at the Metal article, you'll see that the (Swift) struct >>>>> "Vertex" is used to specify the data that is sent to Metal for creating a >>>>> buffer (using MTLDevice.makeBuffer). The result that the GPU will produce >>>>> surely depends on the fields of the Vertex struct (x, y, z, r, g, b, a) >>>>> being in the specified order (ie swapping the red channel with the x >>>>> coordinate would produce an unexpected result). >>>>> >>>>> And regarding the second example, pixel structs used for manipulating >>>>> raster image data. Manipulating raster image data presumably includes >>>>> stuff like displaying to screen, loading and saving raster images. >>>>> I currently use this way of doing this right now without any problems, >>>>> but if the order of the fields (eg a, r, g, b) should change in the >>>>> future, then my code would break (the colors of the images would at least >>>>> not come out as expected). >>>>> >>>>> /Jens >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>> On Sun, Jul 9, 2017 at 5:53 PM, Chris Lattner <clatt...@nondot.org> >>>>>> wrote: >>>>>> >>>>>>> On Jul 9, 2017, at 12:23 AM, Jens Persson <j...@bitcycle.com> wrote: >>>>>>> >>>>>>> >>>>>>>> On Sat, Jul 8, 2017 at 6:28 PM, Chris Lattner via swift-evolution >>>>>>>> <swift-evolution@swift.org> wrote: >>>>>>>> Hi Susan, >>>>>>>> >>>>>>>> Swift does not currently specify a layout for Swift structs. You >>>>>>>> shouldn’t be using them for memory mapped i/o or writing to a file, >>>>>>>> because their layout can change. When ABI stability for fragile >>>>>>>> structs lands, you will be able to count on it, but until then >>>>>>>> something like this is probably a bad idea. >>>>>>>> >>>>>>>> -Chris >>>>>>> >>>>>>> Does this imply that you should never use Swift structs to eg interact >>>>>>> with Metal? >>>>>> >>>>>> No. >>>>>> >>>>>>> This seems to be a very common practice. Here is a typical example >>>>>>> (from a Metal tutorial at raywenderlich.com): >>>>>>> >>>>>>> struct Vertex { >>>>>>> var x,y,z: Float // position data >>>>>>> var r,g,b,a: Float // color data >>>>>>> >>>>>>> func floatBuffer() -> [Float] { >>>>>>> return [x,y,z,r,g,b,a] >>>>>>> } >>>>>>> } >>>>>> >>>>>> This doesn’t appear to expose the layout of the struct. >>>>>>> Also, does it imply that we cannot use structs (of only primitive >>>>>>> types) like: >>>>>>> >>>>>>> struct RgbaFloatsLinearGamma { >>>>>>> var r, g, b, a: Float >>>>>>> … >>>>>>> } >>>>>>> struct BgraBytesSrgbGamma { >>>>>>> var b, g, r, a: UInt8 >>>>>>> } >>>>>>> >>>>>>> for manipulating raster image data? >>>>>> >>>>>> I don’t see why that would be a problem. >>>>>> >>>>>>> I vaguely remember a swift evo discussion where it was concluded that >>>>>>> such usage was considered OK provided the stored properties of the >>>>>>> structs was only primitive types, but I can't find it now. >>>>>>> >>>>>>> Perhaps it could be considered OK at least when the intended platforms >>>>>>> are known to be only iOS devices? >>>>>> >>>>>> I think you’re misunderstanding what I’m saying. It isn’t correct to >>>>>> take (e.g.) an unsafepointer to the beginning of a struct, and serialize >>>>>> that out to disk, and expect that the fields are emitted in some order >>>>>> with some specific padding between them. None of the uses above try to >>>>>> do this. >>>>>> >>>>>> -Chris >>>>>> >>>>>> >>>>> >>>> >>>> >>>> _______________________________________________ >>>> swift-evolution mailing list >>>> swift-evolution@swift.org >>>> https://lists.swift.org/mailman/listinfo/swift-evolution >>>> >>> >>> _______________________________________________ >>> swift-evolution mailing list >>> swift-evolution@swift.org >>> https://lists.swift.org/mailman/listinfo/swift-evolution >> > > _______________________________________________ > swift-evolution mailing list > swift-evolution@swift.org > https://lists.swift.org/mailman/listinfo/swift-evolution
_______________________________________________ swift-evolution mailing list swift-evolution@swift.org https://lists.swift.org/mailman/listinfo/swift-evolution