> On Jul 14, 2017, at 7:24 PM, Taylor Swift <kelvin1...@gmail.com> wrote: > > For the sake of argument, I’m gonna claim that instead, > UnsafeMutableBufferPointer is the low level necessity, and > UnsafeMutablePointer is the low level convenience structure. > > Suppose that every block of memory has a starting point, and a length. An > UnsafeBufferPointer represents that, since all an UnsafeBufferPointer is, is > a start address, and a length value, bundled together into one variable > instead of two. Then we can say that UnsafePointer is a special case of > UnsafeBufferPointer, where the length value is unknown. > > Mind you I didn’t say that the length value equals 1. But it might as well > be. Because when the length is unknown, the only index where you can > reasonably expect there to be a valid value is index zero. You wouldn’t feel > confident trying to do something to the memory address 3 strides down from > the pointer. If you did feel confident doing that, well now you have a length > value. The length value is 4. Your regular Pointer is now a BufferPointer. > > The only reason any of this matters, is because all of our memory and > memorystate -related functions take size arguments. You have to fill in the > size argument with something. That something is the buffer `count`. Even if > you don’t actually know the number for the `count`, you still have to supply > some value, in which case the “only” sensible choice is “1”. That’s why you > could argue that UnsafePointer is just a special case of a buffer pointer > where `count` is 1. The UnsafePointer API demands information that only > BufferPointers know. You could design an API where buffer pointers are the > only pointers that exist, and everything would still work fine. Just because > they have high-level capabilities doesn’t mean they can’t do everything plain > pointers can do just as efficiently. A plain pointer takes up one word of > storage, and you use another word of storage yourself to track the size. A > buffer pointer stores the two words next to each other. You could ignore the > size word and track the size yourself, taking up three words of storage, but > that would just be silly. > > But wait! I don’t need to track the size, you say! The pointer just points to > one item! I just need the address word, not the size word! Well, yeah. That’s > why I proposed a sizeless singular pointer API, so you don’t have to go > around plugging “1”s everywhere and save yourself a word of storage. You can > argue that’s the actual high-level API, since it abstracts away the length > stuff. But if “1” isn’t your answer to the question “on how many instances > should this method operate on”, go back to paragraph 3 and convince yourself > that what you really have is an buffer pointer, not a plain pointer. > > Now I thought of an exception to this, like when you’re assigning 8-bit RGBA > values to an image buffer and it might make sense to write something like > this in the current API: > > let size:Int = height * width << 2 > let base = UnsafeMutablePointer<UInt8>.allocate(capacity: size) > defer > { > base.deallocate(capacity: -314159) // anything goes > } > var pixel:UnsafeMutablePointer<UInt8> = base > while pixel < base + size > { > pixel.initialize(from: bg_color_rgba, count: 4) > pixel += 4 > } > > And it’s convenient to be able to initialize 4 instances at a time without > creating a 4-count buffer pointer. But this doesn’t really contradict my > point if you think carefully. The groups of 4 RGBA values are your atom here, > you’re not really working on 4 instances at a time with a stride 1 UInt8 big, > you’re working on 1 single instance at a time with a stride 4 UInt8s big. > Writing this is currently painful without the sized UnsafeMutablePointer API, > but I think this is a deficiency of the UnsafeMutableBufferPointer API and > that of its slice type, not a reason for keeping the sized > UnsafeMutablePointer API, which if you ask me is a hack to get around the > fact that we can’t use buffer pointer slices easily. > > Anyway, I mention all of this only because everyone seems convinced that > UnsafePointer is supposed to be “edgier” than UnsafeBufferPointer. If you > have ideas to make this better, by all means share them
UnsafeBufferPointer was developed to add safety and convenience on top of UnsafePointer. That’s just a historical fact, not necessarily detracting from your argument. The problem with your use case above is that UnsafePointer doesn’t get the debug-mode bounds checks. There’s an argument to be made for keeping the UnsafePointer API complete (as a single point of truth for the memory model), and an argument to be made for removing it (it’s not the *right* API to use in nearly all cases). I would be fine with either approach and people can argue it out. At the moment I would lean toward not breaking source for something that isn’t a critical problem for the language. That doesn’t need to be resolved to make UnsafeBufferPointer clean and convenient. -Andy > On Fri, Jul 14, 2017 at 2:27 PM, Michael Ilseman <milse...@apple.com > <mailto:milse...@apple.com>> wrote: > >> On Jul 13, 2017, at 6:55 PM, Taylor Swift via swift-evolution >> <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote: >> >> >> >> On Thu, Jul 13, 2017 at 6:56 PM, Andrew Trick <atr...@apple.com >> <mailto:atr...@apple.com>> wrote: >> >>> On Jul 12, 2017, at 12:16 PM, Taylor Swift via swift-evolution >>> <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote: >>> >>> Hi all, I’ve written up a proposal to modify the unsafe pointer API for >>> greater consistency, safety, and ease of use. >>> >>> ~~~ >>> >>> Swift currently offers two sets of pointer types — singular pointers such >>> as UnsafeMutablePointer, and vector (buffer) pointers such as >>> UnsafeMutableBufferPointer. This implies a natural separation of tasks the >>> two kinds of pointers are meant to do. For example, buffer pointers >>> implement Collection conformance, while singular pointers do not. >>> >>> However, some aspects of the pointer design contradict these implied roles. >>> It is possible to allocate an arbitrary number of instances from a type >>> method on a singular pointer, but not from a buffer pointer. The result of >>> such an operation returns a singular pointer, even though a buffer pointer >>> would be more appropriate to capture the information about the number of >>> instances allocated. It’s possible to subscript into a singular pointer, >>> even though they are not real Collections. Some parts of the current design >>> turn UnsafePointers into downright DangerousPointers, leading users to >>> believe that they have allocated or freed memory when in fact, they have >>> not. >>> >>> This proposal seeks to iron out these inconsistencies, and offer a more >>> convenient, more sensible, and less bug-prone API for Swift pointers. >>> >>> <https://gist.github.com/kelvin13/a9c033193a28b1d4960a89b25fbffb06 >>> <https://gist.github.com/kelvin13/a9c033193a28b1d4960a89b25fbffb06>> >>> >>> ~~~ >>> >> >> Thanks for taking time to write this up. >> >> General comments: >> >> UnsafeBufferPointer is an API layer on top of UnsafePointer. The role >> of UnsafeBufferPointer is direct memory access sans lifetime >> management with Collection semantics. The role of UnsafePointer is >> primarily C interop. Those C APIs should be wrapped in Swift APIs that >> take UnsafeBufferPointer whenever the pointer represents a C array. I >> suppose making UnsafePointer less convenient would push developers >> toward UnsafeBufferPointer. I don't think that's worth outright >> breaking source, but gradual deprecation of convenience methods, like >> `susbscript` might be acceptable. >> >> Gradual deprecation is exactly what I am proposing. As the document states >> <https://gist.github.com/kelvin13/a9c033193a28b1d4960a89b25fbffb06#proposed-solution>, >> the only methods which should be marked immediately as unavailable are the >> `deallocate(capacity:)` methods, for safety and source compatibility >> reasons. Removing `deallocate(capacity:)` now and forcing a loud compiler >> error prevents catastrophic *silent* source breakage in the future, or >> worse, from having to *support our own bug*. >> >> >> I have mixed feelings about stripping UnsafePointer of basic >> functionality. Besides breaking source, doing that would be >> inconsistent with its role as a lower API layer. The advantage would >> just be descreasing API surface area and forcing developers to use a >> higher-level API. >> >> UnsafePointer is as much a high level API as UnsafeBufferPointer is. You >> wouldn’t create a buffer pointer of length 1 just so you can “stick with the >> high level API”. UnsafePointer and UnsafeBufferPointer are two tools that do >> related but different things and they can exist at whatever abstract level >> you need them at. After all, UnsafeBufferPointer is nothing but an >> UnsafePointer? with a length value attached to it. If you’re allocating more >> than one instance of memory, you almost certainly need to track the length >> of the buffer anyway. >> > > I disagree. UnsafePointer can be viewed as the low level representation of > Swift’s memory model. UnsafeBufferPointer is a convenient facility for > programming at this level, as illustrated by e.g. RandomAccessCollection > conformance. One could drop UnsafeBufferPointer from the standard library > without compromising Swift’s core capabilities, it would just be a large > convenience regression. > > UnsafePointer is a low level necessity; UnsafeBufferPointer is a low level > convenience. > >> The additive changes you propose are fairly obvious. See [SR-3088] >> UnsafeMutableBufferPointer doesn't have an allocating init. >> >> I haven't wanted to waste review cycles on small additive >> changes. It may make sense to batch them up into one coherent >> proposal. Here are a few more to consider. >> >> - [SR-3929] UnsafeBufferPointer should have init from mutable >> - [SR-4340] UnsafeBufferPointer needs a withMemoryRebound method >> - [SR-3087] No way to arbitrarily initialise an Array's storage >> >> The feature requests you mention are all very valuable, however with >> Michael’s point about fixing the memorystate API’s, the size of this >> proposal has already grown to encompass dozens of methods in five types. I >> think this says a lot about just how broken the current system is > > I fail to see how the current system is broken. It accurately reflects > Swift’s memory model while providing the facilities to stay in the bounds of > defined behavior. I do agree that it would be far more convenient for > UnsafeBufferPointer to provide allocation/deallocation and > initialization/deinitialization convenience functionality as well, as I’ve > often wanted it myself. > >> , but I think it’s better to try to fix one class of problems at a time, and >> save the less closely-related issues for separate proposals. > > These seem very closely related. Isn’t the overall goal of this proposal to > have API parity for UnsafeBufferPointer? > >> >> >> Point by point: >> >> > drop the capacity parameter from UnsafeMutablePointer.allocate() and >> > deallocate(). >> >> I do not agree with removing the capacity parameter and adding a >> single-instance allocation API. UnsafePointer was not designed for >> single instances, it was primarily designed for C-style arrays. I >> don't see the value in providing a different unsafe API for single >> vs. multiple values. >> >> Although it’s common to *receive* Unsafe__Pointers from C API’s, it’s rare >> to *create* them from the Swift side. 95% of the time your Swift data lives >> in a Swift Array, and you use withUnsafePointer(_:) to send them to the C >> API, or just pass them directly with Array bridging. >> >> The only example I can think of where I had to allocate memory from the >> Swift side to pass to a C API is when I was using the Cairo C library and I >> wanted the Swift code to own the image buffer backing the Cairo C structs >> and I wanted to manage the memory manually to prevent the buffer backing >> from getting deallocated prematurely. I think I ended up using >> UnsafeMutableBufferPointer and extracting baseAddresses to manage the >> memory. This proposal tries to mitigate that pain of extracting >> baseAddresses by giving buffer pointers their own memory management methods. >> >> As for the UnsafePointers you get from C APIs, they almost always come with >> a size (or you specify it beforehand with a parameter) so you’re probably >> going to be turning them into UnsafeBufferPointers anyway. >> >> I also have to say it’s not common to deallocate something in Swift that you >> didn’t previously allocate in Swift. >> > > I’m not sure it is even defined to use `deallocate` on memory that wasn’t > `allocated` in Swift. Andy, thoughts here? Perhaps more clarity in > deallocate’s documentation is needed? > >> >> I agree the primary allocation API should be >> UnsafeMutableBufferPointer.allocate(capacity:). There is an argument >> to be made for removing UnsafeMutablePointer.allocate(capacity:) >> entirely. But, as Michael Ilseman pointed out, that would involve >> reevaluating several other members of the UnsafePointer API. I think >> it's reasonable for UnsafePointer to retain all its functionality as a >> lower level API. >> >> >> I think duplication of functionality is something to be avoided if possible. >> > > You have to weigh duplication against convenience. If avoiding duplication is > more important, then we should keep the APIs only on UnsafePointer. But, I > would prefer convenience in this scenario. > >> I don't understand what is misleading about >> UnsafePointer.deallocate(capacity:). It *is* inconvenienent for the >> user to keep track of memory capacity. Presumably that was done so >> either the implementation can move away from malloc/free or some sort >> of memory tracking can be implemented on the standard library >> side. Obviously, UnsafeBufferPointer.deallocate() would be cleaner in >> most cases. >> >> It’s misleading because it plain doesn’t deallocate `capacity` instances. It >> deletes the whole memory block regardless of what you pass in the capacity >> argument. If the implementation is ever “fixed” so that it actually >> deallocates `capacity` instances, suddenly every source that uses >> `deallocate(capacity:)` will break, and *no one will know* until their app >> starts mysteriously crashing. If the method is not removed, we will have to >> support this behavior to avoid breaking sources, and basically say “yes the >> argument label says it deallocates a capacity, but what it *really* does is >> free the whole block and we can’t fix it because existing code assumes this >> behavior”. >> > > Note that many of these things are basically programming at the memory > semantics level, not necessarily at a execution level. > >> >> > add an allocate(count:) type method to UnsafeMutableBufferPointer >> >> `capacity` should be used for allocating uninitialized memory not >> `count`. `count` should only refer to a number of initialized objects! >> >> We can decide on what the correct term should be, but the current state of >> Swift pointers is that *neither* convention is being followed. Just look at >> the API for UnsafeMutableRawPointer. It’s a mess. This proposal at the >> minimum establishes a consistent convention. It can be revised if you feel >> `capacity` is more appropriate than `count`. If what you mean is that it’s >> important to maintain the distinction between “initialized counts” and >> “uninitialized counts”, well that can be revised in too. >> > > So I just looked over all of the APIs, and they are very consistent in their > use of `count` and `capacity`. Where are you seeing a violation? > >> > add a deallocate() instance method to UnsafeMutableBufferPointer >> >> Yes, of course! I added a mention of that in SR-3088. >> >> > remove subscripts from UnsafePointer and UnsafeMutablePointer >> >> It's often more clear to perform arithmetic on C array indices rather >> than pointers. That said, I'm happy to push developers to use >> UnsafeBufferPointer whenever that have a known capacity. To me, this >> is a question of whether the benefit of making a dangerous thing less >> convenient is worth breaking source compatibility. >> >> Again, I think this is more about what the real use patterns are. If you are >> subscripting into a C array with integers, then UnsafeBufferPointer is the >> tool for the job, since it give you Collection conformance. If you can’t >> make an UnsafeBufferPointer, it’s probably because you don’t know the length >> of the array, and so you’re probably iterating through it one element at a >> time. UnsafeMutablePointer.successor() is perfect for this job. If you want >> to extract or set fields at fixed but irregular offsets, UnsafeRawPointer is >> the tool for the job. But I’m hard-pressed to think of a use case for random >> access into a singular typed pointer. >> > > I’m also very interested in more convenience facilities for low level > programming and I view UnsafeBufferPointer as the first low level convenience > construct. The point of the very careful dance that the Unsafe*Pointers go > through is to help the programmer avoid unwittingly falling into undefined > behavior. > > >> _______________________________________________ >> swift-evolution mailing list >> swift-evolution@swift.org <mailto:swift-evolution@swift.org> >> https://lists.swift.org/mailman/listinfo/swift-evolution >> <https://lists.swift.org/mailman/listinfo/swift-evolution> >
_______________________________________________ swift-evolution mailing list swift-evolution@swift.org https://lists.swift.org/mailman/listinfo/swift-evolution