Re: [swift-evolution] SE-0138 UnsafeBytes

[Andrew Trick via swift-evolution]

> On Sep 2, 2016, at 5:14 PM, Andrew Trick via swift-evolution 
>  wrote:
> 
>> 
>> On Sep 2, 2016, at 11:14 AM, Dave Abrahams via swift-evolution 
>>  wrote:
>> 
>> 
>> on Thu Sep 01 2016, Andrew Trick  wrote:
>> 
>>> I’m resending this for Review Manager Dave A. because the announce list is 
>>> dropping his messages...
>>> 
>>> Hello Swift community,
>>> 
>>> The review of "UnsafeBytes" begins now and runs through September
>>> 7th. This late addition to Swift 3 is a follow-up to SE-0107:
>>> UnsafeRawPointer. It addresses common use cases for UnsafeRawPointer,
>>> allowing developers to continue working with collections of UInt8 values,
>>> but now doing so via a type safe API. The UnsafeBytes API will not require 
>>> direct manipulation of raw pointers or reasoning about binding memory.
>>> 
>>> The proposal is available here:
>>> 
>>> >>  
>>> >
>>> 
>>> * What is your evaluation of the proposal?
>> 
>> I strongly support inclusion of the feature, but I have issues with the
> 
> Clearly, otherwise you wouldn't have announced it 4 times ;)
> 
>> name.  It seems to me that in order to fit into the standard library, it
>> should be called Unsafe[Mutable]RawBufferPointer.  Each part of the name
> 
> Well, that's natural from a stdlib designer's viewpoint. It is almost
> identical functionality, but it also exposes the UnsafeRawPointer API
> for loading and storing arbitrary types. This naming issue was
> discussed for a couple weeks on swift-evolution. Let's see if I can
> recap inline with your comments.
> 
>> conveys something important, and for the same reasons we're using
>> Unsafe[Mutable]BufferPointer instead of UnsafeMutableElements, we should
>> stick to the scheme:
>> 
>> - “Unsafe,” because you can break memory safety with this tool
> 
> OK. Let's not drop that one!
> 
>> - “Raw,” because the fundamental model is that of “raw,” rather than
>> “typed,” memory.
> 
> To me, bytes only exist in memory. Accessing a byte, as opposed to some
> in-memory type, is always a raw access.
> 
>> - “Buffer,” because it works on a series of contiguous elements of known
>> length.
> 
> To me, bytes always represent a contiguous chunk of raw memory. The
> term implies that we're dealing with memory layout, as opposed to just
> some opaque chunk of data, which is I think what Foundation Data is for.
> 
>> - “Pointer,” because it has reference semantics!  When you pass one of
>> these things around by value, you're not passing the bytes; you're
>> passing a shared reference to the bytes.
> 
> Unsafe means that this value doesn't own the memory. I agree with you
> that reference semantics are important, and we need to clearly
> distinguished this from something like Data. I just think Unsafe is
> enough for the name.
> 
> UnsafeMutableRawBufferPointer does not actually convey that it can be
> viewed as a collection of 8-bit values, which is fairly important.
> 
> Now that I've satisfied my pedantic side, let's look at it from the 
> developer's side.
> To me it's a question of whether a longer or shorter name is more meaningful 
> in
> the natural setting of users' source code:
> 
> func foo(bytes: UnsafeMutableRawBufferPointer)
> 
> withUnsafeMutableRawBufferPointer(to: &header) {
>  foo(bytes: $0)
> }
> ---
> func foo(bytes: UnsafeMutableBytes)
> 
> withUnsafeBytes(of: &header) {
>  write(bytes: $0)
> }
> 
> I don't think the longer name is more descriptive. I do think the
> shorter name is more intuitive and meaningful.
> 
> UnsafeMutableRawPointer is already too long to be recognizable to
> users. A benefit of UnsafeBytes is that the most developers won't need
> to know how to work directly with raw pointers. So the name doesn’t
> need to evoke them.
> 
> -Andy

Update on the naming debate...

I've tentatively updated this proposal renaming UnsafeBytes to 
UnsafeRawBufferPointer. It's not bad as long as the `withUnsafeBytes` name 
remains:

https://github.com/atrick/swift-evolution/blob/unsafebytes/proposals/0138-unsaferawbufferpointer.md

I've heard a few compelling arguments to use a long type name. I'm the only 
person who's argued in favor of the short name. Here are the key points that 
have convinced me to give in:

- Although I would like "bytes" to only refer to raw, untyped memory,
  in reality it means different things to different people.

- We do not want to promote using this type in public API, except as
  an alternative to other UnsafePointer related types. APIs should
  really migrate to safe, managed types.

- In practice, we may end up with overloads that the more descriptive
  type can help clarify. For example, Data.withUnsafeBytes already
  passes an UnsafePointer to its closure. That API already shipped, but
  we may want a "raw" variant of it. Having the closure take
  `UnsafeRawBufferPointer` clarifies the distinction.

Re: [swift-evolution] SE-0138 UnsafeBytes

[Andrew Trick via swift-evolution]

> On Sep 3, 2016, at 3:36 PM, Drew Crawford  wrote:
> 
> 
> On September 2, 2016 at 2:36:43 AM, Andrew Trick (atr...@apple.com 
> ) wrote:
> 
>> After thinking about this for a moment, I like the approach of extending 
>> UnsafeBytes with release-mode bounds checked versions of subscript, load, 
>> and storeBytes.
> 
> I agree with this, I think it's mostly a question of naming and defaults.  My 
> concern here is letting a swift developer accidentally write heartbleed, 
> which we can't actually prevent, but we can make it harder.
> 
> IMO 
> 
> 1.  There should be clear consistency in the checked-ness of the API surface. 
>  Agree that checked iterator makes no sense, but I think the most important 
> thing is to avoid creating a job interview trivia game where `set` is checked 
> but `store` is unchecked, spot the bug in this function.
> 
> 2.  For consistency with UnsafeBufferPointer it may make the most sense to 
> just ship unchecked or ship an opt-in checked wrapper.  I believe however 
> that the existing precedent is all wrong on this point, and I'd like to see 
> us revisit this question across both interfaces in Swift 4, but I don't want 
> to lay out a whole case here that should be its own thread.
> 
I generally agree with what you said. I think the vague plan is later in Swift 
4 to ship a bounds-checked variant of both UnsafeBufferPointer and UnsafeBytes 
(or  UnsafeRawBufferPointer if you prefer).

I don’t want to eliminate the debug-mode checks though. I did try to make it 
clear in the comments that bounds-checking only applied to debug mode, so 
developers should not accidentally become too reliant on them.

So, the only question is whether the UnsafeBytes.copyBytes() API should have 
debug or release-mode checks. My decision to keep the stronger checks here was 
probabilistic—it seems unlikely to be a performance issue but likely to catch 
most buffer overruns. But I agree that it is inconsistent, especially if we 
plan to introduce a release bounds-checked variant later. We don’t want 
developers to begin relying on that check. I’m leaning toward dropping it down 
to a debug-mode check.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-0138 UnsafeBytes

[Drew Crawford via swift-evolution]

On September 2, 2016 at 2:36:43 AM, Andrew Trick (atr...@apple.com) wrote:

After thinking about this for a moment, I like the approach of extending 
UnsafeBytes with release-mode bounds checked versions of subscript, load, and 
storeBytes.
I agree with this, I think it's mostly a question of naming and defaults.  My 
concern here is letting a swift developer accidentally write heartbleed, which 
we can't actually prevent, but we can make it harder.

IMO 

1.  There should be clear consistency in the checked-ness of the API surface.  
Agree that checked iterator makes no sense, but I think the most important 
thing is to avoid creating a job interview trivia game where `set` is checked 
but `store` is unchecked, spot the bug in this function.

2.  For consistency with UnsafeBufferPointer it may make the most sense to just 
ship unchecked or ship an opt-in checked wrapper.  I believe however that the 
existing precedent is all wrong on this point, and I'd like to see us revisit 
this question across both interfaces in Swift 4, but I don't want to lay out a 
whole case here that should be its own thread.

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-0138 UnsafeBytes

[Andrew Trick via swift-evolution]

> On Sep 2, 2016, at 11:14 AM, Dave Abrahams via swift-evolution 
>  wrote:
> 
> 
> on Thu Sep 01 2016, Andrew Trick  wrote:
> 
>> I’m resending this for Review Manager Dave A. because the announce list is 
>> dropping his messages...
>> 
>> Hello Swift community,
>> 
>> The review of "UnsafeBytes" begins now and runs through September
>> 7th. This late addition to Swift 3 is a follow-up to SE-0107:
>> UnsafeRawPointer. It addresses common use cases for UnsafeRawPointer,
>> allowing developers to continue working with collections of UInt8 values,
>> but now doing so via a type safe API. The UnsafeBytes API will not require 
>> direct manipulation of raw pointers or reasoning about binding memory.
>> 
>> The proposal is available here:
>> 
>> >  
>> >
>> 
>> * What is your evaluation of the proposal?
> 
> I strongly support inclusion of the feature, but I have issues with the

Clearly, otherwise you wouldn't have announced it 4 times ;)

> name.  It seems to me that in order to fit into the standard library, it
> should be called Unsafe[Mutable]RawBufferPointer.  Each part of the name

Well, that's natural from a stdlib designer's viewpoint. It is almost
identical functionality, but it also exposes the UnsafeRawPointer API
for loading and storing arbitrary types. This naming issue was
discussed for a couple weeks on swift-evolution. Let's see if I can
recap inline with your comments.

> conveys something important, and for the same reasons we're using
> Unsafe[Mutable]BufferPointer instead of UnsafeMutableElements, we should
> stick to the scheme:
> 
> - “Unsafe,” because you can break memory safety with this tool

OK. Let's not drop that one!

> - “Raw,” because the fundamental model is that of “raw,” rather than
>  “typed,” memory.

To me, bytes only exist in memory. Accessing a byte, as opposed to some
in-memory type, is always a raw access.

> - “Buffer,” because it works on a series of contiguous elements of known
>  length.

To me, bytes always represent a contiguous chunk of raw memory. The
term implies that we're dealing with memory layout, as opposed to just
some opaque chunk of data, which is I think what Foundation Data is for.

> - “Pointer,” because it has reference semantics!  When you pass one of
>  these things around by value, you're not passing the bytes; you're
>  passing a shared reference to the bytes.

Unsafe means that this value doesn't own the memory. I agree with you
that reference semantics are important, and we need to clearly
distinguished this from something like Data. I just think Unsafe is
enough for the name.

UnsafeMutableRawBufferPointer does not actually convey that it can be
viewed as a collection of 8-bit values, which is fairly important.

Now that I've satisfied my pedantic side, let's look at it from the developer's 
side.
To me it's a question of whether a longer or shorter name is more meaningful in
the natural setting of users' source code:

func foo(bytes: UnsafeMutableRawBufferPointer)

withUnsafeMutableRawBufferPointer(to: &header) {
  foo(bytes: $0)
}
---
func foo(bytes: UnsafeMutableBytes)

withUnsafeBytes(of: &header) {
  write(bytes: $0)
}

I don't think the longer name is more descriptive. I do think the
shorter name is more intuitive and meaningful.

UnsafeMutableRawPointer is already too long to be recognizable to
users. A benefit of UnsafeBytes is that the most developers won't need
to know how to work directly with raw pointers. So the name doesn’t
need to evoke them.

-Andy
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-0138 UnsafeBytes

[Dave Abrahams via swift-evolution]

on Fri Sep 02 2016, Andrew Trick  wrote:

>> On Sep 1, 2016, at 5:37 PM, Andrew Trick  wrote:
>> 
>> The proposal is available here:
>> 
>>  
>> 
>> >
>> 
>>> On Sep 1, 2016, at 4:59 PM, Drew Crawford
>>> >> >
>>> wrote:
>>> I'm possibly one of the larger users of raw byte stuff in Swift as
>>> I maintain an entire client/server network protocol stack in Swift
>>> userspace, similar in spirit to one of the examples drawn out a lot
>>> longer.  Grepping my code produces over 200 individual uses of
>>> unsafe byte accesses.
>>> 
>>> I definitely agree that the problem is significant enough to warrant a 
>>> last-minute change.
>>> 
>>> To a first approximation I agree with all the implementation
>>> choices.  The naming, the choice of UInt8, length tracking, and
>>> debug-bounds checking are all correct IMO.  We have been using
>>> something similar for a long time internally [have you been reading
>>> my code? :-) ] so I can speak from experience that the basic plan
>>> here is sound.
>>> 
>>> One thing I would like to see is an (opt-in)
>>> release-mode-bounds-check.  Networking is a core use case for this
>>> feature, but when you are reading from a socket, production is
>>> where you need a guard against out-of-bounds UB the most.  If we
>>> can't solve it for Swift 3, affected users can write a wrapper to
>>> implement the boundscheck, but I think we should at very least take
>>> it up again for Swift 4.
>>> 
>>> Drew
>> 
>> In my current implementation:
>> https://github.com/atrick/swift/blob/unsafebytes/stdlib/public/core/UnsafeBytes.swift.gyb
>> 
>> 
>> The bounds checks in `copyBytes(from:)` are release mode preconditions.
>> 
>> The bounds checks for `subscript`, `load(as:)`, and
>> `storeBytes(of:as:)` are debug only because it’s likely they occur
>> in some loop that could be covered by a single bounds check. By
>> extension, the sequence iterator is only bounds checked in debug
>> mode.
>> 
>> One possibility would be different names for the bounds checked
>> forms of those methods: getByte(atOffset:), setByte(atOffset:),
>> load(fromCheckedOffset:as:),
>> storeBytes(of:toCheckedOffset:as:). Along with some kind of bounds
>> checked Iterator.
>> 
>> I don’t think makes a lot of sense as generic Collection though. 
>> Alternatively, we just have an UnsafeBoundsCheckedBytes wrapper.
>> 
>> This would a good thing to experiment with in your project. We may
>> be able to follow-up with a Swift 4 proposal. The important thing
>> now is to determine whether the proposed Swift 3 design will make
>> that wrapper difficult in any way.
>
> After thinking about this for a moment, I like the approach of
> extending UnsafeBytes with release-mode bounds checked versions of
> subscript, load, and storeBytes. 

I agree, but would like to carefully design how we do this in
conjunction with Unsafe[Mutable]BufferPointer.  For example, maybe

  p.boundsChecked

gives you an UnsafeBoundsChecked[Mutable][Raw]BufferPointer.

So I think this should be a separate proposal.

> It’s not actually meaningful to have a bounds checked iterator for
> UnsafeBytes. A wrapper would only be useful to guard against
> accidentally circumventing the bounds checks, but I’m not sure that’s
> really helpful in practice. It seems that a framework would want to
> provide more abstract Socket I/O or network message abstractions and
> those wrappers would just call the bounds checked version of the
> UnsafeBytes APIs.
>
> -Andy
>
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>

-- 
-Dave

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-0138 UnsafeBytes

[Dave Abrahams via swift-evolution]

on Thu Sep 01 2016, Andrew Trick  wrote:

> I’m resending this for Review Manager Dave A. because the announce list is 
> dropping his messages...
>
> Hello Swift community,
>
> The review of "UnsafeBytes" begins now and runs through September
> 7th. This late addition to Swift 3 is a follow-up to SE-0107:
> UnsafeRawPointer. It addresses common use cases for UnsafeRawPointer,
> allowing developers to continue working with collections of UInt8 values,
> but now doing so via a type safe API. The UnsafeBytes API will not require 
> direct manipulation of raw pointers or reasoning about binding memory.
>
> The proposal is available here:
>
>  
>   
> >
>
>  * What is your evaluation of the proposal?

I strongly support inclusion of the feature, but I have issues with the
name.  It seems to me that in order to fit into the standard library, it
should be called Unsafe[Mutable]RawBufferPointer.  Each part of the name
conveys something important, and for the same reasons we're using
Unsafe[Mutable]BufferPointer instead of UnsafeMutableElements, we should
stick to the scheme:

- “Unsafe,” because you can break memory safety with this tool

- “Raw,” because the fundamental model is that of “raw,” rather than
  “typed,” memory.

- “Buffer,” because it works on a series of contiguous elements of known
  length.

- “Pointer,” because it has reference semantics!  When you pass one of
  these things around by value, you're not passing the bytes; you're
  passing a shared reference to the bytes.

>  * Is the problem being addressed significant enough to warrant a
>change to Swift?

Yes, and it fills an important funcationality gap now that we have the
unsafe pointer model nailed down.

>  
>  * Does this proposal fit well with the feel and direction of Swift?

Yes, except for the name.

>  
>  * If you have used other languages or libraries with a similar
>  feature, how do you feel that this proposal compares to those?  

I don't think any other language distinguishes raw from typed memory in
this way.

>  * How much effort did you put into your review? A glance, a quick
>  reading, or an in-depth study?

Enough ;-)

-- 
-Dave, posting as a reviewer, not a review manager

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-0138 UnsafeBytes

[Rien via swift-evolution]

Probably the biggest argument is that it will make the migration more palatable.
Most (??) apps today use some form of communication, and proving a better 
pointer solution is a big selling point.
I know that my migration would have been faster and more efficient…

Rien.

> On 02 Sep 2016, at 16:45, Félix Cloutier via swift-evolution 
>  wrote:
> 
> I'm +1 on the proposal. Not a lot to say about it; I don't expect the 
> community to have a passionate argument either.
> 
> Félix
> 
>> Le 1 sept. 2016 à 17:37:47, Andrew Trick via swift-evolution 
>>  a écrit :
>> 
>> The proposal is available here:
>> 
>>  
>> 
>> 
>>> On Sep 1, 2016, at 4:59 PM, Drew Crawford  wrote:
>>> I'm possibly one of the larger users of raw byte stuff in Swift as I 
>>> maintain an entire client/server network protocol stack in Swift userspace, 
>>> similar in spirit to one of the examples drawn out a lot longer.  Grepping 
>>> my code produces over 200 individual uses of unsafe byte accesses.
>>> 
>>> I definitely agree that the problem is significant enough to warrant a 
>>> last-minute change.
>>> 
>>> To a first approximation I agree with all the implementation choices.  The 
>>> naming, the choice of UInt8, length tracking, and debug-bounds checking are 
>>> all correct IMO.  We have been using something similar for a long time 
>>> internally [have you been reading my code? :-) ] so I can speak from 
>>> experience that the basic plan here is sound.
>>> 
>>> One thing I would like to see is an (opt-in) release-mode-bounds-check.  
>>> Networking is a core use case for this feature, but when you are reading 
>>> from a socket, production is where you need a guard against out-of-bounds 
>>> UB the most.  If we can't solve it for Swift 3, affected users can write a 
>>> wrapper to implement the boundscheck, but I think we should at very least 
>>> take it up again for Swift 4.
>>> 
>>> Drew
>> 
>> In my current implementation:
>> https://github.com/atrick/swift/blob/unsafebytes/stdlib/public/core/UnsafeBytes.swift.gyb
>> 
>> The bounds checks in `copyBytes(from:)` are release mode preconditions.
>> 
>> The bounds checks for `subscript`, `load(as:)`, and `storeBytes(of:as:)` are 
>> debug only because it’s likely they occur in some loop that could be covered 
>> by a single bounds check. By extension, the sequence iterator is only bounds 
>> checked in debug mode.
>> 
>> One possibility would be different names for the bounds checked forms of 
>> those methods: getByte(atOffset:), setByte(atOffset:), 
>> load(fromCheckedOffset:as:), storeBytes(of:toCheckedOffset:as:). Along with 
>> some kind of bounds checked Iterator.
>> 
>> I don’t think makes a lot of sense as generic Collection though. 
>> Alternatively, we just have an UnsafeBoundsCheckedBytes wrapper.
>> 
>> This would a good thing to experiment with in your project. We may be able 
>> to follow-up with a Swift 4 proposal. The important thing now is to 
>> determine whether the proposed Swift 3 design will make that wrapper 
>> difficult in any way.
>> 
>> -Andy
>>> On September 1, 2016 at 5:19:02 PM, Andrew Trick via swift-evolution 
>>> (swift-evolution@swift.org) wrote:
>>> 
 I’m resending this for Review Manager Dave A. because the announce list is 
 dropping his messages...
 
 Hello Swift community,
 
 The review of "UnsafeBytes" begins now and runs through September
 7th. This late addition to Swift 3 is a follow-up to SE-0107:
 UnsafeRawPointer. It addresses common use cases for UnsafeRawPointer,
 allowing developers to continue working with collections of UInt8 values,
 but now doing so via a type safe API. The UnsafeBytes API will not require 
 direct manipulation of raw pointers or reasoning about binding memory.
 
 The proposal is available here:
 
  
 
 
 Reviews are an important part of the Swift evolution process. All reviews
 should be sent to the swift-evolution mailing list at
 
  
 
 or, if you would like to keep your feedback private, directly to the
 review manager. When replying, please try to keep the proposal link at
 the top of the message:
 
 Proposal link:
  
 
 What goes into a review?
 
 The goal of the review process is to improve the proposal under review
 through constructive criticism and, eventually, determine the direction of
 Swift. When writing your review, here are some questions you might want to
 answer in your review:
 
  * What is your evaluation of the proposal?
  * Is the problem being addressed significant enough to warrant a
change to Swift?
  * Does this proposal fit well with the feel and

Re: [swift-evolution] SE-0138 UnsafeBytes

[Félix Cloutier via swift-evolution]

I'm +1 on the proposal. Not a lot to say about it; I don't expect the community 
to have a passionate argument either.

Félix

> Le 1 sept. 2016 à 17:37:47, Andrew Trick via swift-evolution 
>  a écrit :
> 
> The proposal is available here:
> 
>  
>   
> >
> 
>> On Sep 1, 2016, at 4:59 PM, Drew Crawford > > wrote:
>> I'm possibly one of the larger users of raw byte stuff in Swift as I 
>> maintain an entire client/server network protocol stack in Swift userspace, 
>> similar in spirit to one of the examples drawn out a lot longer.  Grepping 
>> my code produces over 200 individual uses of unsafe byte accesses.
>> 
>> I definitely agree that the problem is significant enough to warrant a 
>> last-minute change.
>> 
>> To a first approximation I agree with all the implementation choices.  The 
>> naming, the choice of UInt8, length tracking, and debug-bounds checking are 
>> all correct IMO.  We have been using something similar for a long time 
>> internally [have you been reading my code? :-) ] so I can speak from 
>> experience that the basic plan here is sound.
>> 
>> One thing I would like to see is an (opt-in) release-mode-bounds-check.  
>> Networking is a core use case for this feature, but when you are reading 
>> from a socket, production is where you need a guard against out-of-bounds UB 
>> the most.  If we can't solve it for Swift 3, affected users can write a 
>> wrapper to implement the boundscheck, but I think we should at very least 
>> take it up again for Swift 4.
>> 
>> Drew
> 
> In my current implementation:
> https://github.com/atrick/swift/blob/unsafebytes/stdlib/public/core/UnsafeBytes.swift.gyb
>  
> 
> 
> The bounds checks in `copyBytes(from:)` are release mode preconditions.
> 
> The bounds checks for `subscript`, `load(as:)`, and `storeBytes(of:as:)` are 
> debug only because it’s likely they occur in some loop that could be covered 
> by a single bounds check. By extension, the sequence iterator is only bounds 
> checked in debug mode.
> 
> One possibility would be different names for the bounds checked forms of 
> those methods: getByte(atOffset:), setByte(atOffset:), 
> load(fromCheckedOffset:as:), storeBytes(of:toCheckedOffset:as:). Along with 
> some kind of bounds checked Iterator.
> 
> I don’t think makes a lot of sense as generic Collection though. 
> Alternatively, we just have an UnsafeBoundsCheckedBytes wrapper.
> 
> This would a good thing to experiment with in your project. We may be able to 
> follow-up with a Swift 4 proposal. The important thing now is to determine 
> whether the proposed Swift 3 design will make that wrapper difficult in any 
> way.
> 
> -Andy
>> On September 1, 2016 at 5:19:02 PM, Andrew Trick via swift-evolution 
>> (swift-evolution@swift.org ) wrote:
>> 
>>> I’m resending this for Review Manager Dave A. because the announce list is 
>>> dropping his messages...
>>> 
>>> Hello Swift community,
>>> 
>>> The review of "UnsafeBytes" begins now and runs through September
>>> 7th. This late addition to Swift 3 is a follow-up to SE-0107:
>>> UnsafeRawPointer. It addresses common use cases for UnsafeRawPointer,
>>> allowing developers to continue working with collections of UInt8 values,
>>> but now doing so via a type safe API. The UnsafeBytes API will not require 
>>> direct manipulation of raw pointers or reasoning about binding memory.
>>> 
>>> The proposal is available here:
>>> 
>>>  
>>> >>  
>>> >
>>> 
>>> Reviews are an important part of the Swift evolution process. All reviews
>>> should be sent to the swift-evolution mailing list at
>>> 
>>>  >> >
>>> 
>>> or, if you would like to keep your feedback private, directly to the
>>> review manager. When replying, please try to keep the proposal link at
>>> the top of the message:
>>> 
>>> Proposal link:
>>>  >> >
>>> 
>>> What goes into a review?
>>> 
>>> The goal of the review process is to improve the proposal under review
>>> through constructive criticism and, eventually, determine the direction of
>>> Swift. When writing your review, here are some questions you might want to
>>> answer in your review:
>>> 
>>>  * What is your evaluation of the proposal?
>>>  * Is the problem being addressed significant enough to warrant a
>>>change to Swift?
>>>  * Does this proposal fit

Re: [swift-evolution] SE-0138 UnsafeBytes

[Andrew Trick via swift-evolution]

> On Sep 1, 2016, at 5:37 PM, Andrew Trick  wrote:
> 
> The proposal is available here:
> 
>  
>   
> >
> 
>> On Sep 1, 2016, at 4:59 PM, Drew Crawford > > wrote:
>> I'm possibly one of the larger users of raw byte stuff in Swift as I 
>> maintain an entire client/server network protocol stack in Swift userspace, 
>> similar in spirit to one of the examples drawn out a lot longer.  Grepping 
>> my code produces over 200 individual uses of unsafe byte accesses.
>> 
>> I definitely agree that the problem is significant enough to warrant a 
>> last-minute change.
>> 
>> To a first approximation I agree with all the implementation choices.  The 
>> naming, the choice of UInt8, length tracking, and debug-bounds checking are 
>> all correct IMO.  We have been using something similar for a long time 
>> internally [have you been reading my code? :-) ] so I can speak from 
>> experience that the basic plan here is sound.
>> 
>> One thing I would like to see is an (opt-in) release-mode-bounds-check.  
>> Networking is a core use case for this feature, but when you are reading 
>> from a socket, production is where you need a guard against out-of-bounds UB 
>> the most.  If we can't solve it for Swift 3, affected users can write a 
>> wrapper to implement the boundscheck, but I think we should at very least 
>> take it up again for Swift 4.
>> 
>> Drew
> 
> In my current implementation:
> https://github.com/atrick/swift/blob/unsafebytes/stdlib/public/core/UnsafeBytes.swift.gyb
>  
> 
> 
> The bounds checks in `copyBytes(from:)` are release mode preconditions.
> 
> The bounds checks for `subscript`, `load(as:)`, and `storeBytes(of:as:)` are 
> debug only because it’s likely they occur in some loop that could be covered 
> by a single bounds check. By extension, the sequence iterator is only bounds 
> checked in debug mode.
> 
> One possibility would be different names for the bounds checked forms of 
> those methods: getByte(atOffset:), setByte(atOffset:), 
> load(fromCheckedOffset:as:), storeBytes(of:toCheckedOffset:as:). Along with 
> some kind of bounds checked Iterator.
> 
> I don’t think makes a lot of sense as generic Collection though. 
> Alternatively, we just have an UnsafeBoundsCheckedBytes wrapper.
> 
> This would a good thing to experiment with in your project. We may be able to 
> follow-up with a Swift 4 proposal. The important thing now is to determine 
> whether the proposed Swift 3 design will make that wrapper difficult in any 
> way.

After thinking about this for a moment, I like the approach of extending 
UnsafeBytes with release-mode bounds checked versions of subscript, load, and 
storeBytes. It’s not actually meaningful to have a bounds checked iterator for 
UnsafeBytes. A wrapper would only be useful to guard against accidentally 
circumventing the bounds checks, but I’m not sure that’s really helpful in 
practice. It seems that a framework would want to provide more abstract Socket 
I/O or network message abstractions and those wrappers would just call the 
bounds checked version of the UnsafeBytes APIs.

-Andy

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-0138 UnsafeBytes

[Rien via swift-evolution]

Please include this in Swift 3.

I just performed a migration of client/server software from Swift 2.3 to Swift 
3 beta in Xcode 8 beta 6. And though I like the UnsafeRawPointer approach, I 
believe this proposal is more Swift-like and is easier to understand.
In fact I used to have my own UInt8 buffer definition that I cleaned out in the 
migration. Using the proposed solution would have made this process easier.

Rien.

> On 02 Sep 2016, at 00:18, Andrew Trick via swift-evolution 
>  wrote:
> 
> I’m resending this for Review Manager Dave A. because the announce list is 
> dropping his messages...
> 
> Hello Swift community,
> 
> The review of "UnsafeBytes" begins now and runs through September
> 7th. This late addition to Swift 3 is a follow-up to SE-0107:
> UnsafeRawPointer. It addresses common use cases for UnsafeRawPointer,
> allowing developers to continue working with collections of UInt8 values,
> but now doing so via a type safe API. The UnsafeBytes API will not require 
> direct manipulation of raw pointers or reasoning about binding memory.
> 
> The proposal is available here:
> 
>  
> 
> 
> Reviews are an important part of the Swift evolution process. All reviews
> should be sent to the swift-evolution mailing list at
> 
>  
> 
> or, if you would like to keep your feedback private, directly to the
> review manager. When replying, please try to keep the proposal link at
> the top of the message:
> 
> Proposal link:
>  
> 
> What goes into a review?
> 
> The goal of the review process is to improve the proposal under review
> through constructive criticism and, eventually, determine the direction of
> Swift. When writing your review, here are some questions you might want to
> answer in your review:
> 
>  * What is your evaluation of the proposal?
>  * Is the problem being addressed significant enough to warrant a
>change to Swift?
>  * Does this proposal fit well with the feel and direction of Swift?
>  * If you have used other languages or libraries with a similar
>feature, how do you feel that this proposal compares to those?
>  * How much effort did you put into your review? A glance, a quick
>reading, or an in-depth study?
> 
> More information about the Swift evolution process is available at
> 
>  
> 
> Thank you,
> 
> -Dave Abrahams
> Review Manager
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-0138 UnsafeBytes

[Andrew Trick via swift-evolution]

The proposal is available here:

 
>

> On Sep 1, 2016, at 4:59 PM, Drew Crawford  wrote:
> I'm possibly one of the larger users of raw byte stuff in Swift as I maintain 
> an entire client/server network protocol stack in Swift userspace, similar in 
> spirit to one of the examples drawn out a lot longer.  Grepping my code 
> produces over 200 individual uses of unsafe byte accesses.
> 
> I definitely agree that the problem is significant enough to warrant a 
> last-minute change.
> 
> To a first approximation I agree with all the implementation choices.  The 
> naming, the choice of UInt8, length tracking, and debug-bounds checking are 
> all correct IMO.  We have been using something similar for a long time 
> internally [have you been reading my code? :-) ] so I can speak from 
> experience that the basic plan here is sound.
> 
> One thing I would like to see is an (opt-in) release-mode-bounds-check.  
> Networking is a core use case for this feature, but when you are reading from 
> a socket, production is where you need a guard against out-of-bounds UB the 
> most.  If we can't solve it for Swift 3, affected users can write a wrapper 
> to implement the boundscheck, but I think we should at very least take it up 
> again for Swift 4.
> 
> Drew

In my current implementation:
https://github.com/atrick/swift/blob/unsafebytes/stdlib/public/core/UnsafeBytes.swift.gyb
 


The bounds checks in `copyBytes(from:)` are release mode preconditions.

The bounds checks for `subscript`, `load(as:)`, and `storeBytes(of:as:)` are 
debug only because it’s likely they occur in some loop that could be covered by 
a single bounds check. By extension, the sequence iterator is only bounds 
checked in debug mode.

One possibility would be different names for the bounds checked forms of those 
methods: getByte(atOffset:), setByte(atOffset:), load(fromCheckedOffset:as:), 
storeBytes(of:toCheckedOffset:as:). Along with some kind of bounds checked 
Iterator.

I don’t think makes a lot of sense as generic Collection though. Alternatively, 
we just have an UnsafeBoundsCheckedBytes wrapper.

This would a good thing to experiment with in your project. We may be able to 
follow-up with a Swift 4 proposal. The important thing now is to determine 
whether the proposed Swift 3 design will make that wrapper difficult in any way.

-Andy
> On September 1, 2016 at 5:19:02 PM, Andrew Trick via swift-evolution 
> (swift-evolution@swift.org ) wrote:
> 
>> I’m resending this for Review Manager Dave A. because the announce list is 
>> dropping his messages...
>> 
>> Hello Swift community,
>> 
>> The review of "UnsafeBytes" begins now and runs through September
>> 7th. This late addition to Swift 3 is a follow-up to SE-0107:
>> UnsafeRawPointer. It addresses common use cases for UnsafeRawPointer,
>> allowing developers to continue working with collections of UInt8 values,
>> but now doing so via a type safe API. The UnsafeBytes API will not require 
>> direct manipulation of raw pointers or reasoning about binding memory.
>> 
>> The proposal is available here:
>> 
>>  
>> >  
>> >
>> 
>> Reviews are an important part of the Swift evolution process. All reviews
>> should be sent to the swift-evolution mailing list at
>> 
>>  > >
>> 
>> or, if you would like to keep your feedback private, directly to the
>> review manager. When replying, please try to keep the proposal link at
>> the top of the message:
>> 
>> Proposal link:
>>  > >
>> 
>> What goes into a review?
>> 
>> The goal of the review process is to improve the proposal under review
>> through constructive criticism and, eventually, determine the direction of
>> Swift. When writing your review, here are some questions you might want to
>> answer in your review:
>> 
>>  * What is your evaluation of the proposal?
>>  * Is the problem being addressed significant enough to warrant a
>>change to Swift?
>>  * Does this proposal fit well with the feel and direction of Swift?
>>  * If you have used other languages or libraries with a similar
>>feature, how do you feel that this proposal compares to those?
>>  * How much effort did you put into your review? A glance, a quick
>>reading, or an in-depth study?
>> 
>> More information about the Swift evolution process is available at
>> 
>>  

Re: [swift-evolution] SE-0138 UnsafeBytes

[Drew Crawford via swift-evolution]

Proposal link:
 

I'm possibly one of the larger users of raw byte stuff in Swift as I maintain 
an entire client/server network protocol stack in Swift userspace, similar in 
spirit to one of the examples drawn out a lot longer.  Grepping my code 
produces over 200 individual uses of unsafe byte accesses.

I definitely agree that the problem is significant enough to warrant a 
last-minute change.

To a first approximation I agree with all the implementation choices.  The 
naming, the choice of UInt8, length tracking, and debug-bounds checking are all 
correct IMO.  We have been using something similar for a long time internally 
[have you been reading my code? :-) ] so I can speak from experience that the 
basic plan here is sound.

One thing I would like to see is an (opt-in) release-mode-bounds-check.  
Networking is a core use case for this feature, but when you are reading from a 
socket, production is where you need a guard against out-of-bounds UB the most. 
 If we can't solve it for Swift 3, affected users can write a wrapper to 
implement the boundscheck, but I think we should at very least take it up again 
for Swift 4.

Drew


On September 1, 2016 at 5:19:02 PM, Andrew Trick via swift-evolution 
(swift-evolution@swift.org) wrote:

I’m resending this for Review Manager Dave A. because the announce list is 
dropping his messages...

Hello Swift community,

The review of "UnsafeBytes" begins now and runs through September
7th. This late addition to Swift 3 is a follow-up to SE-0107:
UnsafeRawPointer. It addresses common use cases for UnsafeRawPointer,
allowing developers to continue working with collections of UInt8 values,
but now doing so via a type safe API. The UnsafeBytes API will not require 
direct manipulation of raw pointers or reasoning about binding memory.

The proposal is available here:

 


Reviews are an important part of the Swift evolution process. All reviews
should be sent to the swift-evolution mailing list at

 

or, if you would like to keep your feedback private, directly to the
review manager. When replying, please try to keep the proposal link at
the top of the message:

Proposal link:
 

What goes into a review?

The goal of the review process is to improve the proposal under review
through constructive criticism and, eventually, determine the direction of
Swift. When writing your review, here are some questions you might want to
answer in your review:

 * What is your evaluation of the proposal?
 * Is the problem being addressed significant enough to warrant a
   change to Swift?
 * Does this proposal fit well with the feel and direction of Swift?
 * If you have used other languages or libraries with a similar
   feature, how do you feel that this proposal compares to those?
 * How much effort did you put into your review? A glance, a quick
   reading, or an in-depth study?

More information about the Swift evolution process is available at

 

Thank you,

-Dave Abrahams
Review Manager ___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[swift-evolution] SE-0138 UnsafeBytes

[Andrew Trick via swift-evolution]

I’m resending this for Review Manager Dave A. because the announce list is 
dropping his messages...

Hello Swift community,

The review of "UnsafeBytes" begins now and runs through September
7th. This late addition to Swift 3 is a follow-up to SE-0107:
UnsafeRawPointer. It addresses common use cases for UnsafeRawPointer,
allowing developers to continue working with collections of UInt8 values,
but now doing so via a type safe API. The UnsafeBytes API will not require 
direct manipulation of raw pointers or reasoning about binding memory.

The proposal is available here:

 
>

Reviews are an important part of the Swift evolution process. All reviews
should be sent to the swift-evolution mailing list at

 >

or, if you would like to keep your feedback private, directly to the
review manager. When replying, please try to keep the proposal link at
the top of the message:

Proposal link:
 >

What goes into a review?

The goal of the review process is to improve the proposal under review
through constructive criticism and, eventually, determine the direction of
Swift. When writing your review, here are some questions you might want to
answer in your review:

 * What is your evaluation of the proposal?
 * Is the problem being addressed significant enough to warrant a
   change to Swift?
 * Does this proposal fit well with the feel and direction of Swift?
 * If you have used other languages or libraries with a similar
   feature, how do you feel that this proposal compares to those?
 * How much effort did you put into your review? A glance, a quick
   reading, or an in-depth study?

More information about the Swift evolution process is available at

 >

Thank you,

-Dave Abrahams
Review Manager___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution