Re: [swift-evolution] Proposal to improve C pointer type import

[Florent Bruneau via swift-evolution]

Hi Charlie,

Thanks for your answer.

> Le 7 févr. 2017 à 18:23, Charlie Monroe  a écrit :
> 
>> 
>> On Feb 7, 2017, at 5:56 PM, Florent Bruneau via swift-evolution 
>>  wrote:
>> 
>> Anyone interested in that subject?
>> 
>>> Le 31 janv. 2017 à 09:16, Florent Bruneau via swift-evolution 
>>>  a écrit :
>>> 
>>> Hi swift-evolution, 
>>> 
>>> For the last few weeks, I've been working on introducing some Swift in a 
>>> pure-C codebase. While the Clang importer makes the process quite smooth, 
>>> there are still some rough edges.
>>> 
>>> Here is a (lengthy) proposal resulting from that experience.
>>> Rendered version: 
>>> https://gist.github.com/Fruneau/fa83fe87a316514797c1ee2e5012
>>> 
>>> Introduction
>>> ===
>>> 
>>> Directly importing C APIs is a core feature of the Swift compiler. In that 
>>> process, C pointers are systematically imported as `Unsafe*Pointer` swift 
>>> objects. However, in C we make the distinction between pointers that 
>>> reference a single object, and those pointing to an array of objects. In 
>>> the case of a single object of type `T`, the Swift compiler should be able 
>>> to import the parameter `T *` as a `inout T`, and `T const *` as `T`. Since 
>>> the compiler cannot makes the distinction between pointer types by itself, 
>>> we propose to add an attribute of C pointer for that purpose.
>>> 
>>> Motivation
>>> ===
>>> 
>>> Let consider the following C API:
>>> 
>>> ```c
>>> typedef struct sb_t {
>>>  char * _Nonnull data;
>>>  int len;
>>>  int size;
>>> } sb_t;
>>> 
>>> /** Append the string \p str to \p sb. */
>>> void sb_adds(sb_t * _Nonnull sb, const char * _Nonnull str);
>>> 
>>> /** Append the content of \p other to \p sb. */
>>> void sb_addsb(sb_t * _Nonnull sb, const sb_t * _Nonnull other);
>>> 
>>> /** Returns the amount of available memory of \p sb. */
>>> int sb_avail(const sb_t * _Nonnull sb);
>>> ```
>>> 
>>> This is imported in Swift as follow:
>>> 
>>> ```swift
>>> struct sb_t {
>>>  var data: UnsafeMutablePointer
>>>  var len: Int32
>>>  var size: Int32
>>> }
>>> 
>>> func sb_adds(_ sb: UnsafeMutablePointer, _ str: UnsafePointer)
>>> func sb_addsb(_ sb: UnsafeMutablePointer, _ other: 
>>> UnsafePointer)
>>> func sb_avail(_ sb: UnsafePointer) -> Int32
>>> ```
>>> 
>>> `sb_adds()` takes two pointers: the first one is supposed to point to a 
>>> single object named `sb` that will be mutated in order to add the content 
>>> of `str` which points to a c-string. So we have two kinds of pointers: the 
>>> first points to a single object, the second to a buffer. But both are 
>>> represented using `Unsafe*Pointer`. Swift cannot actually make the 
>>> difference between those two kind of pointers since the C language provides 
>>> no way to express it.
>>> 
>>> `sb_addsb()` takes two objects of type `sb_t`. The first is mutated by the 
>>> function by appending the content of the second one, which is `const`. The 
>>> constness is properly reflected in Swift. However, the usage of the 
>>> imported API is Swift might be surprising since Swift requires usage of an 
>>> `inout` parameter in order to build an `Unsafe*Pointer` object:
>>> 
>>> ```swift
>>> var sb = sb_t(...)
>>> let sb2 = sb_t(...)
>>> sb_addsb(, ) // error: cannot pass immutable value as inout 
>>> argument: 'sb2' is a 'let' constant
> 
> This is because your declaration is const sb_t * _Nonnull other... See 
> http://stackoverflow.com/questions/1143262/what-is-the-difference-between-const-int-const-int-const-and-int-const
> 
> Change it to "const sb_t * const _Nonnull other" and you get a non-mutable 
> pointer and you can use it with let.

Actually, no. In case of a function argument, the difference between `const 
sb_t *` and `const sb_t * const` is the same as between a `var` argument and a 
`let` argument in swift < 3: it only affects the mutability of the variable 
inside the function definition, and has no effect on the outside. When imported 
in swift, both result in the exact same function prototype.

```c
void sb_addsb(sb_t *self, const sb_t *other);
void sb_addsb2(sb_t *self, const sb_t * const other);
void sb_addsb3(sb_t * const self, const sb_t * const other);
```

```swift
public func sb_addsb(_ self: UnsafeMutablePointer!, _ other: 
UnsafePointer!)
public func sb_addsb2(_ self: UnsafeMutablePointer!, _ other: 
UnsafePointer!)
public func sb_addsb3(_ self: UnsafeMutablePointer!, _ other: 
UnsafePointer!)
```

> 
>>> sb_addsb(, sb2) // cannot convert value of type 'sb_t' to expected 
>>> argument type 'UnsafePointer!'
> 
> If the other parameter is const, why not just take in the struct vs. pointer 
> to it? Yes, you run into the risk of copying the structure, but since the 
> structure (unless it's really small and fits into registers on some 
> architectures) gets passed by reference and if the compiler is smart enough 
> during optimization, it won't copy it anyway... (At least from what I 

Re: [swift-evolution] Proposal to improve C pointer type import

[Charlie Monroe via swift-evolution]

> On Feb 7, 2017, at 5:56 PM, Florent Bruneau via swift-evolution 
>  wrote:
> 
> Anyone interested in that subject?
> 
>> Le 31 janv. 2017 à 09:16, Florent Bruneau via swift-evolution 
>>  a écrit :
>> 
>> Hi swift-evolution, 
>> 
>> For the last few weeks, I've been working on introducing some Swift in a 
>> pure-C codebase. While the Clang importer makes the process quite smooth, 
>> there are still some rough edges.
>> 
>> Here is a (lengthy) proposal resulting from that experience.
>> Rendered version: 
>> https://gist.github.com/Fruneau/fa83fe87a316514797c1ee2e5012
>> 
>> Introduction
>> ===
>> 
>> Directly importing C APIs is a core feature of the Swift compiler. In that 
>> process, C pointers are systematically imported as `Unsafe*Pointer` swift 
>> objects. However, in C we make the distinction between pointers that 
>> reference a single object, and those pointing to an array of objects. In the 
>> case of a single object of type `T`, the Swift compiler should be able to 
>> import the parameter `T *` as a `inout T`, and `T const *` as `T`. Since the 
>> compiler cannot makes the distinction between pointer types by itself, we 
>> propose to add an attribute of C pointer for that purpose.
>> 
>> Motivation
>> ===
>> 
>> Let consider the following C API:
>> 
>> ```c
>> typedef struct sb_t {
>>   char * _Nonnull data;
>>   int len;
>>   int size;
>> } sb_t;
>> 
>> /** Append the string \p str to \p sb. */
>> void sb_adds(sb_t * _Nonnull sb, const char * _Nonnull str);
>> 
>> /** Append the content of \p other to \p sb. */
>> void sb_addsb(sb_t * _Nonnull sb, const sb_t * _Nonnull other);
>> 
>> /** Returns the amount of available memory of \p sb. */
>> int sb_avail(const sb_t * _Nonnull sb);
>> ```
>> 
>> This is imported in Swift as follow:
>> 
>> ```swift
>> struct sb_t {
>>   var data: UnsafeMutablePointer
>>   var len: Int32
>>   var size: Int32
>> }
>> 
>> func sb_adds(_ sb: UnsafeMutablePointer, _ str: UnsafePointer)
>> func sb_addsb(_ sb: UnsafeMutablePointer, _ other: UnsafePointer)
>> func sb_avail(_ sb: UnsafePointer) -> Int32
>> ```
>> 
>> `sb_adds()` takes two pointers: the first one is supposed to point to a 
>> single object named `sb` that will be mutated in order to add the content of 
>> `str` which points to a c-string. So we have two kinds of pointers: the 
>> first points to a single object, the second to a buffer. But both are 
>> represented using `Unsafe*Pointer`. Swift cannot actually make the 
>> difference between those two kind of pointers since the C language provides 
>> no way to express it.
>> 
>> `sb_addsb()` takes two objects of type `sb_t`. The first is mutated by the 
>> function by appending the content of the second one, which is `const`. The 
>> constness is properly reflected in Swift. However, the usage of the imported 
>> API is Swift might be surprising since Swift requires usage of an `inout` 
>> parameter in order to build an `Unsafe*Pointer` object:
>> 
>> ```swift
>> var sb = sb_t(...)
>> let sb2 = sb_t(...)
>> sb_addsb(, ) // error: cannot pass immutable value as inout argument: 
>> 'sb2' is a 'let' constant

This is because your declaration is const sb_t * _Nonnull other... See 
http://stackoverflow.com/questions/1143262/what-is-the-difference-between-const-int-const-int-const-and-int-const

Change it to "const sb_t * const _Nonnull other" and you get a non-mutable 
pointer and you can use it with let.

>> sb_addsb(, sb2) // cannot convert value of type 'sb_t' to expected 
>> argument type 'UnsafePointer!'

If the other parameter is const, why not just take in the struct vs. pointer to 
it? Yes, you run into the risk of copying the structure, but since the 
structure (unless it's really small and fits into registers on some 
architectures) gets passed by reference and if the compiler is smart enough 
during optimization, it won't copy it anyway... (At least from what I remember 
reading.)

>> var sb3 = sb_t(...)
>> sb_addsb(, ) // works
>> ```
>> 
>> ```swift
>> sb_avail() // cannot convert value of type 'sb_t' to expected argument 
>> type 'UnsafePointer!'
>> ```
>> 
>> 
>> However, Swift also provides the `swift_name()` attribute that allows 
>> remapping a C function to a Swift method, which includes mapping one of the 
>> parameter to `self:`:
>> 
>> ```c 
>> __attribute__((swift_name("sb_t.add(self:string:)")))
>> void sb_adds(sb_t * _Nonnull sb, const char * _Nonnull str);
>> __attribute__((swift_name("sb_t.add(self:other:)")))
>> void sb_addsb(sb_t * _Nonnull sb, const sb_t * _Nonnull other);
>> __attribute__((swift_name("sb_t.avail(self:)")))
>> int sb_avail(const sb_t * _Nonnull sb);
>> ```

While I do feel your pain dealing with structs imported from C, nothing is 
stopping you from making an extension of that struct and implementing these 
methods on it... Yes, it's a lot of boilerplate, but it can be in a separate 
file until you migrate your C code into Swift, where as 

Re: [swift-evolution] Proposal to improve C pointer type import

[Florent Bruneau via swift-evolution]

Anyone interested in that subject?

> Le 31 janv. 2017 à 09:16, Florent Bruneau via swift-evolution 
>  a écrit :
> 
> Hi swift-evolution, 
> 
> For the last few weeks, I've been working on introducing some Swift in a 
> pure-C codebase. While the Clang importer makes the process quite smooth, 
> there are still some rough edges.
> 
> Here is a (lengthy) proposal resulting from that experience.
> Rendered version: 
> https://gist.github.com/Fruneau/fa83fe87a316514797c1ee2e5012
> 
> Introduction
> ===
> 
> Directly importing C APIs is a core feature of the Swift compiler. In that 
> process, C pointers are systematically imported as `Unsafe*Pointer` swift 
> objects. However, in C we make the distinction between pointers that 
> reference a single object, and those pointing to an array of objects. In the 
> case of a single object of type `T`, the Swift compiler should be able to 
> import the parameter `T *` as a `inout T`, and `T const *` as `T`. Since the 
> compiler cannot makes the distinction between pointer types by itself, we 
> propose to add an attribute of C pointer for that purpose.
> 
> Motivation
> ===
> 
> Let consider the following C API:
> 
> ```c
> typedef struct sb_t {
>char * _Nonnull data;
>int len;
>int size;
> } sb_t;
> 
> /** Append the string \p str to \p sb. */
> void sb_adds(sb_t * _Nonnull sb, const char * _Nonnull str);
> 
> /** Append the content of \p other to \p sb. */
> void sb_addsb(sb_t * _Nonnull sb, const sb_t * _Nonnull other);
> 
> /** Returns the amount of available memory of \p sb. */
> int sb_avail(const sb_t * _Nonnull sb);
> ```
> 
> This is imported in Swift as follow:
> 
> ```swift
> struct sb_t {
>var data: UnsafeMutablePointer
>var len: Int32
>var size: Int32
> }
> 
> func sb_adds(_ sb: UnsafeMutablePointer, _ str: UnsafePointer)
> func sb_addsb(_ sb: UnsafeMutablePointer, _ other: UnsafePointer)
> func sb_avail(_ sb: UnsafePointer) -> Int32
> ```
> 
> `sb_adds()` takes two pointers: the first one is supposed to point to a 
> single object named `sb` that will be mutated in order to add the content of 
> `str` which points to a c-string. So we have two kinds of pointers: the first 
> points to a single object, the second to a buffer. But both are represented 
> using `Unsafe*Pointer`. Swift cannot actually make the difference between 
> those two kind of pointers since the C language provides no way to express it.
> 
> `sb_addsb()` takes two objects of type `sb_t`. The first is mutated by the 
> function by appending the content of the second one, which is `const`. The 
> constness is properly reflected in Swift. However, the usage of the imported 
> API is Swift might be surprising since Swift requires usage of an `inout` 
> parameter in order to build an `Unsafe*Pointer` object:
> 
> ```swift
> var sb = sb_t(...)
> let sb2 = sb_t(...)
> sb_addsb(, ) // error: cannot pass immutable value as inout argument: 
> 'sb2' is a 'let' constant
> sb_addsb(, sb2) // cannot convert value of type 'sb_t' to expected 
> argument type 'UnsafePointer!'
> 
> var sb3 = sb_t(...)
> sb_addsb(, ) // works
> ```
> 
> ```swift
> sb_avail() // cannot convert value of type 'sb_t' to expected argument 
> type 'UnsafePointer!'
> ```
> 
> 
> However, Swift also provides the `swift_name()` attribute that allows 
> remapping a C function to a Swift method, which includes mapping one of the 
> parameter to `self:`:
> 
> ```c 
> __attribute__((swift_name("sb_t.add(self:string:)")))
> void sb_adds(sb_t * _Nonnull sb, const char * _Nonnull str);
> __attribute__((swift_name("sb_t.add(self:other:)")))
> void sb_addsb(sb_t * _Nonnull sb, const sb_t * _Nonnull other);
> __attribute__((swift_name("sb_t.avail(self:)")))
> int sb_avail(const sb_t * _Nonnull sb);
> ```
> 
> ```swift
> struct sb_t {
>var data: UnsafeMutablePointer
>var len: Int32
>var size: Int32
> 
>mutating func add(string: UnsafePointer)
>mutating func add(other: UnsafePointer)
>func avail() -> Int32
> }
> ```
> 
> With that attribute used, there is no need to convert the parameter mapped to 
> `self:` to an `Unsafe*Pointer`. As a consequence, we have an improved API:
> 
> ```swift
> sb2.avail() // This time it works!
> ```
> 
> But we also have some inconsistent behavior since only `self:` is affected by 
> this:
> 
> ```swift
> sb.add(other: )  // error: cannot pass immutable value as inout argument: 
> 'sb2' is a 'let' constant
> sb.add(other: sb2) // cannot convert value of type 'sb_t' to expected 
> argument type 'UnsafePointer!'
> ```
> 
> 
> What we observe here is that mapping an argument to `self:` is enough for the 
> compiler to be able to change its semantics. As soon as it knows the pointer 
> is actually the pointer to a single object, it can deal with it without 
> exposing it as an `Unsafe*Pointer`, making the API safer and less surprising.
> 
> 
> Proposed solution
> 
> 
> A new qualifier could be added to inform 

[swift-evolution] Proposal to improve C pointer type import

[Florent Bruneau via swift-evolution]

Hi swift-evolution, 

For the last few weeks, I've been working on introducing some Swift in a pure-C 
codebase. While the Clang importer makes the process quite smooth, there are 
still some rough edges.

Here is a (lengthy) proposal resulting from that experience.
Rendered version: 
https://gist.github.com/Fruneau/fa83fe87a316514797c1ee2e5012

Introduction
===

Directly importing C APIs is a core feature of the Swift compiler. In that 
process, C pointers are systematically imported as `Unsafe*Pointer` swift 
objects. However, in C we make the distinction between pointers that reference 
a single object, and those pointing to an array of objects. In the case of a 
single object of type `T`, the Swift compiler should be able to import the 
parameter `T *` as a `inout T`, and `T const *` as `T`. Since the compiler 
cannot makes the distinction between pointer types by itself, we propose to add 
an attribute of C pointer for that purpose.

Motivation
===

Let consider the following C API:

```c
typedef struct sb_t {
char * _Nonnull data;
int len;
int size;
} sb_t;

/** Append the string \p str to \p sb. */
void sb_adds(sb_t * _Nonnull sb, const char * _Nonnull str);

/** Append the content of \p other to \p sb. */
void sb_addsb(sb_t * _Nonnull sb, const sb_t * _Nonnull other);

/** Returns the amount of available memory of \p sb. */
int sb_avail(const sb_t * _Nonnull sb);
```

This is imported in Swift as follow:

```swift
struct sb_t {
var data: UnsafeMutablePointer
var len: Int32
var size: Int32
}

func sb_adds(_ sb: UnsafeMutablePointer, _ str: UnsafePointer)
func sb_addsb(_ sb: UnsafeMutablePointer, _ other: UnsafePointer)
func sb_avail(_ sb: UnsafePointer) -> Int32
```

`sb_adds()` takes two pointers: the first one is supposed to point to a single 
object named `sb` that will be mutated in order to add the content of `str` 
which points to a c-string. So we have two kinds of pointers: the first points 
to a single object, the second to a buffer. But both are represented using 
`Unsafe*Pointer`. Swift cannot actually make the difference between those two 
kind of pointers since the C language provides no way to express it.

`sb_addsb()` takes two objects of type `sb_t`. The first is mutated by the 
function by appending the content of the second one, which is `const`. The 
constness is properly reflected in Swift. However, the usage of the imported 
API is Swift might be surprising since Swift requires usage of an `inout` 
parameter in order to build an `Unsafe*Pointer` object:

```swift
var sb = sb_t(...)
let sb2 = sb_t(...)
sb_addsb(, ) // error: cannot pass immutable value as inout argument: 
'sb2' is a 'let' constant
sb_addsb(, sb2) // cannot convert value of type 'sb_t' to expected argument 
type 'UnsafePointer!'

var sb3 = sb_t(...)
sb_addsb(, ) // works
```

```swift
sb_avail() // cannot convert value of type 'sb_t' to expected argument type 
'UnsafePointer!'
```


However, Swift also provides the `swift_name()` attribute that allows remapping 
a C function to a Swift method, which includes mapping one of the parameter to 
`self:`:

```c 
__attribute__((swift_name("sb_t.add(self:string:)")))
void sb_adds(sb_t * _Nonnull sb, const char * _Nonnull str);
__attribute__((swift_name("sb_t.add(self:other:)")))
void sb_addsb(sb_t * _Nonnull sb, const sb_t * _Nonnull other);
__attribute__((swift_name("sb_t.avail(self:)")))
int sb_avail(const sb_t * _Nonnull sb);
```

```swift
struct sb_t {
var data: UnsafeMutablePointer
var len: Int32
var size: Int32

mutating func add(string: UnsafePointer)
mutating func add(other: UnsafePointer)
func avail() -> Int32
}
```

With that attribute used, there is no need to convert the parameter mapped to 
`self:` to an `Unsafe*Pointer`. As a consequence, we have an improved API:

```swift
sb2.avail() // This time it works!
```

But we also have some inconsistent behavior since only `self:` is affected by 
this:

```swift
sb.add(other: )  // error: cannot pass immutable value as inout argument: 
'sb2' is a 'let' constant
sb.add(other: sb2) // cannot convert value of type 'sb_t' to expected argument 
type 'UnsafePointer!'
```


What we observe here is that mapping an argument to `self:` is enough for the 
compiler to be able to change its semantics. As soon as it knows the pointer is 
actually the pointer to a single object, it can deal with it without exposing 
it as an `Unsafe*Pointer`, making the API safer and less surprising.


Proposed solution


A new qualifier could be added to inform the compiler that a pointer points to 
a single object. Then the Swift compiler could use that new piece of the 
information to generate API that use directly the object type instead of the 
pointer type. We propose the introduction of a new qualifier named `_Ref`, 
semantically similar to a C++ reference. That is:

* `_Ref` is applied with the same grammar as the `_Nonnull`,  `_Nullable`, 
family
* A pointer