Re: Returning a reference to be manipulated
On Saturday, 15 April 2023 at 21:00:01 UTC, kdevel wrote: On Saturday, 15 April 2023 at 15:50:18 UTC, Dennis wrote: [...] care about the type / mutability of the pointer. Returning `i`'s address in a long does not trigger the escape detector: It doesn't care about the type of pointer, but it does care about whether the type is/has a pointer in the first place. `T*`, `T[]`, `K[V]` (Associative arrays), `class`, `function`, `delegate` are pointers. Static arrays and structs depend on what they contain. Basic types such as `long` are not pointers, so lifetime checking doesn't apply.
Re: Returning a reference to be manipulated
On Saturday, 15 April 2023 at 15:50:18 UTC, Dennis wrote:
[...]
care about the type / mutability of the pointer.
Returning `i`'s address in a long does not trigger the escape
detector:
```
long foo (long s, return ref int i)
{
s = cast (long)
return s;
}
auto bar ()
{
int i;
long s;
return foo (s, i);
}
```
dmd compiles this without complaints.
Re: Returning a reference to be manipulated
On Saturday, 15 April 2023 at 14:33:52 UTC, kdevel wrote:
Does that make sense?
Whether it makes sense is subjective, but it is by design. Escape
analysis considers every pointer the same, it doesn't care about
the type / mutability of the pointer. In `@system` / `@trusted`
code, you could coerce `i` to become a string and return it:
```D
string foo(string s, return ref int i)
{
return (cast(immutable char*) )[0..4];
}
```
Re: Returning a reference to be manipulated
On Saturday, 15 April 2023 at 14:10:57 UTC, Dennis wrote:
[...]
This adds complexity, just to add some 'intermediate' safety
between `@system` and `@safe` in a few cases. It's better to
keep the rules simple and consistent.
Thanks for the answer! While playing around with return ref I
came across this:
```
string foo (string s, return ref int i)
{
return s;
}
auto bar ()
{
int i;
string s;
return foo (s, i);
}
```
```
$ dmd rr.d
rr.d(10): Error: returning `foo(s, i)` escapes a reference to
local variable `i`
```
Does that make sense?
Re: Returning a reference to be manipulated
On Saturday, 15 April 2023 at 14:10:57 UTC, Dennis wrote: This adds complexity, just to add some 'intermediate' safety between `@system` and `@safe` in a few cases. It's better to keep the rules simple and consistent. To quote my past self: There used to be different rules for lifetime errors in all of these: - explicit `@system` functions - `@system` by default functions (yes, [they were special](https://issues.dlang.org/show_bug.cgi?id=19873)) - inferred functions - `@safe` functions - `@trusted` functions It was really complex and confusing, and I've worked on simplifying it such that all lifetime errors are safety violations like any other. The only exception is directly returning a dangling pointer to a stack variable, which is just an error even in @system code ([issue 19873](https://issues.dlang.org/show_bug.cgi?id=19873)). I don't want to go back to more special cases, especially with the dip1000 by default transition which is complex enough as is. https://forum.dlang.org/post/nzotevvzvbpqscfxs...@forum.dlang.org
Re: Returning a reference to be manipulated
On Saturday, 15 April 2023 at 13:20:09 UTC, kdevel wrote: Under which circumstances is it a mistake to insert the `return` at the indicated position? If there are none why can't it be done implicitly (automatically)? It could be done in the easy example you posted, but generalizing it is harder. When importing a module, the compiler currently doesn't need to analyze function bodies to get the signature of regular (non-auto/template) functions, which would have to change. Programmers can also currently rely on the fact that the signature they see is the signature they get, but not any longer. The identity function is really simple, but as soon as control flow (if-statements) come into play, the annotations and their inference become a conservative approximation, which might give false positives in `@system` code. There would need to be a second system, one which assumes the best instead of assuming the worst. This adds complexity, just to add some 'intermediate' safety between `@system` and `@safe` in a few cases. It's better to keep the rules simple and consistent.
Re: Returning a reference to be manipulated
On Saturday, 15 April 2023 at 13:24:52 UTC, Richard (Rikki)
Andrew Cattermole wrote:
On 16/04/2023 1:20 AM, kdevel wrote:
On Saturday, 15 April 2023 at 12:45:31 UTC, Richard (Rikki)
Andrew Cattermole wrote:
It was bad analysis by the compiler, which has since been
corrected.
It should have been applied only to @safe code.
But why is the @unsafe programmer now left unprotected in
cases like this
```
ref int foo (ref int i)
// `--- automatically insert `return` here?
{
return i;
}
```
where the compiler recognizes that the function returns a ref
parameter?
Under which circumstances is it a mistake to insert the
`return` at the indicated position? If there are none why
can't it be done implicitly (automatically)?
@system (which is currently the default, there is a strong
desire to change this), does no safety checks.
When I insert `return` before `ref int` and compile the code dmd
says
```
returnref2.d(9): Error: returning `foo(i)` escapes a reference to
local variable `i`
```
Isn't this a safety check? (The code does not contain any `@`.)
My question was if it is problematic if dmd inserted a `return`
before any `ref` parameter a function returns.
Did the removed code which checked for returning ref parameters
have false positives?
If not again my question: Under what circumstances would it be a
mistake to insert `return` before the respective `ref` parameter?
Re: Returning a reference to be manipulated
On 16/04/2023 1:20 AM, kdevel wrote:
On Saturday, 15 April 2023 at 12:45:31 UTC, Richard (Rikki) Andrew
Cattermole wrote:
It was bad analysis by the compiler, which has since been corrected.
It should have been applied only to @safe code.
But why is the @unsafe programmer now left unprotected in cases like this
```
ref int foo (ref int i)
// `--- automatically insert `return` here?
{
return i;
}
```
where the compiler recognizes that the function returns a ref parameter?
Under which circumstances is it a mistake to insert the `return` at the
indicated position? If there are none why can't it be done implicitly
(automatically)?
@system (which is currently the default, there is a strong desire to
change this), does no safety checks.
You are on your own to do whatever unsafe thing you want.
If you want safety checks, use @safe.
Re: Returning a reference to be manipulated
On Saturday, 15 April 2023 at 12:45:31 UTC, Richard (Rikki)
Andrew Cattermole wrote:
It was bad analysis by the compiler, which has since been
corrected.
It should have been applied only to @safe code.
But why is the @unsafe programmer now left unprotected in cases
like this
```
ref int foo (ref int i)
// `--- automatically insert `return` here?
{
return i;
}
```
where the compiler recognizes that the function returns a ref
parameter?
Under which circumstances is it a mistake to insert the `return`
at the indicated position? If there are none why can't it be done
implicitly (automatically)?
Re: Returning a reference to be manipulated
It was bad analysis by the compiler, which has since been corrected. It should have been applied only to @safe code.
Re: Returning a reference to be manipulated
On Friday, 14 April 2023 at 11:18:21 UTC, Dennis wrote: On Friday, 14 April 2023 at 10:31:58 UTC, kdevel wrote: But in fact it is returned unless it is `return ref`. When using `return ref`, `return scope`, `scope` etc., you should be using the latest compiler and annotate functions you want checked with `@safe`. We are writing at cross puroposes. I am not asking how I could help the compiler. My point is: I have code from 2019 which produced a deprecation warning ``` returnref2.d(3): Deprecation: returning i escapes a reference to parameter i, perhaps annotate with return ``` when compiled with with the compiler of that time (v2.093.1). When I use a recent compiler (v2.102.2) the code compiles without any complaints. I am asking myself what has deprecacted? I mean: Deprecation notes are built into the compiler in order to enable a "soft landing" for a scheduled breaking change. But I cannot see any breaking change at all regarding the code example in https://forum.dlang.org/post/iysfuhjwyalnnmalb...@forum.dlang.org
Re: Returning a reference to be manipulated
On Friday, 14 April 2023 at 10:31:58 UTC, kdevel wrote:
But in fact it is returned unless it is `return ref`.
When using `return ref`, `return scope`, `scope` etc., you should
be using the latest compiler and annotate functions you want
checked with `@safe`. In previous versions, the compiler would
often conflate `return ref` and `return scope`, and it was also
inconsistent in whether it would do checks in `@safe`, `@system`,
and even 'default/unannotated' functions.
Now, it is more consistent, performing checks in `@safe` code
only.
I don't get it! Is there any legitimate use of returning a ref
such that it outlives the matching argument's lifetime? If not:
Isn't this `return ref` completely redundant?
The annotation is needed because the compiler can't always figure
out what you're doing with a `ref` parameter:
```D
ref int mysteryFunc(ref int x) @safe; // external implementation
ref int escape() @safe
{
int local; // allocated on stack frame, should not escape
this function
return mysteryFunc(local); // is this safe?
}
```
Is this indeed `@safe`? It is, provided that `mysteryFunc`
doesn't return its parameter `x`. It can be implemented like this
for example:
```D
ref int mysteryFunc(ref int x) @safe
{
x++;
return *(new int);
}
```
But it wouldn't be safe if `x` were returned, so the compiler
must know about that when it happens, hence `return ref`:
```D
ref int mysteryFunc(return ref int x) @safe
{
return x;
}
```
Now the compiler can catch that `return mysteryFunc(local)` is
unsafe. Note that if `mysteryFunc` is a template function, nested
function, or returns `auto`, then the compiler infers attributes
automatically, including `return ref`. Then you can still write
it as `mysteryFunc(ref int x)` and it will automatically be
treated as `return ref`.
Re: Returning a reference to be manipulated
On Friday, 14 April 2023 at 09:42:14 UTC, Bastiaan Veelo wrote:
[...]
Up to dmd v2.100.2 I am warned/get an error during compilation:
```
$ dmd returnref2.d
returnref2.d(3): Deprecation: returning `i` escapes a
reference to parameter `i`
returnref2.d(1):perhaps annotate the parameter with
`return`
$ dmd -dip1000 returnref2.d
returnref2.d(3): Error: returning `i` escapes a reference to
parameter `i`
returnref2.d(1):perhaps annotate the parameter with
`return`
```
With later dmd versions (up to including v2.102.2) the code
compiles without complaints. Is this intended?
I think this is intended. Adding `@safe:` on top makes the
complaint come back (in dmd 2.102 it is deprecated, in 2.103
it is an error).
What irritates me is that only with `return ref int`
```
ref int foo (return ref int i)
{
return i;
}
[...]
```
dmd complains:
```
returnref2.d(9): Error: returning `foo(i)` escapes a reference to
local variable `i`
```
The documentation [1] says that a `ref` parameter may not be
returned unless it is `return ref`. But in fact it is returned
unless it is `return ref`. Probably i get/got the meaning of the
English modal verb "may" wrong.
The documentation also says that `return ref` parameters are used
to ensure that the returned reference will not outlive the
matching argument's lifetime. I don't get it! Is there any
legitimate use of returning a ref such that it outlives the
matching argument's lifetime? If not: Isn't this `return ref`
completely redundant?
[1] https://dlang.org/spec/function.html
Re: Returning a reference to be manipulated
On Friday, 14 April 2023 at 00:50:31 UTC, kdevel wrote:
```
ref int foo (ref int i)
{
return i;
}
ref int bar ()
{
int i;
return foo (i);
}
void main ()
{
import std.stdio;
auto i = bar;
i.writeln;
}
```
Up to dmd v2.100.2 I am warned/get an error during compilation:
```
$ dmd returnref2.d
returnref2.d(3): Deprecation: returning `i` escapes a reference
to parameter `i`
returnref2.d(1):perhaps annotate the parameter with
`return`
$ dmd -dip1000 returnref2.d
returnref2.d(3): Error: returning `i` escapes a reference to
parameter `i`
returnref2.d(1):perhaps annotate the parameter with
`return`
```
With later dmd versions (up to including v2.102.2) the code
compiles without complaints. Is this intended?
I think this is intended. Adding `@safe:` on top makes the
complaint come back (in dmd 2.102 it is deprecated, in 2.103 it
is an error).
-- Bastiaan.
Re: Returning a reference to be manipulated
On Thursday, 13 April 2023 at 22:09:48 UTC, Jacob Shtokolov wrote:
[...]
ref opIndex(string key) return
[...]
Regarding the return ref I found this code of 2019 on my harddisk
which is from or was motivated by a dconf talk:
```
ref int foo (ref int i)
{
return i;
}
ref int bar ()
{
int i;
return foo (i);
}
void main ()
{
import std.stdio;
auto i = bar;
i.writeln;
}
```
Up to dmd v2.100.2 I am warned/get an error during compilation:
```
$ dmd returnref2.d
returnref2.d(3): Deprecation: returning `i` escapes a reference
to parameter `i`
returnref2.d(1):perhaps annotate the parameter with
`return`
$ dmd -dip1000 returnref2.d
returnref2.d(3): Error: returning `i` escapes a reference to
parameter `i`
returnref2.d(1):perhaps annotate the parameter with
`return`
```
With later dmd versions (up to including v2.102.2) the code
compiles without complaints. Is this intended?
Re: Returning a reference to be manipulated
On Thursday, 13 April 2023 at 07:05:10 UTC, Chris Katko wrote:
I'm trying to figure out how to return a reference to something
that may not be a reference type.
```d
@safe:
struct Stats
{
float[string] data;
ref opIndex(string key) return
{
// The `require()` takes care of non-existing values,
initializing them
return data.require(key, 0);
}
}
void main()
{
import std.stdio;
Stats foo;
// 1. You can't "capture" a reference, only use it directly
foo["tacos"] += 2;
// 2. You can use an alternative function-calling form
(sometimes replaced by the `with` operator)
((ref float val) => val += 2)(foo["burritos"]);
// 3. As a last resort, use pointers or make struct wrappers
around them
auto x = ["quesadillas"];
*x += 2;
writeln(foo);
}
```
Re: Returning a reference to be manipulated
On Thursday, 13 April 2023 at 07:05:10 UTC, Chris Katko wrote:
Right now, I'm using pointers which resolves to:
```D
// float* opIndex(string key){...} using pointer
(*s["tacos"])++; // works with pointer, but is strange looking
s["tacos"]++; // preferred syntax or something similar
```
You can use a wrapper struct with `alias this` to make the
pointer a little nicer to use:
```d
struct Ref(T)
{
T* ptr;
ref inout(T) deref() inout { return *ptr; }
alias deref this;
}
Ref!T byRef(T)(ref T obj)
{
return Ref!T();
}
struct stats
{
float[string] data;
Ref!float opIndex(string key)
{
return data.require(key, 0).byRef;
}
}
void main()
{
stats foo;
auto x = foo["tacos"];
x++;
assert(foo["tacos"] == 1);
}
```
Re: Returning a reference to be manipulated
On Thursday, 13 April 2023 at 07:05:10 UTC, Chris Katko wrote:
I'm trying to figure out how to return a reference to something
that may not be a reference type.
```D
struct stats
{
float[string] data=0;
float ref opIndex(string key)
{
return data[key]; // want a ref to a specific element
}
}
void test()
{
stats foo;
auto x = foo.bar(); // returns some sort of reference
// data["tacos"] = 0
x++; // data["tacos"] = 1
}
```
Right now, I'm using pointers which resolves to:
```D
// float* opIndex(string key){...} using pointer
(*s["tacos"])++; // works with pointer, but is strange looking
s["tacos"]++; // preferred syntax or something similar
```
I'm pretty sure you can't return `ref` to a value-type without it
being a pointer, or a slice.
Unlike C++, D does not have `ref` variables (`&`). You can have
value-type function parameters that are `ref` though.
But I feel that the easier solution for the example you gave
would be to simply save the string used to index the float,
rather than a reference to the float itself:
```d
import std.stdio: writeln;
struct Stats{
float[string] data;
float opIndex(string key){ return data[key]; }
float opIndexAssign(float val, string key){ data[key] = val;
return val; }
}
void main(){
Stats foo;
foo.data = ["tacos": 0];
string ind = "tacos";
float x = foo[ind];
writeln(foo.data[ind]); //0
x++;
foo[ind] = x;
writeln(foo.data[ind]); //1
}
```
Passing pointers around might be faster, but it's error-prone,
and cumbersome for simple operators as you mentioned. However,
you could always wrap your pointer in a struct with some
necessary operator overloads:
```d
import std.stdio: writeln;
struct Float{
float* data;
float opUnary(string op: "++")(){ (*data)++; return *data; }
}
struct Stats{
float[string] data;
Float opIndex(string key){ return Float(key in data); }
}
void main(){
Stats foo;
foo.data = ["tacos": 0];
Float x = foo["tacos"];
writeln(foo.data["tacos"]); //0
x++;
writeln(foo.data["tacos"]); //1
}
```
Returning a reference to be manipulated
I'm trying to figure out how to return a reference to something
that may not be a reference type.
```D
struct stats
{
float[string] data=0;
float ref opIndex(string key)
{
return data[key]; // want a ref to a specific element
}
}
void test()
{
stats foo;
auto x = foo.bar(); // returns some sort of reference
// data["tacos"] = 0
x++; // data["tacos"] = 1
}
```
Right now, I'm using pointers which resolves to:
```D
// float* opIndex(string key){...} using pointer
(*s["tacos"])++; // works with pointer, but is strange looking
s["tacos"]++; // preferred syntax or something similar
```
