Re: The Phobos Put
On Friday, 31 March 2023 at 02:23:29 UTC, Steven Schveighoffer
wrote:
There's a certain attempt in phobos in some places to try and
ensure code that is going to confuse will not compile. I think
this is one of those attempts.
Consider that if you pass a slice into `put`, then it returns
nothing. There is no indication of what actually was written.
It's essentially an inconclusive call, because the "result" is
the output range itself. How many elements were written? You
can't tell.
This is a general issue with the `put` interface—it has no
standardized way to report failure or partial success to the
caller. And the problem applies to both lvalue and rvalue output
ranges. For example:
```d
void main()
{
import std.stdio, std.range;
auto sink = stdout.lockingTextWriter;
put(sink, "hello");
}
```
How many characters were written? Nobody knows! Maybe you can
find out with `File.tell`, if the file is seekable, but there's
no guarantee it is. And if you're working in a generic context,
where you don't know the specific type of output range you're
dealing with? Forget it.
I'd argue that the way input ranges are used as output ranges
today is extremely confusing. It makes sort of a logical sense,
but the fact that you need to store your "original" range, and
then do some length math to figure out what was written makes
such code very awkward all around. The output is decipherable,
but not obvious.
I stand by my assertion that probably lvalue input ranges
should never have been treated as output ranges implicitly.
They should have had to go through some sort of wrapper.
IMO this is downstream of the issue with `put`. If `put` itself
returned information about what was written, then there would be
no need to try and recreate that information via side channels
like this (or like `File.tell`).
I agree that there are ways we could change things to make the
side channels easier to use, but ultimately that's treating the
symptom, not the disease.
Re: The Phobos Put
On 3/30/23 11:44 AM, Paul Backus wrote: It should be fine to have both a `ref` and non-`ref` overload for `put`, though, right? If the non-`ref` overload is only called with rvalues, then it's fine to leave them in an undetermined state, because nothing can access them afterward anyway. There's a certain attempt in phobos in some places to try and ensure code that is going to confuse will not compile. I think this is one of those attempts. Consider that if you pass a slice into `put`, then it returns nothing. There is no indication of what actually was written. It's essentially an inconclusive call, because the "result" is the output range itself. How many elements were written? You can't tell. I'd argue that the way input ranges are used as output ranges today is extremely confusing. It makes sort of a logical sense, but the fact that you need to store your "original" range, and then do some length math to figure out what was written makes such code very awkward all around. The output is decipherable, but not obvious. I stand by my assertion that probably lvalue input ranges should never have been treated as output ranges implicitly. They should have had to go through some sort of wrapper. -Steve
Re: The Phobos Put
On Thursday, 30 March 2023 at 13:27:33 UTC, Steven Schveighoffer
wrote:
But you can do `dig.copy(buf[])` since a dynamic array is.
Apparently, we will not be able to get rid of the necessity of
using slices. Neither with "copy" nor with "put"...
```d
import std.algorithm.mutation : copy;
import std.range.primitives : put;
void main()
{
int[8] buf;// = new int[8];
auto dig = [1, 2, 3, 4];
//dig.copy(buf[2..$-2]);/*
auto slice = buf[2..$-2];
dig.copy(slice);
assert(buf == [0, 0, 1, 2, 3, 4, 0, 0]);
buf = 0;
slice.put(dig);//*/
assert(buf == [0, 0, 1, 2, 3, 4, 0, 0]);
}
```
SDB@79
Re: The Phobos Put
On Wednesday, 29 March 2023 at 20:50:04 UTC, Steven Schveighoffer wrote: On 3/29/23 4:29 PM, ag0aep6g wrote: But regardless of Salih's exact intent, the broader point is: a non-ref overload could be added to Phobos. And that would enable `a[1..$-1].phobos_put([2, 3])`. Which is what he asked about originally. I think the idea of requiring ref output ranges is that you can then let the range keep track of its output state. An input range with lvalue elements is therefore an output range, but only if it's accepted via ref, since it has to be iterated as it goes. If you iterate it only internally, then it's either in an undetermined state when you exit `put`, or it is a forward range that was copied without using `save`. It's not the greatest situation. I feel like we probably shouldn't have made lvalue input ranges be output ranges automatically. It should be fine to have both a `ref` and non-`ref` overload for `put`, though, right? If the non-`ref` overload is only called with rvalues, then it's fine to leave them in an undetermined state, because nothing can access them afterward anyway.
Re: The Phobos Put
On 3/29/23 11:01 PM, Salih Dincer wrote:
```d
import std.algorithm.mutation : copy;
void main()
{
int[8] buf;
auto dig = [1, 2, 3, 4];
auto rem = dig.copy(buf);
assert(rem.length == 4);
}
```
Looks like 'copy' has the same overload issue.
A static array is not a range of any kind. What would `popFront` do on a
static array, since the length is part of the type?
But you can do `dig.copy(buf[])` since a dynamic array is.
-Steve
Re: The Phobos Put
On Wednesday, 29 March 2023 at 20:50:04 UTC, Steven Schveighoffer
wrote:
On 3/29/23 4:29 PM, ag0aep6g wrote:
But regardless of Salih's exact intent, the broader point is:
a non-ref overload could be added to Phobos. And that would
enable `a[1..$-1].phobos_put([2, 3])`. Which is what he asked
about originally.
I think the idea of requiring ref output ranges is that you can
then let the range keep track of its output state.
So why not copying the range to a static array? The error during
compilation of the code is as follows:
onlineapp.d(6): Error: none of the overloads of template
`std.algorithm.mutation.copy` are callable using argument types
`!()(int[], int[8])`
/dlang/dmd/linux/bin64/../../src/phobos/std/algorithm/mutation.d(367):
Candidate is: `copy(SourceRange, TargetRange)(SourceRange source, TargetRange
target)`
with `SourceRange = int[],
TargetRange = int[8]`
must satisfy the following constraint:
` isOutputRange!(TargetRange, ElementType!SourceRange)`
```d
import std.algorithm.mutation : copy;
void main()
{
int[8] buf;
auto dig = [1, 2, 3, 4];
auto rem = dig.copy(buf);
assert(rem.length == 4);
}
```
Looks like 'copy' has the same overload issue.
SDB@79
Re: The Phobos Put
On 3/29/23 4:29 PM, ag0aep6g wrote: But regardless of Salih's exact intent, the broader point is: a non-ref overload could be added to Phobos. And that would enable `a[1..$-1].phobos_put([2, 3])`. Which is what he asked about originally. I think the idea of requiring ref output ranges is that you can then let the range keep track of its output state. An input range with lvalue elements is therefore an output range, but only if it's accepted via ref, since it has to be iterated as it goes. If you iterate it only internally, then it's either in an undetermined state when you exit `put`, or it is a forward range that was copied without using `save`. It's not the greatest situation. I feel like we probably shouldn't have made lvalue input ranges be output ranges automatically. -Steve
Re: The Phobos Put
On Wednesday, 29 March 2023 at 20:29:24 UTC, ag0aep6g wrote: But regardless of Salih's exact intent, the broader point is: a non-ref overload could be added to Phobos. And that would enable `a[1..$-1].phobos_put([2, 3])`. Which is what he asked about originally. Yes, that was it, but even more. Both functions serve the same thing. Why don't we combine them? SDB@79
Re: The Phobos Put
On Wednesday, 29 March 2023 at 19:49:47 UTC, Ali Çehreli wrote:
On 3/29/23 12:21, ag0aep6g wrote:
> As far as I understand, you're saying that we cannot overload
on `ref`.
> But we can. Salih's code demonstrates just that.
>
> void f(ref int x) {}
> void f(int x) {}
> void main() { int x; f(x); f(42); } /* no errors */
I thought Salih was proposing two more overloads to the
existing put(). When I copy the existing put(), which takes
'ref R', not R[], then the code does not compile:
Wait a minute, isn't `copy` actually a `put` as well? Forget
about `ref` for a moment, please. Actually, logically, the
parameters have been swapped between the two functions. Well, if
we only had `copy` and we used `put` like `copy`, what is the
need for `put`? Examples are below: :)
```d
//version = simple;/*
version = standart;//*/
version(simple) {
auto put(R)(R[] range, R[] source)
=> copyImpl(source, range);
auto copy(R)(R[] source, R[] range)
=> copyImpl(source, range);
auto copyImpl(R)(R[] source, R[] range)
{
assert(source.length <= range.length);
foreach(element; source)
{
range[0] = element; // range.front()
range = range[1..$]; // range.popFront()
}
return range;
}
void swap (ref int x, ref int y)
{
x = x ^ y;
y = y ^ x;
x = x ^ y;
}
}
version(standart)
import std.algorithm.mutation,
std.range : put;
void main()
{
// copy() example:
enum len = 10;
auto buf = new int[len]; // buffer
auto dig = [7, 1, 2, 3]; // digits
auto diff = len - dig.length;
auto rem = copy(dig, buf);
assert(buf[0..$ - diff] == [7, 1, 2, 3]);
swap(buf[0], rem[0]);
assert(rem == [7, 0, 0, 0, 0, 0]);
// put() example 1:
put(rem, [4, 5, 6, 7, 8, 9]);
assert(buf == [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]);
// put() example 2:
enum data = [1, 0, 0, 4];
auto arr = data;
auto slice = arr[1..$-1];
version(standart)
put(slice, [2]);
version(simple)
slice = put(slice, [2]);
assert(arr == [1, 2, 0, 4]);
version(standart)
put(slice, [3]);
version(simple)
slice = put(slice, [3]);
assert(arr == [1, 2, 3, 4]);
version(standart) {
auto slc = arr[1..$-1];
put(slc, [0, 0]);
}
version(simple)
arr[1..$-1].put([0, 0]);
assert(arr == data);
}
```
All you have to do is remove the // mark in the first line. The
code compiles for me, what about you?
SDB@79
Re: The Phobos Put
On Wednesday, 29 March 2023 at 19:49:47 UTC, Ali Çehreli wrote:
I thought Salih was proposing two more overloads to the
existing put(). When I copy the existing put(), which takes
'ref R', not R[], then the code does not compile:
auto put(R)(R[] range, R[] source)
=> putImpl(range, source);
auto put(R)(ref R[] range, R[] source)
=> putImpl(range, source);
[...]
void put(R, E)(ref R r, E e)
{
// This is from Phobos <---
}
I understand Salih's ref `put` to be same as yours: a stand-in
for the one that is already in Phobos. So he's proposing to add
the non-ref one.
But regardless of Salih's exact intent, the broader point is: a
non-ref overload could be added to Phobos. And that would enable
`a[1..$-1].phobos_put([2, 3])`. Which is what he asked about
originally.
Re: The Phobos Put
On 3/29/23 12:21, ag0aep6g wrote:
> As far as I understand, you're saying that we cannot overload on `ref`.
> But we can. Salih's code demonstrates just that.
>
> void f(ref int x) {}
> void f(int x) {}
> void main() { int x; f(x); f(42); } /* no errors */
I thought Salih was proposing two more overloads to the existing put().
When I copy the existing put(), which takes 'ref R', not R[], then the
code does not compile:
auto put(R)(R[] range, R[] source)
=> putImpl(range, source);
auto put(R)(ref R[] range, R[] source)
=> putImpl(range, source);
void putImpl(R)(ref R[] range, R[] source)
{
assert(source.length <= range.length);
foreach(element; source)
{
range[0] = element; // range.front()
range = range[1..$]; // range.popFront()
}
}
void put(R, E)(ref R r, E e)
{
// This is from Phobos <---
}
void main() {
enum data = [1, 0, 0, 4];
auto arr = data;
auto slice = arr[1..$-1];
slice.put([2]); // <-- ERROR
assert(arr == [1, 2, 0, 4]);
slice.put([3]);
assert(arr == [1, 2, 3, 4]);
arr[1..$-1].put([0, 0]);
assert(arr == data);
}
Ali
Re: The Phobos Put
On Wednesday, 29 March 2023 at 16:44:31 UTC, Ali Çehreli wrote:
On 3/29/23 09:27, Salih Dincer wrote:
> In this way,
> it could also be used directly with slices. For example:
> auto put(R)(R[] range, R[] source)
>=> putImpl(range, source);
That's for rvalues.
> auto put(R)(ref R[] range, R[] source)
>=> putImpl(range, source);
That's for lvalues.
If you are proposing keeping the current ref-taking Phobos
put() as well, then the following call would be ambiguous:
slice.put([2]); // Compilation ERROR
As far as I understand, you're saying that we cannot overload on
`ref`. But we can. Salih's code demonstrates just that.
void f(ref int x) {}
void f(int x) {}
void main() { int x; f(x); f(42); } /* no errors */
Re: The Phobos Put
On 3/29/23 09:27, Salih Dincer wrote:
> In this way,
> it could also be used directly with slices. For example:
> auto put(R)(R[] range, R[] source)
>=> putImpl(range, source);
That's for rvalues.
> auto put(R)(ref R[] range, R[] source)
>=> putImpl(range, source);
That's for lvalues.
If you are proposing keeping the current ref-taking Phobos put() as
well, then the following call would be ambiguous:
slice.put([2]); // Compilation ERROR
That can be remedied by using template constraints for 'slices' vs.
other output ranges. But that would be against the idea of "D slices are
most capable ranges". What I mean is, I should be able to write any
range algorithm and pass a slice to it and it should work. If we go with
your proposal, then we would have to check for that case for some
algorithms like put().
Further, I think the user of put() in templates would have to check
whether they are dealing with a slice or not:
void foo(R)(R range) {
range.put(/* ... */);
// If we go with your proposal, whether 'range' changed depends
// on whether R is a slice or not. Do we have to check with
// 'static if' in such range algorithms?
}
Note that I am not defending the behavior of put(). I am just trying to
explain why it is so.
Ali
Re: The Phobos Put
On Wednesday, 29 March 2023 at 15:01:27 UTC, Ali Çehreli wrote:
On 3/29/23 04:48, Dennis wrote:
On the other hand, Phobos's put() works with any OutputRange so
it has to take a 'ref' to advance to know where it is left off.
This behavior makes its use with slices weird but sometimes
such is life. :)
Would not adding a prototype without a ref cause ambiguity? In
this way, it could also be used directly with slices. For example:
```d
auto put(R)(R[] range, R[] source)
=> putImpl(range, source);
auto put(R)(ref R[] range, R[] source)
=> putImpl(range, source);
void putImpl(R)(ref R[] range, R[] source)
{
assert(source.length <= range.length);
foreach(element; source)
{
range[0] = element; // range.front()
range = range[1..$]; // range.popFront()
}
}
void main() {
enum data = [1, 0, 0, 4];
auto arr = data;
auto slice = arr[1..$-1];
slice.put([2]);
assert(arr == [1, 2, 0, 4]);
slice.put([3]);
assert(arr == [1, 2, 3, 4]);
arr[1..$-1].put([0, 0]);
assert(arr == data);
}
```
SDB@79
Re: The Phobos Put
On 3/29/23 04:48, Dennis wrote: > On Wednesday, 29 March 2023 at 11:10:42 UTC, Salih Dincer wrote: >> Why does my `put` work but the Phobos `put` doesn't work with a slice? > > Your `put` doesn't take `range` by `ref`, so it allows you to pass an > rvalue. Consequently, it doesn't advance the range from the callers > perspective. And that 'ref' is necessary because not every OutputRange can be sliced for further calls to put(). Salih does not have that problem because he is working with slices, which are (usually) trivially slicable for the next portion to be passed to put(). On the other hand, Phobos's put() works with any OutputRange so it has to take a 'ref' to advance to know where it is left off. This behavior makes its use with slices weird but sometimes such is life. :) Ali
Re: The Phobos Put
On Wednesday, 29 March 2023 at 11:10:42 UTC, Salih Dincer wrote: Why does my `put` work but the Phobos `put` doesn't work with a slice? Your `put` doesn't take `range` by `ref`, so it allows you to pass an rvalue. Consequently, it doesn't advance the range from the callers perspective.
The Phobos Put
Why does my `put` work but the Phobos `put` doesn't work with a
slice?
onlineapp.d(11): Error: none of the overloads of template
`std.range.primitives.put` are callable using argument types
`!()(int[], int[])`
/dlang/dmd/linux/bin64/../../src/phobos/std/range/primitives.d(386):
Candidate is: `put(R, E)(ref R r, E e)`
```d
void main()
{
import std.range : phobos_put = put;
enum testLen = 4;
auto a = new int[testLen];
auto b = new int[testLen];
auto slice = a[1..$-1];
slice.phobos_put([2, 3]);
//a[1..$-1].phobos_put([2, 3]);
b[1..$-1].put([2, 3]);
import std.conv : text;
assert(a == b, text(a));
}
void put(R)(R[] range, R[] source)
{
assert(source.length <= range.length);
foreach(element; source)
{
range[0] = element; // range.front()
range = range[1..$]; // range popFront()
}
}
```
SDB@79
The Phobos put()
Why does my `put` work but the Phobos `put` doesn't work with a
slice?
onlineapp.d(11): Error: none of the overloads of template
`std.range.primitives.put` are callable using argument types
`!()(int[], int[])`
/dlang/dmd/linux/bin64/../../src/phobos/std/range/primitives.d(386):
Candidate is: `put(R, E)(ref R r, E e)`
```d
void main()
{
import std.range : phobos_put = put;
enum testLen = 4;
auto a = new int[testLen];
auto b = new int[testLen];
auto slice = a[1..$-1];
slice.phobos_put([2, 3]);
//a[1..$-1].phobos_put([2, 3]);
b[1..$-1].put([2, 3]);
import std.conv : text;
assert(a == b, text(a));
}
void put(R)(R[] range, R[] source)
{
assert(source.length <= range.length);
foreach(element; source)
{
range[0] = element; // range.front()
range = range[1..$]; // range popFront()
}
}
```
SDB@79
