[Issue 18807] RefRange behaves very differently for Input Ranges and Forward Ranges

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18807

Iain Buclaw  changed:

   What|Removed |Added

   Priority|P3  |P2

--

[Issue 21201] let std.uuid.parseUUID accept input ranges whose elements are char or wchar, not just dchar

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=21201

Dlang Bot  changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution|--- |FIXED

--- Comment #2 from Dlang Bot  ---
dlang/phobos pull request #7607 "Fix Issue 21201 - let std.uuid.parseUUID
accept input ranges whose elements are char or wchar, not just dchar" was
merged into master:

- ef33b5352fc1ab551def7f342906b78b769c7bd0 by Nathan Sashihara:
  Fix Issue 21201 - let std.uuid.parseUUID accept input ranges whose elements
are char or wchar, not just dchar

  All characters of interest to std.uuid.parseUUID are ASCII so there is
  no reason it cannot work on a range of chars or wchars.

https://github.com/dlang/phobos/pull/7607

--

[Issue 21201] let std.uuid.parseUUID accept input ranges whose elements are char or wchar, not just dchar

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=21201

Dlang Bot  changed:

   What|Removed |Added

   Keywords||pull

--- Comment #1 from Dlang Bot  ---
@n8sh created dlang/phobos pull request #7607 "Fix Issue 21201 - let
std.uuid.parseUUID accept input ranges whose elements are char or wchar, not
just dchar" fixing this issue:

- Fix Issue 21201 - let std.uuid.parseUUID accept input ranges whose elements
are char or wchar, not just dchar

  All characters of interest to std.uuid.parseUUID are ASCII so there is
  no reason it cannot work on a range of chars or wchars.

https://github.com/dlang/phobos/pull/7607

--

[Issue 21201] New: let std.uuid.parseUUID accept input ranges whose elements are char or wchar, not just dchar

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=21201

  Issue ID: 21201
   Summary: let std.uuid.parseUUID accept input ranges whose
elements are char or wchar, not just dchar
   Product: D
   Version: D2
  Hardware: All
OS: All
Status: NEW
  Severity: enhancement
  Priority: P1
 Component: phobos
  Assignee: nob...@puremagic.com
  Reporter: n8sh.second...@hotmail.com

All characters of interest to `std.uuid.parseUUID` are ASCII so there is no
reason it cannot work on a range of chars or wchars.

--

[Issue 21131] Appender with string does not process UTF input ranges properly

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=21131

Dlang Bot  changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution|--- |FIXED

--- Comment #2 from Dlang Bot  ---
dlang/phobos pull request #7583 "Fix 21131 -- proper handling of non-dchar
appending to Appender!wstring and Appender!dstring" was merged into master:

- 9c64fd76ffc2c78981a48d6f9201f52bfbf23760 by Steven Schveighoffer:
  Fix 21131. Detect certain string appending situations and properly
  handle them.

https://github.com/dlang/phobos/pull/7583

--

[Issue 21131] Appender with string does not process UTF input ranges properly

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=21131

Dlang Bot  changed:

   What|Removed |Added

   Keywords||pull

--- Comment #1 from Dlang Bot  ---
@schveiguy created dlang/phobos pull request #7583 "Fix 21131 -- proper
handling of non-dchar appending to Appender!wstring and Appender!dstring"
fixing this issue:

- Fix 21131. Detect certain string appending situations and properly
  handle them.

https://github.com/dlang/phobos/pull/7583

--

[Issue 21131] Appender with string does not process UTF input ranges properly

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=21131

Steven Schveighoffer  changed:

   What|Removed |Added

   Hardware|x86 |All
 OS|Mac OS X|All

--

[Issue 21131] New: Appender with string does not process UTF input ranges properly

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=21131

  Issue ID: 21131
   Summary: Appender with string does not process UTF input ranges
properly
   Product: D
   Version: D2
  Hardware: x86
OS: Mac OS X
Status: NEW
  Severity: enhancement
  Priority: P1
 Component: phobos
  Assignee: nob...@puremagic.com
  Reporter: schvei...@yahoo.com

When putting ranges of code units into an appender of a different width string,
Appender uses integer promotion instead of decoding and reencoding to append to
the string.

Example:

auto str = "ウェブサイト";
auto wstr = appender!wstring();
put(wstr, str.byCodeUnit);
writeln(wstr.data);

This outputs:

ã¦ã§ããµã¤ã

The fix is to detect this unique situation and decode the data into dchars.

--

[Issue 19532] chunkBy assert error involving non-forward input ranges.

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=19532

github-bugzi...@puremagic.com changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution|--- |FIXED

--

[Issue 19532] chunkBy assert error involving non-forward input ranges.

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=19532

--- Comment #7 from github-bugzi...@puremagic.com ---
Commits pushed to master at https://github.com/dlang/phobos

https://github.com/dlang/phobos/commit/b17984059e3dd0a0a4afc696b7f5696bf3c54826
Fix Issue 19532 - chunkBy errors involving non-forward input ranges

https://github.com/dlang/phobos/commit/98d74d5c16923ca0460594b8e230dea885cf591d
Merge pull request #6841 from jondegenhardt/fix-19532-chunkBy-input-range

Fix Issue 19532 - chunkBy errors involving non-forward input ranges
merged-on-behalf-of: Nicholas Wilson 

--

[Issue 19532] chunkBy assert error involving non-forward input ranges.

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=19532

--- Comment #6 from Jon Degenhardt  ---
PR 6841 (https://github.com/dlang/phobos/pull/6841)

--

[Issue 19532] chunkBy assert error involving non-forward input ranges.

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=19532

--- Comment #5 from Jon Degenhardt  ---
It turns out that chunkBy has several issues with non-forward input ranges. I
am working on a PR to address them.

--

[Issue 19532] chunkBy assert error involving non-forward input ranges.

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=19532

Jon Degenhardt  changed:

   What|Removed |Added

Summary|chunkBy assert error|chunkBy assert error
   |involving reference input   |involving non-forward input
   |ranges. |ranges.

--

[Issue 19532] chunkBy assert error involving reference input ranges.

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=19532

--- Comment #4 from Jon Degenhardt  ---
A bit more explanation - I encountered problems with chunkBy when writing an
merge and aggregation of sorted files. Basically, an external merge sort,
except also combining equivalent key entries and aggregating values associated
with the keys. This is essentially a merge-chunkby-map/fold operation, except
with a custom merge operation.

The examples in this report are cases I found while trying to identify a
simpler sample case. These samples still don't show all the behaviors I
encountered, but they should be enough to motivate the issues.

--

[Issue 19532] chunkBy assert error involving reference input ranges.

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=19532

--- Comment #3 from Jon Degenhardt  ---
One more:

-- case4.d --
import std.stdio;
import std.range;
import std.algorithm : chunkBy, fold, map;
import std.container.binaryheap;

void main(string[] args)
{
auto x = [3, 1, 0, 6, 0, 2, 4, 7, 6, 8, 8, 4, 1, 2, 9, 5, 3, 1, 6];
auto y = x
.heapify!"a > b"
.chunkBy!((a, b) => a == b)
.map!(c => c.fold!((a, b) => a + b));

writeln(y);
}


The above will fail with assert errors similar to the others. However, the
following are fine:
   a) Removing the '.map!(c => c.fold...' line. 
   b) Adding a '.array' step after 'heapify.

The run:

$ dmd case4.d
$ ./case4
core.exception.AssertError@/Users/jondegenhardt/devtools/dmd2-2.083.1/osx/bin/../../src/phobos/std/algorithm/iteration.d(598):
Attempting to popFront an empty map.

??:? _d_assert_msg [0xc3c548e]
??:? pure @safe void
std.algorithm.iteration.MapResult!(case4.main(immutable(char)[][]).__lambda3,
std.algorithm.iteration.ChunkByImpl!(case4.main(immutable(char)[][]).__lambda2,
std.container.binaryheap.BinaryHeap!(int[], "a >
b").BinaryHeap).ChunkByImpl).MapResult.popFront() [0xc3ace89]

... more lines ...

??:? _Dmain [0xc3aa1e9]
[0, 3, 4, 6, 8, 5, 18, 7, 16, 9$

--

[Issue 19532] chunkBy assert error involving reference input ranges.

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=19532

--- Comment #2 from Jon Degenhardt  ---
Another example, this one involving multiwayMerge.

 case3.d -
import std.stdio;
import std.range;
import std.algorithm : chunkBy, multiwayMerge;

auto inputRangeStruct(R)(R range) { return InputRangeStruct!R(range); }

void main(string[] args)
{
auto x1 = [0, 1, 3, 6];
auto x2 = [0, 2, 4, 6, 7];
auto x3 = [1, 2, 4, 6, 8, 8, 9];
auto x4 = [1, 3, 5, 6];

auto y = [x1, x2, x3, x4]
.multiwayMerge
.chunkBy!((a, b) => a == b);

writeln(y);
}
---

This fails similarly to the other cases.

$ dmd case3.d
jondegenhardt@mac:merge_groupby$ ./case3
core.exception.AssertError@/Users/jondegenhardt/devtools/dmd2-2.083.1/osx/bin/../../src/phobos/std/range/primitives.d(2219):
Attempting to popFront() past the end of an array of int

??:? _d_assert_msg [0xea781fe]
??:? pure nothrow @nogc @safe void
std.range.primitives.popFront!(int).popFront(ref int[]) [0xea5ee5e]
??:? pure void std.algorithm.setops.MultiwayMerge!("a < b",
int[][]).MultiwayMerge.popFront() [0xea5c623]
??:? pure void
std.algorithm.iteration.ChunkByChunkImpl!(case3.main(immutable(char)[][]).__lambda2,
std.algorithm.setops.MultiwayMerge!("a < b",
int[][]).MultiwayMerge).ChunkByChunkImpl.popFront() [0xea5f494]

... more lines of stack trace ...

??:? _Dmain [0xea5c44d]
[[0, 0], [1, 1, 1], [2, 2], [3, 3], [4, 4], [5], [6, 6, 6, 6], [7$

--

[Issue 19532] chunkBy assert error involving reference input ranges.

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=19532

Jon Degenhardt  changed:

   What|Removed |Added

   Severity|enhancement |normal

--

[Issue 19532] chunkBy assert error involving reference input ranges.

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=19532

Jon Degenhardt  changed:

   What|Removed |Added

Summary|chunkBy assert error|chunkBy assert error
   |involving merge and |involving reference input
   |reference ranges.   |ranges.

--

[Issue 18807] RefRange behaves very differently for Input Ranges and Forward Ranges

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18807

--- Comment #4 from Eyal  ---
Indeed, and both are worth fixing.

So I think it makes sense to keep these 2 issues:

issue 14619 should fix compiler's foreach behavior (calling opSlice
unnecessarily)

this issue should fix RefRange in phobos, to avoid defining opSlice().

--

[Issue 18807] RefRange behaves very differently for Input Ranges and Forward Ranges

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18807

Steven Schveighoffer  changed:

   What|Removed |Added

 CC||schvei...@yahoo.com

--- Comment #3 from Steven Schveighoffer  ---
There are 2 issues here. One is the same from 14619 -- the compiler is assuming
opSlice  on a valid range is a no-op.

The second issue is that for some reason opSlice is defined in RefRange iff
save is defined. That makes no sense.

Fixing either of these issues will fix the problem I think.

--

[Issue 18807] RefRange behaves very differently for Input Ranges and Forward Ranges

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18807

--- Comment #2 from Eyal  ---
It's related -- but that issue is about "foreach" behavior, and this issue is
about RefRange behavior, which probably should not define opSlice (as mentioned
in passing in issue 14619).

--

[Issue 18807] RefRange behaves very differently for Input Ranges and Forward Ranges

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18807

ag0aep6g  changed:

   What|Removed |Added

 CC||ag0ae...@gmail.com
   See Also||https://issues.dlang.org/sh
   ||ow_bug.cgi?id=14619

--- Comment #1 from ag0aep6g  ---
Possibly a duplicate of issue 14619.

--

[Issue 18807] New: RefRange behaves very differently for Input Ranges and Forward Ranges

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18807

  Issue ID: 18807
   Summary: RefRange behaves very differently for Input Ranges and
Forward Ranges
   Product: D
   Version: D2
  Hardware: All
   URL: http://dlang.org/
OS: All
Status: NEW
  Severity: major
  Priority: P3
 Component: phobos
  Assignee: nob...@puremagic.com
  Reporter: e...@weka.io

Example program:

import std.range: refRange, iota;

struct R1 {
auto r = iota(3);
alias r this;
}
void works() {
R1 r1;
foreach(x; refRange()) {}
assert(r1.empty);
}

struct R2 {
auto r = iota(3);
alias r this;
@property auto save() { return this; }
}
void explodes() {
R2 r2;
import std.range;
foreach(x; refRange()) {}
assert(r2.empty);   // <-- BOOM
}

works() is fine.
explodes() fails on the assertion.

The only difference is that R2 defines save().

What happens is that the foreach calls opSlice (if it exists), which calls
save. opSlice exists iff save exists.

Upgrading an existing input range to a forward range invisibly breaks any code
that used foreach on a refRange of that range.

--

[Issue 14373] std.range.refRange doesn't work on mere input ranges

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=14373

--- Comment #3 from github-bugzi...@puremagic.com ---
Commits pushed to dmd-cxx at https://github.com/dlang/phobos

https://github.com/dlang/phobos/commit/ca4e7d22ecdc53d1529fd88c71ff94198ac44303
fix Issue 14373 - std.range.refRange doesn't work on mere input ranges

https://github.com/dlang/phobos/commit/f4894c6302286c95fc63b48bdffeb1d658f5cd7a
Merge pull request #3123 from aG0aep6G/14373

--

[Issue 15982] std.array.array treats dynamic arrays as input ranges and allocates new memory

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15982

Vladimir Panteleev  changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 CC||dlang-bugzilla@thecybershad
   ||ow.net
 Resolution|--- |WONTFIX

--- Comment #11 from Vladimir Panteleev  ---
As discussed above, std.array.array currently guarantees that the data it
returns is unique. This guarantee in turn allows some assumptions, such as that
writing to the result will not have side effects, that it is safe to pass to
other parts of the program, or even delete it. I suppose that if the array were
to point to immutable elements, avoiding reallocation might be worth
considering; though, currently, even .idup will duplicate an immutable array.

As it is, this issue is missing a use case, and it will probably need to be a
compelling one to warrant changing the function's contract and risking code
breakage. Other than that... well, if you don't want your array reallocated,
then just don't call array()?

Please reopen if you have a good argument why this should be changed despite
the above.

--

Re: Swap front for char[] input ranges

[RazvanN via Digitalmars-d-learn]

On Monday, 19 December 2016 at 20:26:26 UTC, Ali Çehreli wrote:

On 12/19/2016 06:09 AM, RazvanN wrote:
> [...]
wrote:
>> [...]
InputRanges.
>> [...]
following
> [...]
char[]
> [...]
function, so
> [...]
http://dlang.org/phobos/std_algorithm_mutation.html#bringToFront

[...]


No need to mention it. It is an honor to be able contribute
to this great language.


[...]


That is right, I am working on that particular bug. After talking 
with AndreiA,
I submitted this PR [1]. It treats the case of char[], but I am 
not really sure how the constraints should be applied to the 
function since it seems there are

a lot of combinations (especially if char[] is supported).

[1] https://github.com/dlang/phobos/pull/4970

Re: Swap front for char[] input ranges

[Ali Çehreli via Digitalmars-d-learn]

On 12/19/2016 06:09 AM, RazvanN wrote:
> On Monday, 19 December 2016 at 12:25:02 UTC, Ali Çehreli wrote:
>> On 12/19/2016 02:41 AM, RazvanN wrote:
>> > [...]
>>
>> As your comments make it clear below, they cannot be InputRanges.
>>
>> > [...]
>> swapping code
>> >  [...]
>>
>> Obivously, tmp1 and tmp2 are unusued there. :)
>>
>> > [...]
>> passed.
>> > [...]
>> state
>> > [...]
>> is:
>> > [...]
>> input ranges
>> > [...]
>>
>> Not possible... It's ok to use something similar to the following
>> template constraint:
>>
>> void foo(R1, R2)(R1 r1, R2 r2)
>> if ((hasSwappableElements!R1 && hasSwappableElements!R2) ||
>> (hasLvalueElements!R1 && hasLvalueElements!R2)) {
>> // ...
>> }
>>
>> Ali
>
> In this case, the bringToFront function [1] should not accept char[]
> as parameters? Or a special path should be added in the function, so
> that char[] are treated specially?
>
> [1] http://dlang.org/phobos/std_algorithm_mutation.html#bringToFront

First, thanks to you and to your colleagues very much for improving D 
with these fixes. :)


I think you're working on the following bug:

  https://issues.dlang.org/show_bug.cgi?id=16959

Regarding the compilation error inside swapFront, its template 
constraints should be fixed as I suggested earlier. It cannot work with 
any two InputRanges. I think it warrants its separate bug.


private void swapFront(R1, R2)(R1 r1, R2 r2)
if (isInputRange!R1 && isInputRange!R2)
{
static if (is(typeof(swap(r1.front, r2.front
{
swap(r1.front, r2.front);
}
else
{
auto t1 = moveFront(r1), t2 = moveFront(r2);
r1.front = move(t2);
r2.front = move(t1);
}
}

Regarding bringToFront:

1) Reading its documentation, bringToFront() seems to be an algorithms 
that mutates its ranges (not just give a brought-to-front view of 
existing data). If so, its argument cannot simply be an InputRange 
because there are InputRanges out there where you cannot write it their 
.front. (char[] is just one example.):


size_t bringToFront(Range1, Range2)(Range1 front, Range2 back)
if (isInputRange!Range1 && isForwardRange!Range2)
{
// ...
}

Its template constraint must also be changed to include a combination of 
hasLvalueElements() and hasMobileElements().


2) After fixing that, a char[] can indeed bring characters to front but 
it would be an expensive operation where a multi-byte Unicode character 
would necessarily move a single-byte Unicode character to the right. 
(Additionally, depending on its length, the first argument may allow 
only a partial UTF-8 encoding at its end. Fail! :) )


I don't know how to allow or encode such an expensive operation which is 
outside the documented "Ο(max(front.length, back.length))" complexity of 
bringToFront(). If it were me, I would look for possibilities to change 
the behavior and make bringToFront() a non-mutating algorithm.


What do others think?

Ali

Re: Swap front for char[] input ranges

[RazvanN via Digitalmars-d-learn]

On Monday, 19 December 2016 at 12:25:02 UTC, Ali Çehreli wrote:

On 12/19/2016 02:41 AM, RazvanN wrote:
> [...]

As your comments make it clear below, they cannot be 
InputRanges.


> [...]
swapping code
>  [...]

Obivously, tmp1 and tmp2 are unusued there. :)

> [...]
passed.
> [...]
state
> [...]
is:
> [...]
input ranges
> [...]

Not possible... It's ok to use something similar to the 
following template constraint:


void foo(R1, R2)(R1 r1, R2 r2)
if ((hasSwappableElements!R1 && hasSwappableElements!R2) ||
(hasLvalueElements!R1 && hasLvalueElements!R2)) {
// ...
}

Ali


In this case, the bringToFront function [1] should not accept 
char[]
as parameters? Or a special path should be added in the function, 
so that char[] are treated specially?


[1] 
http://dlang.org/phobos/std_algorithm_mutation.html#bringToFront

Re: Swap front for char[] input ranges

[Basile B. via Digitalmars-d-learn]

On Monday, 19 December 2016 at 10:41:46 UTC, RazvanN wrote:

Hi,

I have a function which accepts 2 input Ranges and swaps the 
first element in Range1 with the first element in Range2. The 
swapping code looks something like this :


 static if (is(typeof(swap(r1.front, r2.front
 {
 swap(r1.front, r2.front);
 }
 else
 {
 auto t1 = moveFront(r1), t2 = moveFront(r2);
 auto tmp1 = r1.front;
 auto tmp2 = r2.front;
 r1.front = move(t2);
 r2.front = move(t1);
 }

This code works for most cases, except when 2 char[] are passed.
In that case, the compilation fails with error messages which 
state

that r1.front and r2.front are not Lvalues. So, my question is:
how can I swap the 2 elements since in the case of char[] input 
ranges

the front property does not return a reference?


does it work if you cast the 2 char[] as 2 ubyte[] ?
If so, it's a problem with narrow strings and utf8 decoding...i.e 
there's no guarantee that each code unit in r1 matches to a code 
unit in r2.

Swap front for char[] input ranges

[RazvanN via Digitalmars-d-learn]

Hi,

I have a function which accepts 2 input Ranges and swaps the 
first element in Range1 with the first element in Range2. The 
swapping code looks something like this :


 static if (is(typeof(swap(r1.front, r2.front
 {
 swap(r1.front, r2.front);
 }
 else
 {
 auto t1 = moveFront(r1), t2 = moveFront(r2);
 auto tmp1 = r1.front;
 auto tmp2 = r2.front;
 r1.front = move(t2);
 r2.front = move(t1);
 }

This code works for most cases, except when 2 char[] are passed.
In that case, the compilation fails with error messages which 
state

that r1.front and r2.front are not Lvalues. So, my question is:
how can I swap the 2 elements since in the case of char[] input 
ranges

the front property does not return a reference?

Re: I wrote a function that accepts input ranges, and I get compile errors when passing an array

[Seb via Digitalmars-d-learn]

On Saturday, 28 May 2016 at 20:43:00 UTC, pineapple wrote:

On Saturday, 28 May 2016 at 16:25:02 UTC, Seb wrote:
If you are interested how it works under the hood - it's 
pretty simple & elegant:


I checked up on the phobos implementation and found that arrays 
are mutated when iterated over as ranges, which didn't rest 
well with me. Nor did the idea of importing some module having 
such a significant side-effect as whether some type can act as 
a range or not. So I ended up making a sort of factory that 
turns arbitrary objects into ranges, including arrays. Seems to 
work pretty well.


Arrays in D work differently to other languages. That's why we 
call them Slices ;-)


See: https://dlang.org/d-array-article.html

Phobos just modifies the pointer - so `a = a[1 .. $];` is nothing 
more than creating a new pointer.



are mutated when iterated over as ranges,


Btw all ranges are modified during iteration, they have a state 
and with every popFront you change that. If you want to save the 
state before, you can call `.save` if it's at least a ForwardRange

Re: I wrote a function that accepts input ranges, and I get compile errors when passing an array

[pineapple via Digitalmars-d-learn]

On Saturday, 28 May 2016 at 16:25:02 UTC, Seb wrote:
If you are interested how it works under the hood - it's pretty 
simple & elegant:


I checked up on the phobos implementation and found that arrays 
are mutated when iterated over as ranges, which didn't rest well 
with me. Nor did the idea of importing some module having such a 
significant side-effect as whether some type can act as a range 
or not. So I ended up making a sort of factory that turns 
arbitrary objects into ranges, including arrays. Seems to work 
pretty well.

Re: I wrote a function that accepts input ranges, and I get compile errors when passing an array

[Seb via Digitalmars-d-learn]

On Friday, 27 May 2016 at 14:59:25 UTC, Adam D. Ruppe wrote:

On Friday, 27 May 2016 at 14:54:30 UTC, pineapple wrote:
I've encountered one remarkable difference: The phobos 
function accepts arrays and mine does not.


add `import std.array;` i think to your module and it should 
make arrays ranges


you would get a more fine-grained import with
std.range (or more even more detailed std.range.primitives).

If you are interested how it works under the hood - it's pretty 
simple & elegant:


https://github.com/dlang/phobos/blob/master/std/range/primitives.d#L2038

Re: I wrote a function that accepts input ranges, and I get compile errors when passing an array

[Adam D. Ruppe via Digitalmars-d-learn]

On Friday, 27 May 2016 at 14:54:30 UTC, pineapple wrote:
I've encountered one remarkable difference: The phobos function 
accepts arrays and mine does not.


add `import std.array;` i think to your module and it should make 
arrays ranges

I wrote a function that accepts input ranges, and I get compile errors when passing an array

[pineapple via Digitalmars-d-learn]

I'm writing my own map function modeled after the one in phobos. 
(because I feel like it, that's why. good learning experience.) 
I've encountered one remarkable difference: The phobos function 
accepts arrays and mine does not. I understand why - I'm calling 
methods that arrays don't have - but what I don't understand is 
why the phobos function _does_ work. I haven't been able to find 
what in the phobos code accounts for iterables that aren't ranges.


What am I missing?


enum canMap(T) = isInputRange!(Unqual!T);

auto map(alias func, Range)(Range range) if(canMap!Range){
return Mapping!(func, Range)(range);
}

struct Mapping(alias func, Range) if(canMap!Range){

alias URange = Unqual!Range;
Range input;

this(URange input){
this.input = input;
}

void popFront(){
this.input.popFront();
}
@property auto ref front(){
return func(this.input.front);
}

static if(isBidirectionalRange!URange){
@property auto ref back(){
return func(this.input.back);
}
void popBack(){
this.input.popBack();
}
}

static if(isInfinite!URange){
enum bool empty = false;
}else{
@property bool empty(){
return this.input.empty;
}
}

static if(isRandomAccessRange!URange){
static if(is(typeof(URange.opIndex) == function)){
alias Index = Parameters!(URange.opIndex)[0];
}else{
alias Index = size_t;
}
auto ref opIndex(Index index){
return func(this.input[index]);
}
}

static if(is(typeof(URange.opDollar))){
alias opDollar = URange.opDollar;
}

static if(hasLength!URange){
@property auto length(){
return this.input.length;
}
}

static if(hasSlicing!URange){
static if(is(typeof(URange.opIndex) == function)){
alias SliceIndex = Parameters!(URange.opIndex)[0];
}else{
alias SliceIndex = size_t;
}
auto opSlice(SliceIndex low, SliceIndex high){
return typeof(this)(this.input[low .. high]);
}
}

static if(isForwardRange!URange){
@property auto save(){
return typeof(this)(this.input.save);
}
}

}

version(unittest) import mach.error.unit;
unittest{
import std.stdio;
//import std.algorithm : map; // Works with this

// no property 'popFront', etc for type 'int[]'
writeln(
[1, 2, 3].map!((item) => (item * item))
);
}

Tangentially related question - Why does phobos use 
isInputRange!(Unqual!T) instead of just isInputRange!T? What's 
the functional difference here?

[Issue 15982] std.array.array treats dynamic arrays as input ranges and allocates new memory

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15982

--- Comment #10 from sigod  ---
(In reply to Jack Stouffer from comment #9)
> (In reply to sigod from comment #8)
> > I'm not betting on anything. It was a figure of speech.
> 
> The best way to call someone's bluff is to put money on the table. If you
> were sure that no one was using this for duplication, then you would have
> taken my bet.

I don't gamble. Especially when it's impossible to win.

(Let's end off-topic here.)

--

[Issue 15982] std.array.array treats dynamic arrays as input ranges and allocates new memory

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15982

--- Comment #9 from Jack Stouffer  ---
(In reply to sigod from comment #8)
> I'm not betting on anything. It was a figure of speech.

The best way to call someone's bluff is to put money on the table. If you were
sure that no one was using this for duplication, then you would have taken my
bet.

--

[Issue 15982] std.array.array treats dynamic arrays as input ranges and allocates new memory

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15982

--- Comment #8 from sigod  ---
I'm not betting on anything. It was a figure of speech.

--

[Issue 15982] std.array.array treats dynamic arrays as input ranges and allocates new memory

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15982

Jack Stouffer  changed:

   What|Removed |Added

 CC||j...@jackstouffer.com

--- Comment #7 from Jack Stouffer  ---
(In reply to sigod from comment #4)
> I bet no one uses `array()` for duplicating
> arrays.

Maybe, maybe not. But I'd bet $500 that people rely on the functionality and
don't know it. If this is changed you'll have people suddenly wondering why
their arrays are getting filled garbage data.

With a function as popular as std.array.array, I'd say this change is way too
dangerous.

--

[Issue 15982] std.array.array treats dynamic arrays as input ranges and allocates new memory

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15982

--- Comment #6 from ag0ae...@gmail.com ---
(In reply to ag0aep6g from comment #5)
> Here's a little generic function that relies on std.array.array's current
> behavior:
> 
> immutable(ElementType!R)[] toImmutableArray(R)(R range)
> if (isInputRange!R && !hasIndirections!(ElementType!R))
> {
> import std.array: array;
> import std.exception: assumeUnique;
> return assumeUnique(range.array);
> }
> 

Missing imports:

import std.range: ElementType, isInputRange;
import std.traits: hasIndirections;


--

[Issue 15982] std.array.array treats dynamic arrays as input ranges and allocates new memory

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15982

--- Comment #5 from ag0ae...@gmail.com ---
(In reply to sigod from comment #4)
> I bet no one uses `array()` for duplicating
> arrays.

I don't think betting on these things is a good course of action. The function
is documented to allocate a new array. Changing that is a breaking change.

> Function called `array` for a reason. Because it gives you array where you
> have only an input range. It doesn't make sense to do anything if you
> already have an array. In normal code you will never call `array` for array.

Here's a little generic function that relies on std.array.array's current
behavior:

immutable(ElementType!R)[] toImmutableArray(R)(R range)
if (isInputRange!R && !hasIndirections!(ElementType!R))
{
import std.array: array;
import std.exception: assumeUnique;
return assumeUnique(range.array);
}


--

[Issue 15982] std.array.array treats dynamic arrays as input ranges and allocates new memory

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15982

--- Comment #4 from sigod  ---
(In reply to ag0aep6g from comment #3)
> (In reply to sigod from comment #2)
> > It's meaningless for dynamic arrays.
> 
> Currently std.array.array guarantees one level of duplication. So when I
> call it on an int[], I can rely on the new array being independent from the
> original one. I can alter elements without affecting the original. I can
> cast it to immutable(int)[] without running into undefined behavior when the
> original is altered.

One of the first things that I learned in D is that you have `dup` and `idup`
"properties" for this. I bet no one uses `array()` for duplicating arrays.

> I'm not saying that this is the best behavior for a function called "array",
> but that's how it's documented and how it works. Changing it now would be a
> serious breaking change.

Function called `array` for a reason. Because it gives you array where you have
only an input range. It doesn't make sense to do anything if you already have
an array. In normal code you will never call `array` for array.

--

[Issue 15982] std.array.array treats dynamic arrays as input ranges and allocates new memory

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15982

--- Comment #3 from ag0ae...@gmail.com ---
(In reply to sigod from comment #2)
> It's meaningless for dynamic arrays.

Currently std.array.array guarantees one level of duplication. So when I call
it on an int[], I can rely on the new array being independent from the original
one. I can alter elements without affecting the original. I can cast it to
immutable(int)[] without running into undefined behavior when the original is
altered.

I'm not saying that this is the best behavior for a function called "array",
but that's how it's documented and how it works. Changing it now would be a
serious breaking change.

--

[Issue 15982] std.array.array treats dynamic arrays as input ranges and allocates new memory

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15982

--- Comment #2 from sigod  ---
It's meaningless for dynamic arrays.

Currently, if you change you code from ranges to arrays you have to remove use
of `array()` or it just adds hidden cost.

--

[Issue 15982] std.array.array treats dynamic arrays as input ranges and allocates new memory

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15982

ag0ae...@gmail.com changed:

   What|Removed |Added

 CC||ag0ae...@gmail.com

--- Comment #1 from ag0ae...@gmail.com ---
This would be a serious breaking change. The documentation says that the
function allocates and copies.

--

[Issue 15982] New: std.array.array treats dynamic arrays as input ranges and allocates new memory

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15982

  Issue ID: 15982
   Summary: std.array.array treats dynamic arrays as input ranges
and allocates new memory
   Product: D
   Version: D2
  Hardware: All
OS: All
Status: NEW
  Severity: enhancement
  Priority: P1
 Component: phobos
  Assignee: nob...@puremagic.com
  Reporter: sigod.m...@gmail.com

import std.array : array;
int[] a = [1, 2, 3];
assert(a.ptr == array(a).ptr); // fails

I think for dynamic arrays `array()` should just return provided value without
doing anything.

--

[Issue 14373] std.range.refRange doesn't work on mere input ranges

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=14373

Andrei Alexandrescu and...@erdani.com changed:

   What|Removed |Added

Version|unspecified |D2

--

Re: oversight with input ranges

[Andrei Alexandrescu via Digitalmars-d]

On 4/23/15 5:52 AM, Steven Schveighoffer wrote:

I'd really like to hear from Andrei on this.


Such an abstraction (a range without the ability to copy its front) is a 
bit of a rarity so only a small subset of algorithms would work with it. 
So probably it deserves its name e.g. SingleReadRange. -- Andrei

Re: oversight with input ranges

[Steven Schveighoffer via Digitalmars-d]

On 4/23/15 5:03 AM, Dominikus Dittes Scherkl wrote:

On Wednesday, 22 April 2015 at 19:37:21 UTC, Steven Schveighoffer wrote:

Yeah, I like this. But now we have to name the flag :)

I don't think it should be a bool, because:

isInputRange!(R, true)

Is pretty obtuse. I'd rather see something like:

isInputRange!(R, RangeOption.NonCopyable)

Or whatever name we come up with.

The nice thing about this solution is that the range options could be
passed down the trait chain, so isForwardRange easily gets this
ability as well.

-Steve


I don't like that, because this is NOT an input range, it is something
inferior but it is enough that some algorithms happen to work also on
this crippled input range. And option would indicate that it has some
extra features over and above what a general input range provides, which
is not the case.
This would require to change the code of every algorithm taking input
ranges to check if the range is copyable - exactly what we don't wanted!

I would name it clumsy isNonCopyableInputRange, and change only those
algorithms that can cope with such a thing.


I've given some more thought to how this is playing out. If you look 
around std.algorithm, etc., you will see many different additive 
traits for input ranges, i.e. hasAssignableElements, hasSlicing, all 
these things that work on top of the basic building blocks of ranges to 
further tune what an algorithm can do.


This is kind of a new thing that wasn't considered, and something where 
we can't say is input range but doesn't have x, because isInputRange 
doesn't provide a less specific version. It's like we thought we had 
discovered the atom, only to find we actually just discovered a molecule.


Note, that the only *language* requirements for ranges is that foreach 
works with them, and foreach works with these. But the library adds this 
requirement that the elements be copyable. I'm actually not convinced 
this was intentional, as when ranges came about, I don't believe we had 
the ability to disable copying.


I begrudgingly agree with you that the flag idea isn't going to cut it.

The alternatives here are, we fix isInputRange so it allows non-copyable 
elements (I don't think we can do this immediately, it's an API change 
to a fundamental building block), or we have a fleet of 
isNonCopyableXRange. That lacks appeal as well, especially in the DRY 
department.


I wonder if a possible fix is to define hasRangeTroika or whatever, and 
then hasCopyableElements, and define isInputRange to mean both. Then 
later we can deprecate the meaning of isInputRange to mean both, and 
just have it mean the first. I don't know of a good way to do this so 
people are warned that their code won't work. The nice thing about 
isInputRange is that it's just a template constraint. If the code 
expects the copyable nature, it would simply mean a compile error in 
compiling the function vs. a compile error in finding a suitable 
template. It might be a distinction without a difference.


I'd really like to hear from Andrei on this.

-Steve

Re: oversight with input ranges

[Dominikus Dittes Scherkl via Digitalmars-d]

On Wednesday, 22 April 2015 at 19:37:21 UTC, Steven Schveighoffer 
wrote:

Yeah, I like this. But now we have to name the flag :)

I don't think it should be a bool, because:

isInputRange!(R, true)

Is pretty obtuse. I'd rather see something like:

isInputRange!(R, RangeOption.NonCopyable)

Or whatever name we come up with.

The nice thing about this solution is that the range options 
could be passed down the trait chain, so isForwardRange easily 
gets this ability as well.


-Steve


I don't like that, because this is NOT an input range, it is 
something inferior but it is enough that some algorithms happen 
to work also on this crippled input range. And option would 
indicate that it has some extra features over and above what a 
general input range provides, which is not the case.
This would require to change the code of every algorithm taking 
input ranges to check if the range is copyable - exactly what we 
don't wanted!


I would name it clumsy isNonCopyableInputRange, and change only 
those algorithms that can cope with such a thing.

Re: oversight with input ranges

[John Colvin via Digitalmars-d]

On Wednesday, 22 April 2015 at 16:02:51 UTC, Steven Schveighoffer 
wrote:

On 4/21/15 7:31 PM, ketmar wrote:

On Tue, 21 Apr 2015 18:57:50 -0400, Steven Schveighoffer wrote:


On 4/21/15 3:53 PM, ketmar wrote:

here's the interesting oversight for isInputRange:
https://issues.dlang.org/show_bug.cgi?id=14478

so be careful: ranges with non-copyable elements aren't 
input ranges

for now. ;-)


This does seem like an incorrect limitation, and I'm not sure 
if it was
intentional. However, there is a lot of code out there that 
expects this
to work when isInputRange is true. I don't know if we should 
change it,
as the range definition is pretty clear in the documentation 
that auto h
= r.front is a required feature for ranges. What is the use 
case for

this?


one possible use case was shown in bugreport. array of 
non-copyable
struct, yet i still want chain 'em, for example. or filter. 
or...


struct S {
  int n;
  @disable this (this);
}

void main () {
  S[2] arr;
  arr[0].n = 1;
  arr[1].n = 2;
  foreach (ref el; arr[].filter!((ref a) = a.n  1)) {
writeln(el.n);
  }
}


this code is perfectly valid, it doesn't require copying at 
all, yet it

is invalid now.



Yes, I agree this is a valid use case. I don't think we can 
change isInputRange, because other code may depend on that 
requirement of copyability, but it is probably possible to 
alter algorithms so they can accept these not-quite input 
ranges. Definitely proxy or decorator type adapters that 
provide a proxy for front shouldn't be restricted to using only 
copyable range elements.


1st step is we need a trait to define what we are looking for, 
then the next step is to simply alter all the appropriate 
algorithms to use it. It shouldn't hinder any code that exists 
if we do that.


So what should be the name of this beast?

-Steve


We could just add a flag as a second argument to isInputRange.

Re: oversight with input ranges

[ketmar via Digitalmars-d]

On Wed, 22 Apr 2015 12:02:51 -0400, Steven Schveighoffer wrote:

 So what should be the name of this beast?

isRestrictedInputRange? ;-)

no, really, it's not a fully valid input range, so let's name the trait 
with a long word to indicate that something is very unusual with it.

signature.asc
Description: PGP signature

Re: oversight with input ranges

[Steven Schveighoffer via Digitalmars-d]

On 4/21/15 7:31 PM, ketmar wrote:

On Tue, 21 Apr 2015 18:57:50 -0400, Steven Schveighoffer wrote:


On 4/21/15 3:53 PM, ketmar wrote:

here's the interesting oversight for isInputRange:
https://issues.dlang.org/show_bug.cgi?id=14478

so be careful: ranges with non-copyable elements aren't input ranges
for now. ;-)


This does seem like an incorrect limitation, and I'm not sure if it was
intentional. However, there is a lot of code out there that expects this
to work when isInputRange is true. I don't know if we should change it,
as the range definition is pretty clear in the documentation that auto h
= r.front is a required feature for ranges. What is the use case for
this?


one possible use case was shown in bugreport. array of non-copyable
struct, yet i still want chain 'em, for example. or filter. or...

struct S {
   int n;
   @disable this (this);
}

void main () {
   S[2] arr;
   arr[0].n = 1;
   arr[1].n = 2;
   foreach (ref el; arr[].filter!((ref a) = a.n  1)) {
 writeln(el.n);
   }
}


this code is perfectly valid, it doesn't require copying at all, yet it
is invalid now.



Yes, I agree this is a valid use case. I don't think we can change 
isInputRange, because other code may depend on that requirement of 
copyability, but it is probably possible to alter algorithms so they can 
accept these not-quite input ranges. Definitely proxy or decorator 
type adapters that provide a proxy for front shouldn't be restricted to 
using only copyable range elements.


1st step is we need a trait to define what we are looking for, then the 
next step is to simply alter all the appropriate algorithms to use it. 
It shouldn't hinder any code that exists if we do that.


So what should be the name of this beast?

-Steve

Re: oversight with input ranges

[ketmar via Digitalmars-d]

On Wed, 22 Apr 2015 08:41:38 +, Dominikus Dittes Scherkl wrote:

 On Tuesday, 21 April 2015 at 23:33:38 UTC, ketmar wrote:
 On Tue, 21 Apr 2015 15:48:25 -0700, Jonathan M Davis via Digitalmars-d
 wrote:
 auto h = r.front;

 the thing is that chain, or filter, or other algorithms are perfectly
 able to work with such ranges, yet it is forbidden now. it looks like
 arbitrary limitation to me.
 
 why not introducing a new trait isNonCopyingInputRange with the old
 definition and define isInputRange = isNonCopyingInputRange and
 compiles( auto h = r.front) and then make filters and chain and all
 algorithms that don't need copying requiring only the new trait?

i already suggested that. but the thing is that:
1. someone has to code that.
2. it has to convince at least WA to accept the code.

that won't be me, especially considering 2.

signature.asc
Description: PGP signature

Re: oversight with input ranges

[Jesse Phillips via Digitalmars-d]

On Wednesday, 22 April 2015 at 05:31:03 UTC, ketmar wrote:
many algorithms in std.algo doesn't copy if you'll use `(ref 
a)` labmdas.


I understand that, but the compiler won't complain hey that 
isn't valid for all input ranges! when it sees that someone 
copied front.

Re: oversight with input ranges

[ketmar via Digitalmars-d]

On Wed, 22 Apr 2015 18:32:55 +, Jesse Phillips wrote:

 On Wednesday, 22 April 2015 at 05:31:03 UTC, ketmar wrote:
 many algorithms in std.algo doesn't copy if you'll use `(ref a)`
 labmdas.
 
 I understand that, but the compiler won't complain hey that isn't valid
 for all input ranges! when it sees that someone copied front.

yes, that's why introducing new trait is better. i was trying to minify 
support burden with changes to `isInputRange`, yet it seems to bring more 
problems that it is intended to solve, and i was wrong.

but i still believe that allowing proxying algos work on such 
restriced ranges is a viable addition. think about `Unique`, for 
example, as Peter noted in bugzilla.

signature.asc
Description: PGP signature

Re: oversight with input ranges

[Steven Schveighoffer via Digitalmars-d]

On 4/22/15 12:10 PM, John Colvin wrote:

On Wednesday, 22 April 2015 at 16:02:51 UTC, Steven Schveighoffer wrote:

Yes, I agree this is a valid use case. I don't think we can change
isInputRange, because other code may depend on that requirement of
copyability, but it is probably possible to alter algorithms so they
can accept these not-quite input ranges. Definitely proxy or
decorator type adapters that provide a proxy for front shouldn't be
restricted to using only copyable range elements.

1st step is we need a trait to define what we are looking for, then
the next step is to simply alter all the appropriate algorithms to use
it. It shouldn't hinder any code that exists if we do that.

So what should be the name of this beast?


We could just add a flag as a second argument to isInputRange.


Yeah, I like this. But now we have to name the flag :)

I don't think it should be a bool, because:

isInputRange!(R, true)

Is pretty obtuse. I'd rather see something like:

isInputRange!(R, RangeOption.NonCopyable)

Or whatever name we come up with.

The nice thing about this solution is that the range options could be 
passed down the trait chain, so isForwardRange easily gets this ability 
as well.


-Steve

Re: oversight with input ranges

[Atila Neves via Digitalmars-d]

snip
from the very beginning. This isn't new. And given that the C++ 
STL gets
away with its containers not working with non-copyable 
elements, I think
that that proves that you can have a major component in the 
standard library
- containers no less - not support copyable elements and have 
it work and be

acceptable.


Semi-acceptable, and even then, no longer the case in C++. Before 
move semantics, yes, C++ containers were limited to copyable 
elements, which meant using raw pointers since auto_ptr couldn't 
be copied. It was a royal pain.


But for 4 years now I've been able to do this:

std::vectorstd::unique_ptrMyClass stuff;
stuff.emplace_back(new MyClass);
auto ptr = std::unique_ptrMyClass(new MyClass);
stuff.push_back(std::move(ptr));

Which won't compile with the `std::move` because unique_ptr isn't 
copyable. D has move semantics. C++98/03 did not. The comparison 
with C++ is no longer justified.


Ketmar has a point though. So do you. The worst part of D move 
semantics is not being able to copy a const variable, since 
copying is move + postblit and you can't move from a const object.


Atila

Re: oversight with input ranges

[Dominikus Dittes Scherkl via Digitalmars-d]

On Tuesday, 21 April 2015 at 23:33:38 UTC, ketmar wrote:
On Tue, 21 Apr 2015 15:48:25 -0700, Jonathan M Davis via 
Digitalmars-d wrote:

auto h = r.front;


the thing is that chain, or filter, or other algorithms are 
perfectly
able to work with such ranges, yet it is forbidden now. it 
looks like

arbitrary limitation to me.


why not introducing a new trait isNonCopyingInputRange with the 
old definition and define isInputRange = isNonCopyingInputRange 
and compiles( auto h = r.front) and then make filters and chain 
and all algorithms that don't need copying requiring only the new 
trait?

oversight with input ranges

[ketmar via Digitalmars-d]

here's the interesting oversight for isInputRange:
https://issues.dlang.org/show_bug.cgi?id=14478

so be careful: ranges with non-copyable elements aren't input ranges for 
now. ;-)

signature.asc
Description: PGP signature

Re: oversight with input ranges

[ketmar via Digitalmars-d]

On Tue, 21 Apr 2015 15:48:25 -0700, Jonathan M Davis via Digitalmars-d
wrote:

 On Tuesday, April 21, 2015 19:53:47 ketmar via Digitalmars-d wrote:
 here's the interesting oversight for isInputRange:
 https://issues.dlang.org/show_bug.cgi?id=14478

 so be careful: ranges with non-copyable elements aren't input ranges
 for now. ;-)
 
 If
 
 auto h = r.front;
 
 doesn't work, then I don't think that it can be a range. And if that
 means that elements have to be copyable to be in a range, then I think
 that they're going to have to be copyable. There is going to be _way_
 too much code out there that does
 
 auto h = r.front;
 
 for it to work to try and claim that something is an input range when
 that line doesn't work. And IIRC, C++ STL containers require that their
 elements be copyable in order to work, so it's not like we'd be the
 first to require this sort of thing.

the thing is that chain, or filter, or other algorithms are perfectly 
able to work with such ranges, yet it is forbidden now. it looks like 
arbitrary limitation to me.

signature.asc
Description: PGP signature

Re: oversight with input ranges

[Jesse Phillips via Digitalmars-d]

On Tuesday, 21 April 2015 at 19:53:47 UTC, ketmar wrote:

here's the interesting oversight for isInputRange:
https://issues.dlang.org/show_bug.cgi?id=14478

so be careful: ranges with non-copyable elements aren't input 
ranges for

now. ;-)



The problem with allowing the inability to copy elements to be an 
input range is that you must then preven an algorithm from 
copying the range elements, how do you do that without preventing 
input ranges from having copyable elements?

Re: oversight with input ranges

[ketmar via Digitalmars-d]

On Tue, 21 Apr 2015 18:30:44 -0700, Jonathan M Davis via Digitalmars-d
wrote:

 And since isInputRange has guaranteed that elements have been copyable,
 I think that it's perfectly reasonable that it's been assumed that they
 would be, and changing that at this point just isn't worth it.

yet disallowing some algorithms on ranges that are (algos) working ok is 
limiting too. fixing `isInputRange` was the easiest way i found to allow 
usage of such algorithms.

another possibility is to check all algo implementations and see if they 
can be used with ranges that have non-copyable elements, and then mark 
them as such, replacing `isInputRange` to newly created trait. this can 
be automated, i believe -- at least checking.

i understand that this will add another trait to the pile, but i believe 
that this will make std.algorithm better, as it will accept more range 
types. from the user POV i don't care about range definition purity, 
the only thing i see is that std.algorithm is rejecting a perfectly valid 
range, and i'm doing no copies at all -- so i must forget about all 
std.algo niceties and fall back to stupid loops.

i.e. allowing such ranges in std.algo (where appropriate) will be a win 
for end-user. i understand that it will add some support burden, though. 
but it shouldn't be that big, considering that implementations are more 
or less stable now.

signature.asc
Description: PGP signature

Re: oversight with input ranges

[Jonathan M Davis via Digitalmars-d]

On Wednesday, April 22, 2015 02:05:38 ketmar via Digitalmars-d wrote:
 On Tue, 21 Apr 2015 18:30:44 -0700, Jonathan M Davis via Digitalmars-d
 wrote:

  And since isInputRange has guaranteed that elements have been copyable,
  I think that it's perfectly reasonable that it's been assumed that they
  would be, and changing that at this point just isn't worth it.

 yet disallowing some algorithms on ranges that are (algos) working ok is
 limiting too. fixing `isInputRange` was the easiest way i found to allow
 usage of such algorithms.

 another possibility is to check all algo implementations and see if they
 can be used with ranges that have non-copyable elements, and then mark
 them as such, replacing `isInputRange` to newly created trait. this can
 be automated, i believe -- at least checking.

 i understand that this will add another trait to the pile, but i believe
 that this will make std.algorithm better, as it will accept more range
 types. from the user POV i don't care about range definition purity,
 the only thing i see is that std.algorithm is rejecting a perfectly valid
 range, and i'm doing no copies at all -- so i must forget about all
 std.algo niceties and fall back to stupid loops.

 i.e. allowing such ranges in std.algo (where appropriate) will be a win
 for end-user. i understand that it will add some support burden, though.
 but it shouldn't be that big, considering that implementations are more
 or less stable now.

An alternative would be to create a range of pointers to the non-copyable
objects and operate on those instead. That may not be quite what you want,
but it should work, and it avoids having to complicate ranges any further
for what amounts to a very uncommon use case.

- Jonathan M Davis

Re: oversight with input ranges

[Jonathan M Davis via Digitalmars-d]

On Tuesday, April 21, 2015 23:33:38 ketmar via Digitalmars-d wrote:
 On Tue, 21 Apr 2015 15:48:25 -0700, Jonathan M Davis via Digitalmars-d
 wrote:

  On Tuesday, April 21, 2015 19:53:47 ketmar via Digitalmars-d wrote:
  here's the interesting oversight for isInputRange:
  https://issues.dlang.org/show_bug.cgi?id=14478
 
  so be careful: ranges with non-copyable elements aren't input ranges
  for now. ;-)
 
  If
 
  auto h = r.front;
 
  doesn't work, then I don't think that it can be a range. And if that
  means that elements have to be copyable to be in a range, then I think
  that they're going to have to be copyable. There is going to be _way_
  too much code out there that does
 
  auto h = r.front;
 
  for it to work to try and claim that something is an input range when
  that line doesn't work. And IIRC, C++ STL containers require that their
  elements be copyable in order to work, so it's not like we'd be the
  first to require this sort of thing.

 the thing is that chain, or filter, or other algorithms are perfectly
 able to work with such ranges, yet it is forbidden now. it looks like
 arbitrary limitation to me.

Just because a few algorithms happen to work with non-copyable elements
doesn't meant that it works in general. I can understand why you'd want to
be able to do it, but not being able to do

auto h = r.front;

is such a huge limitation that I don't think that it's even vaguely
acceptable. I can guarantee that there is a lot of code out there that does
it. Just grepping Phobos quickly finds a few cases, let alone digging into
all of the code that exists in the wild. If anything, the problem that we've
run into is folks who keep front around after popFront has been called
(which doesn't always work with input ranges - e.g. byLine), and to do that,
you have to be able to copy front, so clearly, folks have been doing that.

The definition of input ranges has quite clearly included

auto h = r.front;

from the very beginning. This isn't new. And given that the C++ STL gets
away with its containers not working with non-copyable elements, I think
that that proves that you can have a major component in the standard library
- containers no less - not support copyable elements and have it work and be
acceptable. Also, AFAIK, this is the first time anyone has ever brought up
this complaint for D ranges. So, while I'm sure that it's a limitation that
might be irritating in some cases, I also think that it's pretty clear that
it affects a very small percentage of the users out there.

If we were starting over, then maybe we could have something like
hasCopyableElements and indicate that range elements aren't guaranteed to be
copyable without that, but that would break too much code at this point, and
honestly, we have too many traits like that already. Writing fully correct
range code that takes into all of the stray cases is ridiculously hard -
especially if you try and wrap a range without losing any of its
capabilities while wrapped. We really need the common case to be easy, and
disallowing copying range elements without testing for that capability first
is not user-friendly at all. Even if it were considered more correct from a
technical standpoint, folks just wouldn't do it.

And since isInputRange has guaranteed that elements have been copyable,
I think that it's perfectly reasonable that it's been assumed that they
would be, and changing that at this point just isn't worth it.

- Jonathan M Davis

Re: oversight with input ranges

[Jonathan M Davis via Digitalmars-d]

I should also point out that std.array.array can't work with non-copyable
elements, and that's a pretty major function in Phobos. So, if we _were_
going to support non-copyable elements, we couldn't do it simply by making
isInputRange accept them, but regardless, I don't think that it's worth the
extra complication to support non-copyable elements.

- Jonathan M Davis

Re: oversight with input ranges

[ketmar via Digitalmars-d]

On Wed, 22 Apr 2015 03:00:18 +, Jesse Phillips wrote:

 On Tuesday, 21 April 2015 at 19:53:47 UTC, ketmar wrote:
 here's the interesting oversight for isInputRange:
 https://issues.dlang.org/show_bug.cgi?id=14478

 so be careful: ranges with non-copyable elements aren't input ranges
 for now. ;-)
 
 
 The problem with allowing the inability to copy elements to be an input
 range is that you must then preven an algorithm from copying the range
 elements, how do you do that without preventing input ranges from having
 copyable elements?

many algorithms in std.algo doesn't copy if you'll use `(ref a)` labmdas.

signature.asc
Description: PGP signature

Re: oversight with input ranges

[Jonathan M Davis via Digitalmars-d]

On Tuesday, April 21, 2015 19:53:47 ketmar via Digitalmars-d wrote:
 here's the interesting oversight for isInputRange:
 https://issues.dlang.org/show_bug.cgi?id=14478

 so be careful: ranges with non-copyable elements aren't input ranges for
 now. ;-)

If

auto h = r.front;

doesn't work, then I don't think that it can be a range. And if that means
that elements have to be copyable to be in a range, then I think that
they're going to have to be copyable. There is going to be _way_ too much
code out there that does

auto h = r.front;

for it to work to try and claim that something is an input range when that
line doesn't work. And IIRC, C++ STL containers require that their elements
be copyable in order to work, so it's not like we'd be the first to require
this sort of thing.

- Jonathan M Davis

Re: oversight with input ranges

[Steven Schveighoffer via Digitalmars-d]

On 4/21/15 3:53 PM, ketmar wrote:

here's the interesting oversight for isInputRange:
https://issues.dlang.org/show_bug.cgi?id=14478

so be careful: ranges with non-copyable elements aren't input ranges for
now. ;-)


This does seem like an incorrect limitation, and I'm not sure if it was 
intentional. However, there is a lot of code out there that expects this 
to work when isInputRange is true. I don't know if we should change it, 
as the range definition is pretty clear in the documentation that auto h 
= r.front is a required feature for ranges. What is the use case for this?


-Steve

Re: oversight with input ranges

[ketmar via Digitalmars-d]

On Tue, 21 Apr 2015 18:57:50 -0400, Steven Schveighoffer wrote:

 On 4/21/15 3:53 PM, ketmar wrote:
 here's the interesting oversight for isInputRange:
 https://issues.dlang.org/show_bug.cgi?id=14478

 so be careful: ranges with non-copyable elements aren't input ranges
 for now. ;-)
 
 This does seem like an incorrect limitation, and I'm not sure if it was
 intentional. However, there is a lot of code out there that expects this
 to work when isInputRange is true. I don't know if we should change it,
 as the range definition is pretty clear in the documentation that auto h
 = r.front is a required feature for ranges. What is the use case for
 this?

one possible use case was shown in bugreport. array of non-copyable 
struct, yet i still want chain 'em, for example. or filter. or...

struct S {
  int n;
  @disable this (this);
}

void main () {
  S[2] arr;
  arr[0].n = 1;
  arr[1].n = 2;
  foreach (ref el; arr[].filter!((ref a) = a.n  1)) {
writeln(el.n);
  }
}


this code is perfectly valid, it doesn't require copying at all, yet it 
is invalid now.

signature.asc
Description: PGP signature

Re: Input ranges

[Jonathan M Davis via Digitalmars-d]

On Monday, April 20, 2015 07:54:32 via Digitalmars-d wrote:
 Hi,

 I tried to learn about input ranges and got some anonymous advice
 http://forum.dlang.org/thread/dezlxxygufocmafvl...@forum.dlang.org
 , still the topic seems to warrant broader attention.

 The question is how pure input ranges (as in non-forward) should
 be build. Algorithms like take and groupBy require reference
 semantics. I did not find that communicated. Now I am looking for
 the official ruling. As in: What the D rule book says.

Honestly, the cleanest behavior is that all ranges be reference types, and
save be required to copy the range. However, there are no actual
requirements that that be the case, and structs and arrays are what is most
frequently used for ranges, and they tend to be half-reference types or
value types. So, passing a range to a function usually ends up being
equivalent to a call to save, which makes it so that many range-based
functions don't work properly with class ranges, since they usually aren't
tested with them. And since arrays certainly aren't going to be full
reference types, it just isn't possible to require that a forward range be a
full reference type, so we need to support value type ranges, half reference
type ranges, and full reference type ranges when it comes to forward ranges,
and we have to deal with the fact that copying them _can_ be equivalent to
saving them without assuming that it is. :|

So, as far as forward ranges go, you shouldn't rely on copying a range being
equivalent to save (otherwise save wouldn't need to exist), but
realistically, if you pass a range to a function, you can't simply use that
range after. Many ranges will have saved themselves in the process, but not
all of them will have. The range in the caller is still valid (fortunately),
but whether it's in the same state that it was before calling the other
function depends entirely on whether copying the range is equivalent to save
or not, meaning that you just can't use a range after you've passed it to
another function unless you called save on it (forcing copy semantics) or
passed it by ref (forcing reference semantics). Unfortunately, there are
almost certainly lingering bugs in Phobos due to functions which assume that
passing range to another function implicitly saves it, simply because that's
what happens with such a large percentage of ranges.

With regard to pure input ranges, they're not forward ranges, because what
they're iterating over cannot be saved. So, by their very nature, they have
to be at least partially reference types and folks will often assume that
they are full reference types. The problem is that they can also be half
reference types in that they could have saved some state internally (e.g.
front) but still not be able to actually have a proper save function which
saves all of their state. So, if you pass a pure input range to a function,
then just like forward ranges, you cannot assume that you can use it again
without passing it by reference, even though by definition, their internal
state will have changed at least partially due to whatever iteration is done
within the function that it's passed to (since it has to be at least a
partial reference type to be a pure input range). However, unlike with a
forward range, the input range is not a full reference type, then it won't
be left in a valid state, since it won't have be saved, just had some of its
contents passed by reference and some of them by value.

All in all, I think that input ranges are incredibly annoying to work with,
and I'm frequently tempted to argue that they just shouldn't exist. They're
just too restrictive and tend to have weird behaviors, but unfcortunately,
there are cases where it's prohibitively expensive to have a forward range.
:|

In any case, all that is presently _required_ of ranges is what the traits
isInputRange, isForwardRange, etc. test for. We can't actually require more
than that, because the compiler can't check for more. What we _can_ do (and
has been discussed but never fully sorted out) is write up some additional
rules about what is expected by Phobos and encourage the community at large
to follow them with their ranges, but those can't actually be enforced, and
we've never been able to agree on those rules (particularly since, in most
cases, they aren't necessary, meaning that it only comes up in the corner
cases and hasn't been enough of a pain point to force change). For instance,
many of us argued that it should not be legal for empty to do any work, but
counter-examples were given where it was necessary. So, it becomes very
difficult to place additional requirements beyond what the API itself says -
even as guidelines with regards to what the standard library expects.

Now, as to pure input ranges and reference semantics, allowing half
reference type ranges is very dangerous, because any time that they're
copied, and the copy is iterated at all, the original will be left in an
invalid state (unlike

Re: Input ranges

[via Digitalmars-d-learn]

On Sunday, 19 April 2015 at 23:49:08 UTC, anonymous wrote:

On Sunday, 19 April 2015 at 21:42:23 UTC, Ulrich Küttler wrote:
groupBy is a nice example as it laboriously adds reference 
semantics to forward ranges but assumes input ranges to posses 
reference semantics by themselves.


All ranges are input ranges, though. Input ranges are the least 
specialised category. I think it's a mistake to assume/require 
anything only for input ranges that are not forward ranges.


Yes and no. It is reasonable to provide special implementations 
for ranges that are just input ranges. This is what groupBy does. 
The user gets a function that works with all kinds of ranges.




I guess we could require reference semantics for all input 
ranges (including forward ranges and higher-ups). That would be 
a rather clean way out of this mess. But it would be a major 
effort. And I guess it would hurt performance, maybe a lot.


Definitely, general reference semantics would solve a ton of 
difficulty. However, this would not be D anymore.


This seems to be the catch: For optimal performance memory layout 
is important. You cannot hide it behind an interface.





At this point the solution in byLineCopy feels ad hoc.


I think byLineCopy solves a different problem. ByLine already 
has this:

https://github.com/D-Programming-Language/phobos/blob/v2.067.0/std/stdio.d#L1592-L1598


The same indirection in byLineCopy. This is what I was referring 
to:


https://github.com/D-Programming-Language/phobos/blob/v2.067.0/std/stdio.d#L1783-L1789

Whoever did that knew very well what is going on.

Input ranges

[via Digitalmars-d]

Hi,

I tried to learn about input ranges and got some anonymous advice 
http://forum.dlang.org/thread/dezlxxygufocmafvl...@forum.dlang.org 
, still the topic seems to warrant broader attention.


The question is how pure input ranges (as in non-forward) should 
be build. Algorithms like take and groupBy require reference 
semantics. I did not find that communicated. Now I am looking for 
the official ruling. As in: What the D rule book says.

Re: Input ranges

[anonymous via Digitalmars-d-learn]

On Saturday, 18 April 2015 at 22:01:56 UTC, Ulrich Küttler wrote:

Input ranges from std.stdio are used for reading files. So
assuming we create a file

auto f = File(test.txt, w);
f.writeln(iota(5).map!(a = repeat(to!string(a), 
4)).joiner.joiner(\n));

f.close();

We should be able groupBy (chunkBy) its lines:

writeln(File(test.txt).byLine.groupBy!((a,b) = a == b));

The result is just one group, that is all lines are considered 
equal:


[[0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 
2, 3, 3, 3, 3, 4, 4, 4, 4]]


Alas, byLine reuses the same buffer for each line and thus
groupBy keeps comparing each line with itself. There is a 
version

of byLine that makes copies:

writeln(File(test.txt).byLineCopy.groupBy!((a,b) = a == 
b));


Indeed, the result is as expected:

[[0, 0, 0, 0], [1, 1, 1, 1], [2, 2, 
2, 2], [3, 3, 3, 3], [4, 4, 4, 4]]


Yeah, byLine is dangerous. byLineCopy should probably have been 
the default. Maybe we should rename byLine to byLineNoCopy (doing 
the proper deprecation dance, of course).



A final test with the undocumented byRecord method (the mapping
after groupBy is for beauty only and does not change the 
result):


writeln(File(test.txt)
.byRecord!string(%s)
.groupBy!((a,b) = a == b)
.map!(map!(a = a[0])));

Here, the result is most peculiar:

[[0, 0, 0, 0], [1, 1, 1], [2, 2, 2], 
[3, 3, 3], [4, 4, 4]]


Is byRecord broken? (It is undocumented after all.) In a way,
because it does not contain any indirection. The current fields
tuple is a simple member of the ByRecord struct.

In contrast, the ByLineCopy struct is just a wrapper to a ref
counted ByLineCopyImpl struct with a simple note:

/* Ref-counting stops the source range's ByLineCopyImpl
 * from getting out of sync after the range is copied, 
e.g.
 * when accessing range.front, then using 
std.range.take,

 * then accessing range.front again. */

I am uncomfortable at this point. Simple and efficient input
ranges fail in unexpected ways. Internal indirections make all
the difference. It feels like input ranges are hiding something
that should not be hidden.

What am I missing?


I guess the problem is the mix of value and reference semantics. 
ByRecord's `current` is a value, but its `file` has reference 
semantics. So, a copy of a ByRecord affects one part of the 
original but not the other.


Maybe copying should be `@disable`d for such ranges/structs. Then 
you couldn't pass it by value to groupBy. Instead you would have 
to use something like (the fixed version of) refRange, which 
works properly.

Re: Input ranges

[anonymous via Digitalmars-d-learn]

On Sunday, 19 April 2015 at 21:42:23 UTC, Ulrich Küttler wrote:
groupBy is a nice example as it laboriously adds reference 
semantics to forward ranges but assumes input ranges to posses 
reference semantics by themselves.


All ranges are input ranges, though. Input ranges are the least 
specialised category. I think it's a mistake to assume/require 
anything only for input ranges that are not forward ranges.


I guess we could require reference semantics for all input ranges 
(including forward ranges and higher-ups). That would be a rather 
clean way out of this mess. But it would be a major effort. And I 
guess it would hurt performance, maybe a lot.


[...]
Again, I agree. Disallow copying for such ranges would prevent 
the error, still it would be a bit harsh. It is not just 
groupBy that goes astray. You can also get strange results from 
take:


auto r = File(test.txt).byRecord!string(%s);
foreach (i; 0 .. 5)
writeln(r.take(4).map!(a = a[0]));


That would also not be possible if ByRecord had an `@disable 
this(this);`. But I'm not at all sure that this is the way to go. 
It's bound to be forgotten, and there are probably places where 
it's needlessly restrictive.


I was hoping to find some communication how input ranges should 
be done.


I'm right there with you. This is all a bit iffy. I suspect that 
this is an oversight in the design of ranges, as the 
documentation of std.range doesn't say anything on the topic.


This may be too advanced for D.learn. I don't think Andrei and 
Walter come down here very often. Maybe take it to the general 
board.



At this point the solution in byLineCopy feels ad hoc.


I think byLineCopy solves a different problem. ByLine already has 
this:

https://github.com/D-Programming-Language/phobos/blob/v2.067.0/std/stdio.d#L1592-L1598

Re: Input ranges

[via Digitalmars-d-learn]

On Sunday, 19 April 2015 at 11:33:26 UTC, anonymous wrote:
I guess the problem is the mix of value and reference 
semantics. ByRecord's `current` is a value, but its `file` has 
reference semantics. So, a copy of a ByRecord affects one part 
of the original but not the other.


I agree. Yet I am convinced most (all?) proper input ranges read 
input from an external source. (Reference semantic right there.) 
Input ranges are one-pass ranges after all. Therefore, reference 
semantics are required in any case (unless the use of the range 
is known beforehand.)


groupBy is a nice example as it laboriously adds reference 
semantics to forward ranges but assumes input ranges to posses 
reference semantics by themselves.


Maybe copying should be `@disable`d for such ranges/structs. 
Then you couldn't pass it by value to groupBy. Instead you 
would have to use something like (the fixed version of) 
refRange, which works properly.


Again, I agree. Disallow copying for such ranges would prevent 
the error, still it would be a bit harsh. It is not just groupBy 
that goes astray. You can also get strange results from take:


auto r = File(test.txt).byRecord!string(%s);
foreach (i; 0 .. 5)
writeln(r.take(4).map!(a = a[0]));

I was hoping to find some communication how input ranges should 
be done. At this point the solution in byLineCopy feels ad hoc.

Input ranges

[via Digitalmars-d-learn]

It seems input ranges without any indirection in memory are not
working well with algorithms. This seems to be understood by the
D community. I did not know. Here is my story on the topic so
far:

Recently, I learned that I did not know input ranges much at all,
totally misjudging std.range.refRange in its usefulness to input
ranges:
https://github.com/D-Programming-Language/phobos/pull/3123

At this point some experiments might be in order. (using 2.067.0)

Input ranges from std.stdio are used for reading files. So
assuming we create a file

auto f = File(test.txt, w);
f.writeln(iota(5).map!(a = repeat(to!string(a), 
4)).joiner.joiner(\n));

f.close();

We should be able groupBy (chunkBy) its lines:

writeln(File(test.txt).byLine.groupBy!((a,b) = a == b));

The result is just one group, that is all lines are considered 
equal:


[[0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 
3, 3, 3, 3, 4, 4, 4, 4]]


Alas, byLine reuses the same buffer for each line and thus
groupBy keeps comparing each line with itself. There is a version
of byLine that makes copies:

writeln(File(test.txt).byLineCopy.groupBy!((a,b) = a == 
b));


Indeed, the result is as expected:

[[0, 0, 0, 0], [1, 1, 1, 1], [2, 2, 2, 
2], [3, 3, 3, 3], [4, 4, 4, 4]]


A final test with the undocumented byRecord method (the mapping
after groupBy is for beauty only and does not change the result):

writeln(File(test.txt)
.byRecord!string(%s)
.groupBy!((a,b) = a == b)
.map!(map!(a = a[0])));

Here, the result is most peculiar:

[[0, 0, 0, 0], [1, 1, 1], [2, 2, 2], 
[3, 3, 3], [4, 4, 4]]


Is byRecord broken? (It is undocumented after all.) In a way,
because it does not contain any indirection. The current fields
tuple is a simple member of the ByRecord struct.

In contrast, the ByLineCopy struct is just a wrapper to a ref
counted ByLineCopyImpl struct with a simple note:

/* Ref-counting stops the source range's ByLineCopyImpl
 * from getting out of sync after the range is copied, 
e.g.

 * when accessing range.front, then using std.range.take,
 * then accessing range.front again. */

I am uncomfortable at this point. Simple and efficient input
ranges fail in unexpected ways. Internal indirections make all
the difference. It feels like input ranges are hiding something
that should not be hidden.

What am I missing?

[Issue 14373] std.range.refRange doesn't work on mere input ranges

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=14373

github-bugzi...@puremagic.com changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution|--- |FIXED

--

[Issue 14373] std.range.refRange doesn't work on mere input ranges

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=14373

--- Comment #2 from github-bugzi...@puremagic.com ---
Commits pushed to master at https://github.com/D-Programming-Language/phobos

https://github.com/D-Programming-Language/phobos/commit/ca4e7d22ecdc53d1529fd88c71ff94198ac44303
fix Issue 14373 - std.range.refRange doesn't work on mere input ranges

https://github.com/D-Programming-Language/phobos/commit/f4894c6302286c95fc63b48bdffeb1d658f5cd7a
Merge pull request #3123 from aG0aep6G/14373

fix Issue 14373 - std.range.refRange doesn't work on mere input ranges

--

[Issue 14373] New: std.range.refRange doesn't work on mere input ranges

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=14373

  Issue ID: 14373
   Summary: std.range.refRange doesn't work on mere input ranges
   Product: D
   Version: unspecified
  Hardware: All
OS: All
Status: NEW
  Severity: normal
  Priority: P1
 Component: Phobos
  Assignee: nob...@puremagic.com
  Reporter: ag0ae...@gmail.com

struct R
{
@property int front() {return 0;}
void popFront() {empty = true;}
bool empty = false;
}
void main()
{
import std.range: refRange;
R r;
refRange(r).popFront();
assert(r.empty); /* fails */
}

Pull request incoming.

--

Re: Using input ranges with std.regex?

[MrSmith via Digitalmars-d-learn]

On Wednesday, 25 April 2012 at 21:43:11 UTC, Dmitry Olshansky 
wrote:

On 25.04.2012 23:08, H. S. Teoh wrote:
Does std.regex support input ranges to match()? Or do I need 
to convert

to string first?



For now, yes you have to convert them. Any random access range 
of code units should do the trick but stringish template 
constraints might kill that.


I plan to extend this eventually. The problematic point is that 
match internally is delimited by integer offsets (indices). 
Forward ranges technically can work (the match then will return 
something like take(..., n);) with a bunch of extra .save 
calls. Input ranges can't be used at all.


Is there any progress on this thing?

Re: Combine Coroutines and Input Ranges for Dead-Simple D Iteration

[jerro]

What compiler options is that with?


I used DMD and compiler flags -O -inline -release on x86_64 linux.

Re: Combine Coroutines and Input Ranges for Dead-Simple D Iteration

[Rory McGuire]

On Wed, May 2, 2012 at 9:38 AM, jerro a...@a.com wrote:

 What compiler options is that with?


 I used DMD and compiler flags -O -inline -release on x86_64 linux.


It may be slow relative to incrementing a integer:

65 seconds without any command line options.
54 seconds with  -O -inline -release.

vs:

3.2 seconds dmd without any command line options.
2.2 seconds with dmd -O -inline -release.

And that is for 1000_000_000 Fiber context switches.
intel 2600K @ 4.5GHz, gnu/linux, ubuntu 12.04.

Re: Combine Coroutines and Input Ranges for Dead-Simple D Iteration

[SomeDude]

On Tuesday, 1 May 2012 at 08:26:45 UTC, Nick Sabalausky wrote:
A little write-up I just did on something I thought was pretty 
cool:


Combine Coroutines and Input Ranges for Dead-Simple D Iteration
https://www.semitwist.com/articles/article/view/combine-coroutines-and-input-ranges-for-dead-simple-d-iteration


Call me stupid, but I've absolutely no idea what you're doing. 
What problem does the InputVisitor solve ? What are the current 
solutions ? What is the intent of your code ? What are the 
supposed advantages ? Your article says nothing about it.

Re: Combine Coroutines and Input Ranges for Dead-Simple D Iteration

[Nick Sabalausky]

SomeDude lovelyd...@mailmetrash.com wrote in message 
news:ypakkndfsibcbgelj...@forum.dlang.org...
 On Tuesday, 1 May 2012 at 08:26:45 UTC, Nick Sabalausky wrote:
 A little write-up I just did on something I thought was pretty cool:

 Combine Coroutines and Input Ranges for Dead-Simple D Iteration
 https://www.semitwist.com/articles/article/view/combine-coroutines-and-input-ranges-for-dead-simple-d-iteration

 Call me stupid, but I've absolutely no idea what you're doing. What 
 problem does the InputVisitor solve ? What are the current solutions ? 
 What is the intent of your code ? What are the supposed advantages ? Your 
 article says nothing about it.

Just an easier-to-read/write alternative to an opApply or an input range. 
More natural and straightforward than a hand-written input range, and 
cleaner syntax than opApply (and without opApply's downside of not being 
usable as a range).

Re: Combine Coroutines and Input Ranges for Dead-Simple D Iteration

[jerro]

It may be slow relative to incrementing a integer:


The opApply isn't just incrementing an integer - it's
calling a function through a pointer. A loop that just
increments an integer is an order of magnitude faster.
This code (I used assembly because a compiler would
optimize away such a simple loop) runs in 0.27s on my
machine:

auto sum = 0;
auto n = 1000_000_000;

asm
{
mov EAX, n;
mov EBX, sum;
loop:
dec EAX;
inc EBX;
test EAX, EAX;
jne loop;
mov sum, EBX;
}

Ranges like iota are often as fast as using a for loop.
For example this code:

auto sum = 0;
foreach(i; iota(to!int(args[1])))
sum += i;

runs in 0.52 seconds when compiled with gdc with flags
-O2 -finline-functions -frelease. When compiled with -O3,
gcc uses paddd instruction and it runs in 0.1s.


And that is for 1000_000_000 Fiber context switches.


I'm not saying that D fibers are slow - fiber context
switches are way faster than thread context switches.
When using them for IO, such as in vibe.d, overhead
of fibers is negligible. But when used for iteration,
they are way slower than the alternatives, because in
that case there shouldn't be any context switches at all.

Re: Combine Coroutines and Input Ranges for Dead-Simple D Iteration

[Nick Sabalausky]

Nick Sabalausky seewebsitetocontac...@semitwist.com wrote in message 
news:jnr241$nh1$1...@digitalmars.com...
 SomeDude lovelyd...@mailmetrash.com wrote in message 
 news:ypakkndfsibcbgelj...@forum.dlang.org...
 On Tuesday, 1 May 2012 at 08:26:45 UTC, Nick Sabalausky wrote:
 A little write-up I just did on something I thought was pretty cool:

 Combine Coroutines and Input Ranges for Dead-Simple D Iteration
 https://www.semitwist.com/articles/article/view/combine-coroutines-and-input-ranges-for-dead-simple-d-iteration

 Call me stupid, but I've absolutely no idea what you're doing. What 
 problem does the InputVisitor solve ? What are the current solutions ? 
 What is the intent of your code ? What are the supposed advantages ? Your 
 article says nothing about it.

 Just an easier-to-read/write alternative to an opApply or an input range. 
 More natural and straightforward than a hand-written input range, and 
 cleaner syntax than opApply (and without opApply's downside of not being 
 usable as a range).


Of course, based on the timing results Jerro and Rory reported, Adam's mixin 
helper for opApply probably hits a better balance of performance vs 
usability.

Combine Coroutines and Input Ranges for Dead-Simple D Iteration

[Nick Sabalausky]

A little write-up I just did on something I thought was pretty cool:

Combine Coroutines and Input Ranges for Dead-Simple D Iteration
https://www.semitwist.com/articles/article/view/combine-coroutines-and-input-ranges-for-dead-simple-d-iteration
 

Re: Combine Coroutines and Input Ranges for Dead-Simple D Iteration

[Rory McGuire]

Fibers are basically just execution stacks like greenlets from python so I
would imagine you can clone/copy them.

Nice write up, I think it will help people get more fibre in their D diet.
:)

On Tue, May 1, 2012 at 10:27 AM, Nick Sabalausky 
seewebsitetocontac...@semitwist.com wrote:

 A little write-up I just did on something I thought was pretty cool:

 Combine Coroutines and Input Ranges for Dead-Simple D Iteration

 https://www.semitwist.com/articles/article/view/combine-coroutines-and-input-ranges-for-dead-simple-d-iteration

Re: Combine Coroutines and Input Ranges for Dead-Simple D Iteration

[Jacob Carlborg]

On 2012-05-01 10:27, Nick Sabalausky wrote:

A little write-up I just did on something I thought was pretty cool:

Combine Coroutines and Input Ranges for Dead-Simple D Iteration
https://www.semitwist.com/articles/article/view/combine-coroutines-and-input-ranges-for-dead-simple-d-iteration




Cool, that's how easy opApply should have been.

--
/Jacob Carlborg

Re: Combine Coroutines and Input Ranges for Dead-Simple D Iteration

[Robert Clipsham]

On 01/05/2012 09:27, Nick Sabalausky wrote:

A little write-up I just did on something I thought was pretty cool:

Combine Coroutines and Input Ranges for Dead-Simple D Iteration
https://www.semitwist.com/articles/article/view/combine-coroutines-and-input-ranges-for-dead-simple-d-iteration




This got me thinking, so I tried to do something a little similar 
myself, the result is below.


If you only have one type to iterate over, you can make the user code 
look even cleaner! I can't help but feel it can become even cleaner too.


(Scroll to the User code comment to miss out the implementation).

The code:

import core.thread;

class VFiber(Elem) : Fiber
{
Elem elem;
this(void delegate() dg)
{
super(dg);
}
}

auto visitor(T, Elem = T.visitType)(T t)
{
static struct Visitor
{
T t;
VFiber!Elem f;
@disable this();
@disable this(this);
this(T t)
{
this.t = t;
f = new VFiber!Elem(t.visit);
f.call();
}

Elem front() @property
{
return f.elem;
}

void popFront()
{
f.call();
}

bool empty() @property
{
return f.state == Fiber.State.TERM;
}
}
return Visitor(t);
}

void yield(Elem)(Elem el)
{
auto f = cast(VFiber!Elem)cast(void*)Fiber.getThis();
if (f is null) throw new FiberException(Cannot yield a value from 
outside the visit() method);

f.elem = el;
Fiber.yield();
}

// User code starts here
struct Iterable
{
string[] strs = [hello, world];
alias string visitType;

void visit()
{
foreach(str; strs)
yield(str);
}
}

void main()
{
import std.stdio;
Iterable i;
foreach(el; i.visitor)
{
writefln(%s, el);
}
}


--
Robert
http://octarineparrot.com/

Re: Combine Coroutines and Input Ranges for Dead-Simple D Iteration

[jerro]

On Tuesday, 1 May 2012 at 08:26:45 UTC, Nick Sabalausky wrote:
A little write-up I just did on something I thought was pretty 
cool:


Combine Coroutines and Input Ranges for Dead-Simple D Iteration
https://www.semitwist.com/articles/article/view/combine-coroutines-and-input-ranges-for-dead-simple-d-iteration


This is fun and all, but because of the horrible performance we 
really shouldn't be recommending people to use it.

Re: Combine Coroutines and Input Ranges for Dead-Simple D Iteration

[Nick Sabalausky]

jerro a...@a.com wrote in message 
news:sxfngaqnhwxqookrv...@forum.dlang.org...
 On Tuesday, 1 May 2012 at 08:26:45 UTC, Nick Sabalausky wrote:
 A little write-up I just did on something I thought was pretty cool:

 Combine Coroutines and Input Ranges for Dead-Simple D Iteration
 https://www.semitwist.com/articles/article/view/combine-coroutines-and-input-ranges-for-dead-simple-d-iteration

 This is fun and all, but because of the horrible performance we really 
 shouldn't be recommending people to use it.

So it is bad performance?

Re: Combine Coroutines and Input Ranges for Dead-Simple D Iteration

[Sean Kelly]

On May 1, 2012, at 1:50 PM, Nick Sabalausky wrote:

 jerro a...@a.com wrote in message 
 news:sxfngaqnhwxqookrv...@forum.dlang.org...
 On Tuesday, 1 May 2012 at 08:26:45 UTC, Nick Sabalausky wrote:
 A little write-up I just did on something I thought was pretty cool:
 
 Combine Coroutines and Input Ranges for Dead-Simple D Iteration
 https://www.semitwist.com/articles/article/view/combine-coroutines-and-input-ranges-for-dead-simple-d-iteration
 
 This is fun and all, but because of the horrible performance we really 
 shouldn't be recommending people to use it.
 
 So it is bad performance?

Compared to normal iteration schemes, yes.  It may be comparable to opApply in 
terms of performance though.  If you're curious, look at the Fiber context 
switching code (starting at switchIn and switchOut).

Re: Combine Coroutines and Input Ranges for Dead-Simple D Iteration

[jerro]

On Tuesday, 1 May 2012 at 20:49:48 UTC, Nick Sabalausky wrote:

jerro a...@a.com wrote in message
news:sxfngaqnhwxqookrv...@forum.dlang.org...

On Tuesday, 1 May 2012 at 08:26:45 UTC, Nick Sabalausky wrote:
A little write-up I just did on something I thought was 
pretty cool:


Combine Coroutines and Input Ranges for Dead-Simple D 
Iteration

https://www.semitwist.com/articles/article/view/combine-coroutines-and-input-ranges-for-dead-simple-d-iteration


This is fun and all, but because of the horrible performance 
we really shouldn't be recommending people to use it.


So it is bad performance?


It has bad performance compared to something like iota - I get
about 15 million iterations per second for this loop:

struct Iterable
{
 alias string visitType;

 void visit()
 {
 foreach(i; 0 .. 10_000_000)
 yield(i);
 }
}

...

foreach(el; visitor(i))
  sum ++;

 using Robert Clipsham's code, compared to billions iterations
per second with a regular loop. The problem is that people new
to D could see this, assume it performs similar to ranges like
iot and then wonder why their code is slow.

Re: Combine Coroutines and Input Ranges for Dead-Simple D Iteration

[jerro]

Compared to normal iteration schemes, yes.  It may be 
comparable to opApply in terms of performance though.  If 
you're curious, look at the Fiber context switching code 
(starting at switchIn and switchOut).


It's much slower than opApply. This loop takes 70s on my machine:


foreach(el; visitor(Iterable(1000_000_000)))
sum += 1;

And this one takes 2.7s:

auto iter(int n)
{
struct I
{
int n;

int opApply(int delegate(ref int) dg)
{
for(int i = 0; i  n; i++)
if(int r = dg(i))
return r;
return 0;
}
}

return I(n);
}

foreach(el; iter(1000_000_000))
sum += 1;

Re: Combine Coroutines and Input Ranges for Dead-Simple D Iteration

[Nick Sabalausky]

jerro a...@a.com wrote in message 
news:qpipqzzdbpkoxtzvh...@forum.dlang.org...
 Compared to normal iteration schemes, yes.  It may be comparable to 
 opApply in terms of performance though.  If you're curious, look at the 
 Fiber context switching code (starting at switchIn and switchOut).

 It's much slower than opApply. This loop takes 70s on my machine:


 foreach(el; visitor(Iterable(1000_000_000)))
 sum += 1;

 And this one takes 2.7s:

 auto iter(int n)
 {
 struct I
 {
 int n;

 int opApply(int delegate(ref int) dg)
 {
 for(int i = 0; i  n; i++)
 if(int r = dg(i))
 return r;
 return 0;
 }
 }

 return I(n);
 }

 foreach(el; iter(1000_000_000))
 sum += 1;


What compiler options is that with?

Using input ranges with std.regex?

[H. S. Teoh]

Does std.regex support input ranges to match()? Or do I need to convert
to string first?

Thanks!


T

-- 
Tell me and I forget. Teach me and I remember. Involve me and I understand. -- 
Benjamin Franklin

Re: Using input ranges with std.regex?

[Dmitry Olshansky]

On 25.04.2012 23:08, H. S. Teoh wrote:

Does std.regex support input ranges to match()? Or do I need to convert
to string first?



For now, yes you have to convert them. Any random access range of code 
units should do the trick but stringish template constraints might kill 
that.


I plan to extend this eventually. The problematic point is that match 
internally is delimited by integer offsets (indices). Forward ranges 
technically can work (the match then will return something like 
take(..., n);) with a bunch of extra .save calls. Input ranges can't be 
used at all.




--
Dmitry Olshansky

[Issue 5811] [patch] std.conv.parse isFloatingPoint!Target doesn't parse inf/-inf with input ranges

[d-bugmail]

http://d.puremagic.com/issues/show_bug.cgi?id=5811


Kenji Hara k.hara...@gmail.com changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 CC||k.hara...@gmail.com
 Resolution||DUPLICATE


--- Comment #1 from Kenji Hara k.hara...@gmail.com 2011-08-09 19:07:34 PDT ---
This problem was already fixed.

*** This issue has been marked as a duplicate of issue 3369 ***

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---