Re: Working with ranges
On Saturday, 29 May 2021 at 19:55:30 UTC, Elmar wrote: In many or most of the cases the use case doesn't actually require GC-allocation. Btw, I'm talking about core-level and systems software which concentrates on data transformations. When I only want to access a data structure but not mofify it then GC-allocation would not fit the lifetime logic of a variable. With "modification" I mean the data's size or it's order in memory but not the stored data itself.
Re: Working with ranges
On Wednesday, 26 May 2021 at 15:07:12 UTC, Jack wrote: On Wednesday, 26 May 2021 at 13:58:56 UTC, Elmar wrote: On Saturday, 8 December 2018 at 03:51:02 UTC, Adam D. Ruppe wrote: [...] That's amazing, this should be one thing that should appear in every tutorial just right at the start! I was looking hours for a way to generate an "iterator" (a range) from a fixed-size array which doesn't copy the elements (unless elements are deleted/added). [...] maybe array from std.array to make that range in array of its own? The main incentive here is, that I would like to obtain an iterator (some kind of access view) over a background storage which can be anywhere in memory which I don't care about. It might be on stack frame. In many or most of the cases the use case doesn't actually require GC-allocation. `array()` does GC-allocation and personally, I think `array()` should be avoided whereever the use case doesn't justify GC-allocation, at least if you care for *logically correct* memory management of your program. GC-allocation might just work the same way (most of the time even better than with stack-allocated storage due to design of D) and it adds convenience for you to omit explicit destruction calls which can spare you some conditional checks if the need for destruction depends on runtime cases. But with logical correctness I mean appropriateness here, an allocation scheme which reflects the nature of a variable's lifetime correctly. For example, if the lifetime, maximum storage requirements or the de-/allocation points in code are already known at compile-time then GC-allocation isn't appropriate. It has many drawbacks in performance critical sections, such as non-deterministic destruction time (which probably is the worst), the overhead of scanning GC-allocated regions and the memory fragmentation caused by dynamic allocation (i.e. non-deterministic available storage space) and in the worst case provides additional attack vectors, e.g. with heap overflows or use-after-free. In many cases, it is just better to GC-allocate an entire growable pool or slaps of objects for fast use-case specific allocation. So whatfor I would like to use an iterator? An iterator basically is a meta-data structure which stores meta data (like indices and pointers) for accessing another data structure's contents. And if I just want to change the access of or iteration over a data structure then I don't need to touch how the actual data or memory is stored and I don't even require expensive memory allocation when I could rearrange the iterator contents inplace and if the meta data is much smaller than the actual data. All that is not achieved by `array()`. `array()` is not an iterator but a dynamically allocated copy. Using an iterator like `array[]` saves me expensive GC-allocations. When I only want to access a data structure but not mofify it then GC-allocation would not fit the lifetime logic of a variable. When I understand correctly then the iterator concept in D is called "range". Ranges neither designate a data structure nor a specific data arrangement but it defines a generic access interface of aggregate data whose purpose is to work independent of whatever data structure is accessed via this interface. Now, I'm only missing methods to allocate range iterators on the stack or modifying iterators inplace.
Re: Working with ranges
On Wednesday, 26 May 2021 at 13:58:56 UTC, Elmar wrote: This example will not compile: ``` auto starts = arr[0..$].stride(2); auto ends = arr[1..$].stride(2); randomNumbers[] = ends[] - starts[]; ``` Because `[]` is not defined for the Result range. Is there a standard wrapper function which wraps an elementwise `[]` operator implementation around a range? Something like this ought to work: ```d import std.range: zip; import std.algorithm: map, copy; /// calls `fun` with the members of a struct or tuple as arguments alias apply(alias fun) = args => fun(args.tupleof); zip(starts, ends) .map!(apply!((start, end) => end - start)) .copy(randomNumbers[]); ``` In general, array operators like `[]` only work with arrays. The Result ranges you get from `stride` are not arrays, so to work with them, you need to use range algorithms like the ones in `std.range` and `std.algorithm`. (Some ranges actually do support `[]`, but it is never guaranteed. You can check for such support with [`std.range.primitives.hasSlicing`][1].) If you would prefer a more convenient syntax for working with things like strided arrays, I recommend giving [libmir][2] a look. It's a high-quality collection of D libraries for numerical and scientific computing. [1]: https://phobos.dpldocs.info/std.range.primitives.hasSlicing.html [2]: https://www.libmir.org/
Re: Working with ranges
On 5/26/21 8:07 AM, Jack wrote:
maybe array from std.array to make that range in array of its own?
Yes, something like this:
import std;
void main() {
auto arr = 10.iota.map!(i => uniform(0, 100));
auto starts = arr[0..$].stride(2);
auto ends = arr[1..$].stride(2);
auto randomNumbers = zip(ends, starts)
.map!(t => t[0] - t[1])
.array; // <-- Only when necessary
writeln(randomNumbers);
}
Ali
Re: Working with ranges
On Wednesday, 26 May 2021 at 13:58:56 UTC, Elmar wrote: On Saturday, 8 December 2018 at 03:51:02 UTC, Adam D. Ruppe wrote: [...] That's amazing, this should be one thing that should appear in every tutorial just right at the start! I was looking hours for a way to generate an "iterator" (a range) from a fixed-size array which doesn't copy the elements (unless elements are deleted/added). [...] maybe array from std.array to make that range in array of its own?
Re: Working with ranges
On Saturday, 8 December 2018 at 03:51:02 UTC, Adam D. Ruppe wrote: On Saturday, 8 December 2018 at 03:48:10 UTC, Murilo wrote: Try passing `ps[]` to the function instead of plain `ps` and see what happens. How do I transform an array into a range? With the slicing operator, []. That's amazing, this should be one thing that should appear in every tutorial just right at the start! I was looking hours for a way to generate an "iterator" (a range) from a fixed-size array which doesn't copy the elements (unless elements are deleted/added). But my issue now is, I have strided arrays (or just any Result range) and I want to use the slicing operator `[]` with that range to copy it into a fixed-size array or apply element-wise operations on it. How can I do that? This example will not compile: ``` auto starts = arr[0..$].stride(2); auto ends = arr[1..$].stride(2); randomNumbers[] = ends[] - starts[]; ``` Because `[]` is not defined for the Result range. Is there a standard wrapper function which wraps an elementwise `[]` operator implementation around a range?
Re: Working with ranges
Hi guys, thank you for helping me out here, there is this facebook group for the D language, here we can help and teach each other. It is called Programming in D. Please join. https://www.facebook.com/groups/662119670846705/?ref=bookmarks
Re: Working with ranges
On 12/7/18 11:16 PM, Adam D. Ruppe wrote: On Saturday, 8 December 2018 at 04:11:03 UTC, Murilo wrote: What is the difference between declaring "int[3] a = [1,2,3];" and declaring "int[] a = [1,2,3];"? Is the first an array and the second a range? They are both arrays, just the former one has a fixed size and the latter does not. Ranges require a way to iterate and consume elements, meaning they cannot be fixed size. I always thought that leaving the square brackets empty would create an array of flexible size, it never occurred to me that it was creating something else. That's what it is, just a flexible array also happens to be an array, whereas a fixed-size array is not one. I think, you mean "a flexible array also happens to be *a range*..." -Steve
Re: Working with ranges
On Saturday, 8 December 2018 at 04:16:25 UTC, Adam D. Ruppe wrote: On Saturday, 8 December 2018 at 04:11:03 UTC, Murilo wrote: What is the difference between declaring "int[3] a = [1,2,3];" and declaring "int[] a = [1,2,3];"? Is the first an array and the second a range? They are both arrays, just the former one has a fixed size and the latter does not. Ranges require a way to iterate and consume elements, meaning they cannot be fixed size. I always thought that leaving the square brackets empty would create an array of flexible size, it never occurred to me that it was creating something else. That's what it is, just a flexible array also happens to be an array, whereas a fixed-size array is not one. But a slice of a fixed size one yields a flexible one.. which is why the ps[] thing works to create a range out of it. Thank you guys so much for the explanation, it is all making more sense now.
Re: Working with ranges
On Saturday, 8 December 2018 at 04:11:03 UTC, Murilo wrote: What is the difference between declaring "int[3] a = [1,2,3];" and declaring "int[] a = [1,2,3];"? Is the first an array and the second a range? They are both arrays, just the former one has a fixed size and the latter does not. Ranges require a way to iterate and consume elements, meaning they cannot be fixed size. I always thought that leaving the square brackets empty would create an array of flexible size, it never occurred to me that it was creating something else. That's what it is, just a flexible array also happens to be an array, whereas a fixed-size array is not one. But a slice of a fixed size one yields a flexible one.. which is why the ps[] thing works to create a range out of it.
Re: Working with ranges
On Saturday, 8 December 2018 at 03:51:02 UTC, Adam D. Ruppe wrote: On Saturday, 8 December 2018 at 03:48:10 UTC, Murilo wrote: Try passing `ps[]` to the function instead of plain `ps` and see what happens. How do I transform an array into a range? With the slicing operator, []. Thank you very much, it worked now. What is the difference between declaring "int[3] a = [1,2,3];" and declaring "int[] a = [1,2,3];"? Is the first an array and the second a range? I always thought that leaving the square brackets empty would create an array of flexible size, it never occurred to me that it was creating something else.
Re: Working with ranges
On Friday, December 7, 2018 8:46:11 PM MST Adam D. Ruppe via Digitalmars-d- learn wrote: > On Saturday, 8 December 2018 at 03:37:56 UTC, Murilo wrote: > > Hi guys, I have created an array of strings with "string[12] ps > > string[12] isn't a range, but string[] is. > > Try passing `ps[]` to the function instead of plain `ps` and see > what happens. Specifically, the problem is that static arrays have a fixed length, which means that you can't pop elements off as is required for ranges. Dynamic arrays on the other hand are ranges (at least as long as you import std.range.primitives to get the range functions for dynamic arrays). Slicing a static array gives you a dynamic array which is a slice of the static array. So, mutating the elements of the dynamic array will mutate the elements of the static array, but the dynamic array can have elements popped off as is required for ranges, whereas the static array can't. - Jonathan M Davis
Re: Working with ranges
On Saturday, 8 December 2018 at 03:48:10 UTC, Murilo wrote: Try passing `ps[]` to the function instead of plain `ps` and see what happens. How do I transform an array into a range? With the slicing operator, [].
Re: Working with ranges
On Saturday, 8 December 2018 at 03:37:56 UTC, Murilo wrote: Hi guys, I have created an array of strings with "string[12] ps string[12] isn't a range, but string[] is. Try passing `ps[]` to the function instead of plain `ps` and see what happens.
Re: Working with ranges
On Saturday, 8 December 2018 at 03:46:11 UTC, Adam D. Ruppe wrote: On Saturday, 8 December 2018 at 03:37:56 UTC, Murilo wrote: Hi guys, I have created an array of strings with "string[12] ps string[12] isn't a range, but string[] is. Try passing `ps[]` to the function instead of plain `ps` and see what happens. How do I transform an array into a range?
Working with ranges
Hi guys, I have created an array of strings with "string[12] ps = ["cat", "dog", "lion", "wolf", "coin", "chest", "money", "gold", "A", "B", "C", "D"];". I want to use the array as a range and I want to randomize it, like I want to transform that into several other ranges with the same elements but in different orders, how do I do that? I tried using the function choice() from std.random but it gives an error message for some reason.
Re: Working with ranges: mismatched function return type inference
On Tuesday, October 11, 2016 10:42:42 Ali Çehreli via Digitalmars-d-learn
wrote:
> Those interfaces already exist in Phobos: :)
>
>https://dlang.org/phobos/std_range_interfaces.html
>
> auto foo(int[] ints) {
>import std.range;
>if (ints.length > 10) {
>return
> cast(RandomAccessFinite!int)inputRangeObject(chain(ints[0..5], ints[8..$]));
> } else {
>return cast(RandomAccessFinite!int)inputRangeObject(ints);
>}
> }
>
> void main() {
> import std.stdio;
> import std.range;
> import std.algorithm;
> writeln(foo([1, 2, 3]));
> writeln(foo(iota(20).array));
> }
And in this case, if you were considering doing that, you might as well just
concatenate the dynamic arrays rather than chaining them, because using
interfaces means allocating on the heap just like you would with
concatenating.
About the only time that using interfaces is the right solution with ranges
is when you're dealing with virtual functions (which can't be templatized),
and even then, it's not necessarily the best choice. Here, IMHO, it makes no
sense at all.
- Jonathan M Davis
Re: Working with ranges: mismatched function return type inference
On Tuesday, 11 October 2016 at 18:09:26 UTC, ag0aep6g wrote: You've got some options: Wow, thanks everyone, great information! I think I understand my options now.
Re: Working with ranges: mismatched function return type inference
On 10/11/2016 09:55 AM, orip wrote:
auto foo(int[] ints) {
import std.range;
if (ints.length > 10) {
return chain(ints[0..5], ints[8..$]);
} else {
//return ints; // Error: mismatched function return type inference
of int[] and Result
return chain(ints[0..0], ints[0..$]); // This workaround compiles
}
}
Is there a compatible return type that can be used, or some other
workaround?
You've got some options:
1) OOP with std.range.interfaces. Ali already showed how this work.
Comes at the cost of extra allocations and indirections.
2) std.range.choose wraps two different range types and uses forwards to
one of them based on a condition. Should be cheap. But you need
restructure your code a little:
auto foo(int[] ints) {
import std.range: chain, choose;
return choose(ints.length > 10,
chain(ints[0..5], ints[8..$]),
ints);
}
3) The workaround you already discovered: making a seemingly pointless
call to `chain` to get the types to match. Possibly the most efficient
solution. Looks a little odd.
Re: Working with ranges: mismatched function return type inference
On Tuesday, October 11, 2016 07:55:36 orip via Digitalmars-d-learn wrote:
> I get "Error: mismatched function return type inference" errors
> with choosing the return type for functions that work on ranges
> using, e.g, std.algorithm or std.range functions, but have
> different behavior based on runtime values. The return type is
> always a range with the same underlying type.
>
> Here's an example:
>
> auto foo(int[] ints) {
>import std.range;
>if (ints.length > 10) {
> return chain(ints[0..5], ints[8..$]);
>} else {
> //return ints; // Error: mismatched function return type
> inference of int[] and Result
> return chain(ints[0..0], ints[0..$]); // This workaround
> compiles
>}
> }
>
> Is there a compatible return type that can be used, or some other
> workaround?
> I couldn't find one when searching for the error or looking at
> the phobos source code.
>
> Thanks! orip
You're workaround is basically doing what you need to do. A function can
only return one type. The fact that both return statements are returning
ranges over the same kind of elements is irrelevant. They have to be
_exactly_ the same type. So, either you need to convert the range for the
first return statement into int[] so that it matches the second (e.g. by
calling array on the result or just using ~), or you need to call chain on
two int[]s for the second return statement so that it matches the first.
The second option (which your workaround does) is better if you don't intend
to convert the result to an array, since it avoids allocating an array, but
if you're just going to convert the result to int[] anyway, the first option
would be better.
Regardless, you can't have a function returning different types from
different return statements - even with auto. The compiler needs to know
exactly what the return type is whether you type it or not; auto just
infers it for you rather than requiring you to type it out.
- Jonathan M Davis
Re: Working with ranges: mismatched function return type inference
On 10/11/2016 10:28 AM, TheFlyingFiddle wrote:
On Tuesday, 11 October 2016 at 15:46:20 UTC, orip wrote:
On Tuesday, 11 October 2016 at 13:06:37 UTC, pineapple wrote:
Rewrite `return chain(ints[0..5], ints[8..$]);` as `return ints[0..5]
~ ints[8..$];`
The `chain` function doesn't return an array, it returns a
lazily-evaluated sequence of an entirely different type from `int[]`.
Of course it does! I would like the function to return an "input range
of int", no matter which one specifically. Is this possible?
It is, but you will have to use an interface / class to achieve this
behavior (or use some sort of polymorphic struct). Something like this
will do the trick:
import std.range;
import std.stdio;
interface IInputRange(T)
{
bool empty();
T front();
void popFront();
}
final class InputRange(Range) if(isInputRange!Range)
: IInputRange!(ElementType!Range)
{
Range r;
this(Range r)
{
this.r = r;
}
bool empty() { return r.empty; }
ElementType!Range front() { return r.front; }
void popFront() { r.popFront; }
}
auto inputRange(Range)(Range r)
{
return new InputRange!Range(r);
}
IInputRange!int foo(int[] ints)
{
import std.range;
if(ints.length > 10) {
return inputRange(chain(ints[0 .. 5], ints[8 .. $]));
} else {
return inputRange(ints);
}
}
void main()
{
auto ir = foo([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]);
auto ir2 = foo([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]);
writeln(ir);
writeln(ir2);
}
Those interfaces already exist in Phobos: :)
https://dlang.org/phobos/std_range_interfaces.html
auto foo(int[] ints) {
import std.range;
if (ints.length > 10) {
return
cast(RandomAccessFinite!int)inputRangeObject(chain(ints[0..5], ints[8..$]));
} else {
return cast(RandomAccessFinite!int)inputRangeObject(ints);
}
}
void main() {
import std.stdio;
import std.range;
import std.algorithm;
writeln(foo([1, 2, 3]));
writeln(foo(iota(20).array));
}
Ali
Re: Working with ranges: mismatched function return type inference
On Tuesday, 11 October 2016 at 15:46:20 UTC, orip wrote:
On Tuesday, 11 October 2016 at 13:06:37 UTC, pineapple wrote:
Rewrite `return chain(ints[0..5], ints[8..$]);` as `return
ints[0..5] ~ ints[8..$];`
The `chain` function doesn't return an array, it returns a
lazily-evaluated sequence of an entirely different type from
`int[]`.
Of course it does! I would like the function to return an
"input range of int", no matter which one specifically. Is this
possible?
It is, but you will have to use an interface / class to achieve
this behavior (or use some sort of polymorphic struct). Something
like this will do the trick:
import std.range;
import std.stdio;
interface IInputRange(T)
{
bool empty();
T front();
void popFront();
}
final class InputRange(Range) if(isInputRange!Range)
: IInputRange!(ElementType!Range)
{
Range r;
this(Range r)
{
this.r = r;
}
bool empty() { return r.empty; }
ElementType!Range front() { return r.front; }
void popFront() { r.popFront; }
}
auto inputRange(Range)(Range r)
{
return new InputRange!Range(r);
}
IInputRange!int foo(int[] ints)
{
import std.range;
if(ints.length > 10) {
return inputRange(chain(ints[0 .. 5], ints[8 .. $]));
} else {
return inputRange(ints);
}
}
void main()
{
auto ir = foo([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]);
auto ir2 = foo([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]);
writeln(ir);
writeln(ir2);
}
Re: Working with ranges: mismatched function return type inference
11.10.2016 18:46, orip пишет: On Tuesday, 11 October 2016 at 13:06:37 UTC, pineapple wrote: Rewrite `return chain(ints[0..5], ints[8..$]);` as `return ints[0..5] ~ ints[8..$];` The `chain` function doesn't return an array, it returns a lazily-evaluated sequence of an entirely different type from `int[]`. Of course it does! I would like the function to return an "input range of int", no matter which one specifically. Is this possible? it doesn't. Using runtime argument you can't choose compile time parameter - returned type. So it's impossible. Almost - b/c you can use Algebraic. Again you can do the following: ```D return chain(ints[0..5], ints[8..$]).array; // it returns int[] ```
Re: Working with ranges: mismatched function return type inference
On Tuesday, 11 October 2016 at 13:06:37 UTC, pineapple wrote: Rewrite `return chain(ints[0..5], ints[8..$]);` as `return ints[0..5] ~ ints[8..$];` The `chain` function doesn't return an array, it returns a lazily-evaluated sequence of an entirely different type from `int[]`. Of course it does! I would like the function to return an "input range of int", no matter which one specifically. Is this possible?
Re: Working with ranges: mismatched function return type inference
On Tuesday, 11 October 2016 at 07:55:36 UTC, orip wrote:
I get "Error: mismatched function return type inference" errors
with choosing the return type for functions that work on ranges
using, e.g, std.algorithm or std.range functions, but have
different behavior based on runtime values. The return type is
always a range with the same underlying type.
Here's an example:
auto foo(int[] ints) {
import std.range;
if (ints.length > 10) {
return chain(ints[0..5], ints[8..$]);
} else {
//return ints; // Error: mismatched function return type
inference of int[] and Result
return chain(ints[0..0], ints[0..$]); // This workaround
compiles
}
}
Is there a compatible return type that can be used, or some
other workaround?
I couldn't find one when searching for the error or looking at
the phobos source code.
Thanks! orip
Rewrite `return chain(ints[0..5], ints[8..$]);` as `return
ints[0..5] ~ ints[8..$];`
The `chain` function doesn't return an array, it returns a
lazily-evaluated sequence of an entirely different type from
`int[]`.
Working with ranges: mismatched function return type inference
I get "Error: mismatched function return type inference" errors
with choosing the return type for functions that work on ranges
using, e.g, std.algorithm or std.range functions, but have
different behavior based on runtime values. The return type is
always a range with the same underlying type.
Here's an example:
auto foo(int[] ints) {
import std.range;
if (ints.length > 10) {
return chain(ints[0..5], ints[8..$]);
} else {
//return ints; // Error: mismatched function return type
inference of int[] and Result
return chain(ints[0..0], ints[0..$]); // This workaround
compiles
}
}
Is there a compatible return type that can be used, or some other
workaround?
I couldn't find one when searching for the error or looking at
the phobos source code.
Thanks! orip
