Re: Is there something like a consuming take?
I start to understand: A (lazy) range is something like a voucher: byChunk() does not provide the data immediately, but only a voucher to hand you a chunk (an array of 2 bytes in my example above) a time. This voucher is passed to joiner(), which keeps that voucher and hands me out a new voucher for a byte a time (called "a" in my example above). This voucher is now passed to take(), whick again keeps that voucher and hands me out yet an other voucher for a byte a time, but limits itself to 5 items. Up to now nothing has happend, but the exchange of that vouchers. Now when writeln() is called, it shows take() that voucher and says: "Give me the first of your numbers". take() uses his voucher to do the same with joiner(), which uses its voucher to get the first chunk of byChunk(), takes the first byte of that chunk and hands it to take() and take() hands it to writeln(). Now writeln() uses it's voucher once again, to get the next byte. take() asks joiner() again, but this time joiner() does not ask byChunk(), because it still has one number left in the chunk it got when asking the last time. Now joiner() uses this second byte to hand it back. And so on. After writeln() finished I'm doing something evil: I use a private copy of that voucher, that I allready handed to take() to use it myself. That's sort of stealing back. And hence the results are more or less unpredictable. Especially, when I use this copy, before take() finished it's job, as Jonathan M Davis points out. My misthinking was to assume, that my copy of the voucher is still the same as the voucher, that take() owns after having handed out the 5 items, so I can use it to get the other items. If it where possible to ask take() to hand me back it's voucher, I could do it with this voucher, but I don't see a possibility to get that voucher back. What I actually want to do, when using take(), is not to get a new voucher, but I want to keep my voucher and just use this voucher to get the first 5 items - take() doesn't do that, so I've got to write my own function which cannot be lazy. Or better: I'd like to hand in my voucher and get back two vouchers, one for the first 5 bytes and one for the rest. That's actually, what I thought, take() is doing...
Re: D on ARM laptops?
On Friday, 5 July 2019 at 04:04:13 UTC, Jonathan M Davis wrote: On Thursday, July 4, 2019 2:49:18 AM MDT zoujiaqing via Digitalmars-d-learn wrote: LDC on FreeBSD working fine? It should work on FreeBSD 11. It almost certainly doesn't work on 12 because of bindings issues that still need to be resolved. Certainly, dmd does not work on FreeBSD 12, and ldc can't unless someone changed druntime specifically for ldc. Naturally, the patch for >=12 (64-bit inodes) has existed for a long time in FreeBSD Ports, so you can just `pkg install ldc` on 12 or 13-CURRENT. I've also done the work of porting to FreeBSD/aarch64 (64-bit ARMv8). You can't yet just pkg install it on aarch64 as the update has not yet landed in Ports, but the patch for Ports is in the FreeBSD bugzilla (and my github), so you can apply it and compile the package yourself.
Re: Is there something like a consuming take?
On Saturday, 6 July 2019 at 18:17:26 UTC, Jonathan M Davis wrote: take _always_ consumes the range that it's given not if it hasSlicing. see http://dpldocs.info/experimental-docs/source/std.range.d.html#L2015 but yeah otherwise i agree with you
Re: Is there something like a consuming take?
On Saturday, July 6, 2019 12:17:26 PM MDT Jonathan M Davis via Digitalmars- d-learn wrote: > On Saturday, July 6, 2019 8:12:36 AM MDT berni via Digitalmars-d-learn > > wrote: > > Now it's getting weird. Meanwhile I encountered, that take() > > sometimes consumes and sometimes not. Where can I learn, what is > > the reason behind this behavior? And how can I handle this? > > take _always_ consumes the range that it's given. The problem is that some > types of ranges are implicitly saved when they're copied, whereas others > aren't, and when a range is implicitly saved when it's copied, you end up > with the copy being consumed. Dynamic arrays are implicitly saved when > they're copied, so the range you pass is saved, and the copy is consumed > instead of the original. > > In generic code, you have to assume that once a range has been copied, you > can't use it anymore (just the copy) precisely because the semantics of > copying differ depending on the type of the range. You can only use a > range after copying it if you know what type of range you're dealing with > and how it behaves. So, you can rely on a dynamic array implicitly saving > when it's passed to take, but in generic code, you really shouldn't be > using a range again once you pass it to take, because what actually > happens is dependent on the type of the range. > > In general what this means is that if you pass a range to a function, and > you then want to use the range again afterwards, you need to call save > when passing it to the function, and otherwise, you just assume that it's > consumed and don't use it again. You certainly don't pass it to a > function with the expectation of some elements being consumed and then > continue to use the rest of the range unless the function takes its > argument by ref or by pointer (which relatively few range-based functions > do). > > If you want a function that's guaranteed to not implicitly copy a range, > then it needs to accept the argument by ref or take a pointer to it. In > the case where a function doesn't have to do the work lazily, ref would > work, but in a case like take where you're returning a wrapper range, > pointers would be required. So, a version of take that didn't ever copy > the range it's given and thus never risked implicitly saving the range it > was passed would have to either take a pointer to it or take it by ref > and then take the address of the ref. In either case, the code using such > a take would then have to ensure that the original range didn't leave > scope and get destroyed before the take range was consumed, or the take > range would then refer to invalid memory. Another thing to consider about lazy ranges that take pointers to avoid implicit saving and ensure that they consume the original is that if the original has anything popped from it before the wrapper range is fully consumed, then that will screw up the wrapper range, because the elements in the range it's wrapping will have changed. Wrapper ranges are really designed with the idea that they're operating on their own copy of the range. Even with take as it's currently implemented, you risk bugs if you pass it a range which is a reference type without calling save, because if you don't fully consume the take range before using the original range again, you end up screwing up the elements in the take range when you pop anything off of the original range. It really doesn't work well to have a wrapper range which doesn't have its own independent copy of the range to iterate over. The only case where it works cleanly to have a wrapper range like take gives you and not have to iterate through the elements again to have a range starting at the first element after the take range is to have a random-access range that has slicing, in which case, you can just slice it, and take isn't required. Without a random-access range with slicing, if you want to only iterate through the elements once, you really should just be iterating through the range manually rather than using a wrapper range. - Jonathan M Davis
Re: struct inside struct: Is there a way to call a function of the outside struct from the inner struct?
On Saturday, 6 July 2019 at 12:33:00 UTC, berni wrote: Now I found this: https://forum.dlang.org/thread/eobdqkkczquxoepst...@forum.dlang.org Seems to be intentional, that this doesn't work. In my case I'm able to move d() into the outer struct... You'll need a pointer to the outer struct, or run it in an function where it then passes a pointer of data that's seen in the scope i believe.
Re: Is there something like a consuming take?
On Saturday, July 6, 2019 8:12:36 AM MDT berni via Digitalmars-d-learn wrote: > Now it's getting weird. Meanwhile I encountered, that take() > sometimes consumes and sometimes not. Where can I learn, what is > the reason behind this behavior? And how can I handle this? take _always_ consumes the range that it's given. The problem is that some types of ranges are implicitly saved when they're copied, whereas others aren't, and when a range is implicitly saved when it's copied, you end up with the copy being consumed. Dynamic arrays are implicitly saved when they're copied, so the range you pass is saved, and the copy is consumed instead of the original. In generic code, you have to assume that once a range has been copied, you can't use it anymore (just the copy) precisely because the semantics of copying differ depending on the type of the range. You can only use a range after copying it if you know what type of range you're dealing with and how it behaves. So, you can rely on a dynamic array implicitly saving when it's passed to take, but in generic code, you really shouldn't be using a range again once you pass it to take, because what actually happens is dependent on the type of the range. In general what this means is that if you pass a range to a function, and you then want to use the range again afterwards, you need to call save when passing it to the function, and otherwise, you just assume that it's consumed and don't use it again. You certainly don't pass it to a function with the expectation of some elements being consumed and then continue to use the rest of the range unless the function takes its argument by ref or by pointer (which relatively few range-based functions do). If you want a function that's guaranteed to not implicitly copy a range, then it needs to accept the argument by ref or take a pointer to it. In the case where a function doesn't have to do the work lazily, ref would work, but in a case like take where you're returning a wrapper range, pointers would be required. So, a version of take that didn't ever copy the range it's given and thus never risked implicitly saving the range it was passed would have to either take a pointer to it or take it by ref and then take the address of the ref. In either case, the code using such a take would then have to ensure that the original range didn't leave scope and get destroyed before the take range was consumed, or the take range would then refer to invalid memory. - Jonathan M Davis
Re: Is there something like a consuming take?
On Saturday, 6 July 2019 at 14:48:04 UTC, Adam D. Ruppe wrote: [...] So this is a case of input range behavior - always consuming the underlying file - combined with buffering of two elements at once, leaving 5,6 behind, and the reuse of the buffer meaning you see that 5,6 again on the next call. Thanks for clearifing what happens. In my oppinion the behaviour of take() should be better defined. It's clear, that take() returns a range with the first n elements of the underlaying range (and that is done lazily). But it's not specified what happens with the underlaying range. As the behaviour is unpredictable (or at least hard to predict), one should assume, that the underlaying range is completely destroyed by take(). This makes take() much less usefull, than it could be, in my eyes. :-(
Re: Is there something like a consuming take?
On Saturday, 6 July 2019 at 14:40:23 UTC, berni wrote: .byChunk(BUFFER_SIZE) byChunk is defined to reuse its buffer between calls. http://dpldocs.info/experimental-docs/std.stdio.byChunk.1.html#examples This means previous contents are overwritten when you advance. When I now change BUFFER_SIZE to 2 I get: [ 1, 2, 3, 4, 5 ] [ 5, 6, 7, 8, 9, 10 ] Now the first two buffers have been consumend and the third ([5, 6]) not. So here, the take call gave you a view into the first 5 elements. It read one and two and printed them, then byChunk.popFront was called, overwriting the buffer with 3,4 and take passed that to writeln, then popFront again, overwriting with 5,6. writeln printed out 5, and take, having finished its work, left the buffer alone in its state. Now, you print the other thing, which still has 5,6 in the buffer, then popFront, overwrites with 7,8, etc and so on. So this is a case of input range behavior - always consuming the underlying file - combined with buffering of two elements at once, leaving 5,6 behind, and the reuse of the buffer meaning you see that 5,6 again on the next call.
Re: Is there something like a consuming take?
On Saturday, 6 July 2019 at 14:12:36 UTC, berni wrote:
Meanwhile I encountered, that take() sometimes consumes and
sometimes not.
It depends on what you're passing.
So take is defined as just getting the first N elements from the
given range. So what happens next depends on what it is "taking"
from (I don't like the name "take" exactly because that implies,
well, taking. What the function really does is more like "view
into first N elements".
With input ranges, iterating over them consumes it implicitly, so
you could say that take *always* consumes input ranges (at least
once it gets iterated over).
But, it will frequently consume a copy of the range instead of
the one you have at the top level, since ranges are passed by
value.
If you want it to be seen outside, `ref` is the general answer.
You might find success with refRange in some cases.
http://dpldocs.info/experimental-docs/std.range.refRange.html
But if you are passing something with slicing, like a plain
array, take will never actually consume it, and instead just
slice the input, even if it is ref. This gets the view of those
first elements in cheaper way.
So going back to your original definition:
I want to copy the first n items of a range to an array,
removing these items from the range.
As far as I know, none of the std.range functions are defined to
do this.
I'd probably write your own that:
1) takes the range by ref so changes are visible outside
2) iterates over it with popFront
3) returns the copy
that should fulfill all the requirements. you could slightly
optimize arrays (or other hasSlicing things) like
int[] yourFunction(ref int[] arr, int n) {
auto ret = arr[0 .. n];
arr = arr[n .. $];
return ret;
}
that is, just slicing and consuming in one go for each side and
then you don't even have to actually copy it, just return the
slice.
Re: Is there something like a consuming take?
A small example showing this strange behaviour:
import std.stdio;
import std.algorithm.iteration;
import std.range;
enum BUFFER_SIZE = 1024;
void main(string[] args)
{
auto a = (new File(args[1]))
.byChunk(BUFFER_SIZE)
.joiner;
writeln(a.take(5));
writeln(a);
}
Using a file, containing the bytes 1 to 10 I get:
[ 1, 2, 3, 4, 5 ]
[ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ]
take does not consume.
When I now change BUFFER_SIZE to 2 I get:
[ 1, 2, 3, 4, 5 ]
[ 5, 6, 7, 8, 9, 10 ]
Now the first two buffers have been consumend and the third ([5,
6]) not.
Feels like a bug in Phobos. But maybe I do not understand, what's
happening and this is correct behaviour. Can anyone explain or
confirm, that this is a bug?
Re: Is there something like a consuming take?
Now it's getting weird. Meanwhile I encountered, that take() sometimes consumes and sometimes not. Where can I learn, what is the reason behind this behavior? And how can I handle this?
Re: Is there something like a consuming take?
On Saturday, 6 July 2019 at 12:10:13 UTC, berni wrote:
On Saturday, 6 July 2019 at 11:48:51 UTC, a11e99z wrote:
Maybe I need to explain, what I dislike with this approach:
take() calls popFront n times and drop() calls popFront another
n times giving a total of 2n times (depending on the underlying
range, this might cause lot's of calulcations be done twice.
The first version with the foreach loop calls popFront only n
times.
auto take_consuming( R )( ref R r, int cnt ) {
import std.range.primitives : hasSlicing;
static if (hasSlicing!R) { // without allocations
auto tmp = r[0..cnt];
r = r[cnt..$]; // or r.popFronN( cnt ); // O(1)
return tmp;
} else { // loop range once
auto tmp = uninitializedArray!( ElementType!R[])( cnt);
int k = 0;
for (; !r.empty && kr.front;
return tmp[ 0..k];
}
}
Re: struct inside struct: Is there a way to call a function of the outside struct from the inner struct?
Now I found this: https://forum.dlang.org/thread/eobdqkkczquxoepst...@forum.dlang.org Seems to be intentional, that this doesn't work. In my case I'm able to move d() into the outer struct...
Re: For loop with separator
On Saturday, 6 July 2019 at 11:48:42 UTC, berni wrote:
On Thursday, 4 July 2019 at 17:00:33 UTC, Q. Schroll wrote:
The prime example is printing the comma when printing a list:
There is one between any two elements, but neither is one at
front or behind the last one.
If it is just for printing commas in between, you can use
range.join(", ")
https://dlang.org/phobos/std_array.html#.join
.map!(e=>e.text).join( ", "); // map for non strings
or
.format!"%(%s, %)"; // for anything
Re: Finding Max Value of Column in Multi-Dimesional Array
On Friday, 5 July 2019 at 19:56:54 UTC, ag0aep6g wrote: It works when you pass an actual callable instead, e.g. a lambda: p.map!(a => a[column]).maxElement.writeln; On Friday, 5 July 2019 at 20:22:14 UTC, dwdv wrote: Furthermore, Samir, the parameter `a` can be renamed to whatever you prefer or what fits the code at hand best, e.g. `(row => row[column])`, as opposed to the string version, where only a small set of mostly single character names is supported. Thank you very much for the explanation! As I've mentioned before, I have a lot to learn about the `map` function and the details provided here help immensely. Samir
Re: Is there something like a consuming take?
On Saturday, 6 July 2019 at 11:48:51 UTC, a11e99z wrote:
sure
auto take_consuming( R )( ref R r, int cnt ) {
auto tmp = r.take( cnt ).array;
r = r.drop( cnt );
return tmp;
}
don't thank
Doesn't look like what I'm looking for, as it is exactly the same
I allready found.
Maybe I need to explain, what I dislike with this approach:
take() calls popFront n times and drop() calls popFront another n
times giving a total of 2n times (depending on the underlying
range, this might cause lot's of calulcations be done twice. The
first version with the foreach loop calls popFront only n times.
Re: Is there something like a consuming take?
On Saturday, 6 July 2019 at 11:20:50 UTC, berni wrote:
I want to copy the first n items of a range to an array, I came
up with this now:
data = r.take(n).array;
This works partly, because the values of r are not consumed. So
I have to call afterwards:
r = r.drop(n);
Now I wonder, if it is possible to do this with one single
call, something like
data = r.take_consuming(n).array;
Does there something like this exist?
sure
auto take_consuming( R )( ref R r, int cnt ) {
auto tmp = r.take( cnt ).array;
r = r.drop( cnt );
return tmp;
}
don't thank
Re: For loop with separator
On Thursday, 4 July 2019 at 17:00:33 UTC, Q. Schroll wrote:
The prime example is printing the comma when printing a list:
There is one between any two elements, but neither is one at
front or behind the last one.
If it is just for printing commas in between, you can use
range.join(", ")
https://dlang.org/phobos/std_array.html#.join
struct inside struct: Is there a way to call a function of the outside struct from the inner struct?
struct A
{
void c() {}
struct B
{
void d()
{
c();
}
}
}
When compiling this with rdmd I get the message: "Error: this for
c needs to be type A not type B". Is there a way to call c from d?
Is there something like a consuming take?
I want to copy the first n items of a range to an array, removing
these items from the range.
This works:
foreach (i;0..n)
{
data ~= r.front;
r.popFront();
}
but looks a little bit arkward.
I came up with this now:
data = r.take(n).array;
This works partly, because the values of r are not consumed. So I
have to call afterwards:
r = r.drop(n);
Now I wonder, if it is possible to do this with one single call,
something like
data = r.take_consuming(n).array;
Does there something like this exist?
Re: Why are immutable array literals heap allocated?
On Friday, 5 July 2019 at 23:05:32 UTC, Jonathan M Davis wrote: Yes, I was wondering why the compiler doesn't statically allocate it automatically as an optimization. It would have to be set up to store the literal somewhere else. I was thinking the read-only data segment. Certainly, it can't just put it on the stack, because that risks it going out of scope and causing memory problems. It It can do that with small-ish sized literals if they don't escape (-dip1000). I think Walter may have advocated something like this. This one can be done even with literals of mutable elements. no matter what optimizations the compiler does with array literals, that shouldn't affect whether the function can be @nogc. For it to affect that, it would have to be something that was guaranteed by the language's semantics regardless of whether any optimizations were being done. Yes, just like string literals I think static allocation can be part of the language for immutable data.
Re: Why are immutable array literals heap allocated?
On 06.07.19 01:12, Patrick Schluter wrote:
On Friday, 5 July 2019 at 23:08:04 UTC, Patrick Schluter wrote:
On Thursday, 4 July 2019 at 10:56:50 UTC, Nick Treleaven wrote:
immutable(int[]) f() @nogc {
return [1,2];
}
[...]
and it cannot optimize it away because it doesn't know what the caller
want to do with it. It might in another module invoke it and modify it,
the compiler cannot tell. auto a=f(); a[0]++;
f returns immutable. typeof(a) is immutable(int[]). You can't do a[0]++.
