Re: block file reads and lazy utf-8 decoding
On Thursday, 10 December 2015 at 00:36:27 UTC, Jon D wrote:
Question I have is if there is a better way to do this. For
example, a different way to construct the lazy
'decodeUTF8Range' rather than writing it out in this fashion.
A further thought - The decodeUTF8Range function is basically
constructing a lazy wrapper range around decodeFront, which is
effectively combining a 'front' and 'popFront' operation. So
perhaps a generic way to compose a wrapper for such functions.
auto decodeUTF8Range(Range)(Range charSource)
if (isInputRange!Range && is(Unqual!(ElementType!Range) ==
char))
{
static struct Result
{
private Range source;
private dchar next;
bool empty = false;
dchar front() @property { return next; }
void popFront() {
if (source.empty) {
empty = true;
next = dchar.init;
} else {
next = source.decodeFront;
}
}
}
auto r = Result(charSource);
r.popFront;
return r;
}
block file reads and lazy utf-8 decoding
I want to combine block reads with lazy conversion of utf-8
characters to dchars. Solution I came with is in the program
below. This works fine. Has good performance, etc.
Question I have is if there is a better way to do this. For
example, a different way to construct the lazy 'decodeUTF8Range'
rather than writing it out in this fashion. There is quite a bit
of power in the library and I'm still learning it. I'm wondering
if I overlooked a useful alternative.
--Jon
Program:
---
import std.algorithm: each, joiner, map;
import std.conv;
import std.range;
import std.stdio;
import std.traits;
import std.utf: decodeFront;
auto decodeUTF8Range(Range)(Range charSource)
if (isInputRange!Range && is(Unqual!(ElementType!Range) ==
char))
{
static struct Result
{
private Range source;
private dchar next;
bool empty = false;
dchar front() @property { return next; }
void popFront() {
if (source.empty) {
empty = true;
next = dchar.init;
} else {
next = source.decodeFront;
}
}
}
auto r = Result(charSource);
r.popFront;
return r;
}
void main(string[] args)
{
if (args.length != 2) { writeln("Provide one file name.");
return; }
ubyte[1024*1024] rawbuf;
auto inputStream = args[1].File();
inputStream
.byChunk(rawbuf)// Read in blocks
.joiner // Join the blocks into a single
input char range
.map!(a => to!char(a)) // Cast ubyte to char for
decodeFront. Any better ways?
.decodeUTF8Range// utf8 to dchar conversion.
.each; // Real work goes here.
writeln("done");
}
Re: Reason for 'static struct'
On Wednesday, 9 December 2015 at 21:23:03 UTC, Daniel Kozák wrote:
V Wed, 09 Dec 2015 21:10:43 +
Jon D via Digitalmars-d-learn
napsáno:
There is a fair bit of range related code in the standard
library structured like:
auto MyRange(Range)(Range r)
if (isInputRange!Range)
{
static struct Result
{
private Range source;
// define empty, front, popFront, etc
}
return Result(r);
}
I'm curious about what declaring the Result struct as 'static'
does, and if there are use cases where it be better to exclude
the static qualifier.
--Jon
It make it non-nested struct:
https://dlang.org/spec/struct.html#nested
Thanks. So, is in the example above, would the advantage be that
'static' avoids saving the enclosing state, which is not needed?
Re: Reason for 'static struct'
V Wed, 09 Dec 2015 21:10:43 +
Jon D via Digitalmars-d-learn
napsáno:
> There is a fair bit of range related code in the standard library
> structured like:
>
> auto MyRange(Range)(Range r)
> if (isInputRange!Range)
> {
> static struct Result
> {
> private Range source;
> // define empty, front, popFront, etc
> }
> return Result(r);
> }
>
> I'm curious about what declaring the Result struct as 'static'
> does, and if there are use cases where it be better to exclude
> the static qualifier.
>
> --Jon
It make it non-nested struct: https://dlang.org/spec/struct.html#nested
Re: AA struct hashing bug?
heh. it crashed due to "in" presence. if you'll remove "in", it will work.
Reason for 'static struct'
There is a fair bit of range related code in the standard library
structured like:
auto MyRange(Range)(Range r)
if (isInputRange!Range)
{
static struct Result
{
private Range source;
// define empty, front, popFront, etc
}
return Result(r);
}
I'm curious about what declaring the Result struct as 'static'
does, and if there are use cases where it be better to exclude
the static qualifier.
--Jon
Re: std.algorithm.remove from array of custom classes?
On Wednesday, 9 December 2015 at 13:23:00 UTC, Tim K. wrote: On Wednesday, 9 December 2015 at 13:13:36 UTC, BBaz wrote: Is there a convenience function that allows me to remove an/all object(s) with a certain value from an array or do I need to write one myself? Here are some elements of strategy: import std.algorithm; import std.array; auto arr = [2, 3, 4, 5, 3, 2, 4]; // Remove all arr = arr.remove!(x => x==3); assert(arr == [2, 4, 5, 2, 4];); // remove all (filter works lazilly, we use array to get an array nonetheless) // filter is useful when composing functions, not so much here arr = arr.filter!(x => x==2).array; assert(arr == [4, 5, 4]); // Remove one, throw if the element is not found, // we use countUntil to get the index. arr = arr.remove(arr.countUntil(4)); assert(arr == [5, 4]); also you may be interested in reading this: http://p0nce.github.io/d-idioms/#Adding-or-removing-an-element-from-arrays
Re: benchmark on binary trees
version with apr (like in c version) http://dpaste.dzfl.pl/68c0157225e7 compiled with ldc it's indeed a bit faster on average : real0m1.999s user0m9.810s sys 0m0.148 btw Rust version is even faster than my little bit outdated gcc (4.9) latest try with allocators : http://dpaste.dzfl.pl/86b9b3c4ad71 swallows huge memory, slow ! what am i doing wrong ? (concurrency problem ?)
Re: Container Purity
On Wednesday, 9 December 2015 at 11:46:45 UTC, Kagamin wrote: Allocators usually use global state. Such code is usually treated as impure. What about containers that store their own local allocator? Will DMD infer all members of such containers to be pure if they only access locally allocated structures?
Re: std.algorithm.remove from array of custom classes?
On Wednesday, 9 December 2015 at 13:13:36 UTC, BBaz wrote:
3) opEquals can be 'const' because the method doesn't mutate
the state of the object
4) your cast wasn't safe
http://dlang.org/phobos/std_algorithm_mutation.html#.remove
Maybe something like this works better:
...
override bool opEquals(T)(T obj) const
if (is(T : A))
{
return (s == o.s) && (u == o.u);
}
...
Re: std.algorithm.remove from array of custom classes?
On Wednesday, 9 December 2015 at 13:23:00 UTC, Tim K. wrote:
On Wednesday, 9 December 2015 at 13:13:36 UTC, BBaz wrote:
1) remove works with an index
I guess I did read it wrong. Sorry.
Is there a convenience function that allows me to remove an/all
object(s) with a certain value from an array or do I need to
write one myself?
2) remove does not remove in place
That's fine.
void main(string[] argv)
{
A a = new A("a", 1);
A b = new A("b", 2);
A[] as = [a, b];
as = as.remove!(x => x == a);
writeln(as);
}
If array is sorted you can find the index of that element and
then remove it by index.
Re: std.algorithm.remove from array of custom classes?
On Wednesday, 9 December 2015 at 13:13:36 UTC, BBaz wrote: 1) remove works with an index I guess I did read it wrong. Sorry. Is there a convenience function that allows me to remove an/all object(s) with a certain value from an array or do I need to write one myself? 2) remove does not remove in place That's fine.
Re: std.algorithm.remove from array of custom classes?
On Wednesday, 9 December 2015 at 13:05:31 UTC, Tim K. wrote:
Hi!
I'm trying to remove an item from an array of objects. But I
get error messages when compiling (see below) which I do not
understand. I figured I had to override opEquals for it to
work, but no.
How do I get this to work?
You should read the documentation, there is two errors:
1) remove works with an index
2) remove does not remove in place
so with
~~
import std.stdio;
import std.algorithm;
class A
{
this(string si, uint ui) { s = si; u = ui; }
string s;
uint u;
override bool opEquals(Object obj) const
{
if (A o = cast(A)obj)
return (s == o.s) && (u == o.u);
else
return false;
}
}
void main(string[] argv)
{
A a = new A("a", 1);
A b = new A("b", 2);
A[] as = [a, b];
as = as.remove(0);
writeln(as);
}
~~
also, off topic but 2 other advices/errors
3) opEquals can be 'const' because the method doesn't mutate the
state of the object
4) your cast wasn't safe
http://dlang.org/phobos/std_algorithm_mutation.html#.remove
std.algorithm.remove from array of custom classes?
Hi!
I'm trying to remove an item from an array of objects. But I get
error messages when compiling (see below) which I do not
understand. I figured I had to override opEquals for it to work,
but no.
How do I get this to work?
Regards
class A
{
this(string si, uint ui) { s = si; u = ui; }
string s;
uint u;
override bool opEquals(Object obj)
{
A o = cast(A)obj;
return (s == o.s) && (u == o.u);
}
}
int main(string[] argv)
{
import std.stdio: writeln;
import std.algorithm: remove;
A a = new A("a", 1);
A b = new A("b", 2);
A[] as = [a, b];
as.remove(a);
writeln(as);
return 0;
}
Error:
/usr/include/dmd/phobos/std/algorithm/mutation.d(1503): Error:
template std.range.primitives.popFrontExactly cannot deduce
function from argument types !()(A[], A), candidates are:
/usr/include/dmd/phobos/std/range/primitives.d(1791):
std.range.primitives.popFrontExactly(Range)(ref Range r, size_t
n) if (isInputRange!Range)
/usr/include/dmd/phobos/std/algorithm/mutation.d(1504): Error:
template std.range.primitives.popFrontExactly cannot deduce
function from argument types !()(A[], A), candidates are:
/usr/include/dmd/phobos/std/range/primitives.d(1791):
std.range.primitives.popFrontExactly(Range)(ref Range r, size_t
n) if (isInputRange!Range)
/usr/include/dmd/phobos/std/algorithm/mutation.d(1505): Error:
cannot implicitly convert expression (from) of type dummy.A to
ulong
dummy.d(21): Error: template instance
std.algorithm.mutation.remove!(cast(SwapStrategy)2, A[], A) error
instantiating
Re: Container Purity
Allocators usually use global state. Such code is usually treated as impure.
Re: Container Purity
On Wednesday, 9 December 2015 at 10:47:18 UTC, Nordlöw wrote: that uses std.experimental.container Correction: I mean std.experimental.allocator
Container Purity
Is it currently possible for a container such as https://github.com/economicmodeling/containers/blob/master/src/containers/dynamicarray.d that uses std.experimental.container to be completely pure? If not can be made to? I wonder because only length(), empty(), front() and back() are tagged as pure in DynamicArray at https://github.com/economicmodeling/containers/blob/master/src/containers/dynamicarray.d are marked as pure. Is there a reason for this?
Re: Real Time-ing
On Tuesday, 8 December 2015 at 16:40:04 UTC, Taylor Hillegeist wrote: However i seem to get jitter of around 1 ms. Is there anything else i can do to improve? Do you want to get precision better than period of thread switches?
Re: Real Time-ing
On Wednesday, 9 December 2015 at 10:00:46 UTC, Andrea Fontana
wrote:
On Tuesday, 8 December 2015 at 15:35:18 UTC, Taylor Hillegeist
wrote:
So, I mostly do programming that is of run to completion
verity. But I have a dream of calling functions periodically.
So my question is:
What is the best (most time accurate) way to call a function
every n time units?
What is the best way to measure the jitter of these calls?
I'm also interested in waiting vs calling periodically eg.
call
wait(1 ms)
call
is not the same as 1 ms from call to call, due to the time
duration of the function call.
Thanks!
import core.thread;
import std.datetime;
import std.algorithm.comparison;
import std.math;
import std.stdio;
import core.time;
long loopTime = 0;
long CmdTime = 500_000_000; //Time in ns
void main()
{
for(int x = 0; x<20; x++){
auto time = TickDuration.currSystemTick.nsecs;
myPrinter(loopTime);
while(TickDuration.currSystemTick.nsecs - time
< CmdTime){}
loopTime = TickDuration.currSystemTick.nsecs -
time;
}
}
void myPrinter(long time){
writeln(time," nsecs with jitter of :",
abs(CmdTime-time), " nsecs");
}
What about this?
Anyway in order to avoid error accumulation it's probably a good
idea not to reset timer every time if possibile.
If you know the operations you need to perform, probably it works
better to store target time rather than delta.
So instead of:
- move x for 5 seconds
- move y for 3 seconds
- move z for 4 seconds
- ...
I think this works better:
- move x since second 5
- move y since second 8
- move z since second 12
So errors won't accumulate.
Re: Real Time-ing
On Tuesday, 8 December 2015 at 15:35:18 UTC, Taylor Hillegeist
wrote:
So, I mostly do programming that is of run to completion
verity. But I have a dream of calling functions periodically.
So my question is:
What is the best (most time accurate) way to call a function
every n time units?
What is the best way to measure the jitter of these calls?
I'm also interested in waiting vs calling periodically eg.
call
wait(1 ms)
call
is not the same as 1 ms from call to call, due to the time
duration of the function call.
Thanks!
import core.thread;
import std.datetime;
import std.algorithm.comparison;
import std.math;
import std.stdio;
import core.time;
long loopTime = 0;
long CmdTime = 500_000_000; //Time in ns
void main()
{
for(int x = 0; x<20; x++){
auto time = TickDuration.currSystemTick.nsecs;
myPrinter(loopTime);
while(TickDuration.currSystemTick.nsecs - time <
CmdTime){}
loopTime = TickDuration.currSystemTick.nsecs -
time;
}
}
void myPrinter(long time){
writeln(time," nsecs with jitter of :",
abs(CmdTime-time), " nsecs");
}
What about this?
