Re: Simplest multithreading example
On Friday, 1 September 2017 at 04:43:29 UTC, Ali Çehreli wrote:
On 08/31/2017 06:59 PM, Brian wrote:
> Hello, I am trying to get the most trivial example of
multithreading
> working, but can't seem to figure it out.
> I want to split a task across threads, and wait for all those
tasks to
> finish before moving to the next line of code.
>
> The following 2 attempts have failed :
>
> -
> Trial 1 :
> -
>
> auto I = std.range.iota(0,500);
> int [] X; // output
> foreach (i; parallel(I) )
> X ~= i;
> core.thread.thread_joinAll(); // Apparently no applicable
here ?
As Michael Coulombe said, parallel() does that implicitly.
If the problem is to generate numbers in parallel, I
restructured the code by letting each thread touch only its
element of a results array that has already been resized for
all the results (so that there is no race condition):
import std.stdio;
import std.parallelism;
import std.range;
void main() {
auto arrs = new int[][](totalCPUs);
const perWorker = 10;
foreach (i, arr; parallel(arrs)) {
const beg = cast(int)i * perWorker;
const end = beg + perWorker;
arrs[i] = std.range.iota(beg,end).array;
}
writeln(arrs);
}
If needed, std.algorithm.joiner can be used to make it a single
sequence of ints:
import std.algorithm;
writeln(arrs.joiner);
Ali
Hello, thank you very much for your quick replies !
I was trying to make a trivial example, but the 'real' problem is
trying to split a huge calculation to different threads.
Schematically :
double [] hugeCalc(int i){
// Code that takes a long time
}
so if I do
double[][int] _hugeCalcCache;
foreach(i ; I)
_hugeCalcCache[i] = hugeCalc(i);
of course the required time is I.length * (a long time), so I
wanted to shorten this by splitting to different threads :
foreach(i ; parallel(I) )
_hugeCalcCache[i] = hugeCalc(i);
but as you can guess, it doesn't work that easily.
Very interesting approach about letting only the thread touch a
particular element, I will try that.
FYI I did manage to make the following work, but not sure if this
is really still multi-threaded ?
int [] I;
foreach (i; 0 .. 500) I ~= i;
int [] X; // output
class DerivedThread : Thread {
private int [] i;
this(int [] i){
this.i = i;
super(&run);
}
private void run(){
synchronized{ // Need synchronization here !
foreach( i0; i)
X ~= i0;
}
}
}
Thread [] threads;
foreach (i; std.range.chunks( I, 50 ) )
threads ~= new DerivedThread( i);
foreach( thread; threads)
thread.start();
core.thread.thread_joinAll(); // Does in fact seem to 'join all'
threads
writeln(X);
Re: Simplest multithreading example
On 08/31/2017 06:59 PM, Brian wrote:
> Hello, I am trying to get the most trivial example of multithreading
> working, but can't seem to figure it out.
> I want to split a task across threads, and wait for all those tasks to
> finish before moving to the next line of code.
>
> The following 2 attempts have failed :
>
> -
> Trial 1 :
> -
>
> auto I = std.range.iota(0,500);
> int [] X; // output
> foreach (i; parallel(I) )
> X ~= i;
> core.thread.thread_joinAll(); // Apparently no applicable here ?
As Michael Coulombe said, parallel() does that implicitly.
If the problem is to generate numbers in parallel, I restructured the
code by letting each thread touch only its element of a results array
that has already been resized for all the results (so that there is no
race condition):
import std.stdio;
import std.parallelism;
import std.range;
void main() {
auto arrs = new int[][](totalCPUs);
const perWorker = 10;
foreach (i, arr; parallel(arrs)) {
const beg = cast(int)i * perWorker;
const end = beg + perWorker;
arrs[i] = std.range.iota(beg,end).array;
}
writeln(arrs);
}
If needed, std.algorithm.joiner can be used to make it a single sequence
of ints:
import std.algorithm;
writeln(arrs.joiner);
Ali
Re: Simplest multithreading example
On Friday, 1 September 2017 at 01:59:07 UTC, Brian wrote:
Hello, I am trying to get the most trivial example of
multithreading working, but can't seem to figure it out.
I want to split a task across threads, and wait for all those
tasks to finish before moving to the next line of code.
The following 2 attempts have failed :
-
Trial 1 :
-
auto I = std.range.iota(0,500);
int [] X; // output
foreach (i; parallel(I) )
X ~= i;
core.thread.thread_joinAll(); // Apparently no applicable here ?
writeln(X); // some random subset of indices
Trial 2 : (closer to Java)
class DerivedThread : Thread
{
int [] X;
int i;
this(int [] X, int i){
this.X = X;
this.i = i;
super(&run);
}
private:
void run(){
X ~= i;
}
}
void main(){
auto I = std.range.iota(0,500);
int [] X; // output
Thread [] threads;
foreach (i; I )
threads ~= new DerivedThread( X,i);
foreach( thread; threads)
thread.start();
foreach( thread; threads)
thread.join(); // does not seem to do anything
core.thread.thread_joinAll(); // also not doing anything
writeln(X); // X contains nothing at all
}
How can I get the program to wait until all threads have
finished before moving to the next line of code ?
Thank you !
Just like a sequential loop, when you do "foreach (i; parallel(I)
) { ... }", execution will not continue past the foreach loop
until all the tasks associated with each element of I have
finished.
Your particular example of "X ~= i" in the body of the loop is
not thread-safe, so if that is the code you really intend to run,
you should protect X with a Mutex or something comparable.
Simplest multithreading example
Hello, I am trying to get the most trivial example of
multithreading working, but can't seem to figure it out.
I want to split a task across threads, and wait for all those
tasks to finish before moving to the next line of code.
The following 2 attempts have failed :
-
Trial 1 :
-
auto I = std.range.iota(0,500);
int [] X; // output
foreach (i; parallel(I) )
X ~= i;
core.thread.thread_joinAll(); // Apparently no applicable here ?
writeln(X); // some random subset of indices
Trial 2 : (closer to Java)
class DerivedThread : Thread
{
int [] X;
int i;
this(int [] X, int i){
this.X = X;
this.i = i;
super(&run);
}
private:
void run(){
X ~= i;
}
}
void main(){
auto I = std.range.iota(0,500);
int [] X; // output
Thread [] threads;
foreach (i; I )
threads ~= new DerivedThread( X,i);
foreach( thread; threads)
thread.start();
foreach( thread; threads)
thread.join(); // does not seem to do anything
core.thread.thread_joinAll(); // also not doing anything
writeln(X); // X contains nothing at all
}
How can I get the program to wait until all threads have finished
before moving to the next line of code ?
Thank you !
Re: std.algorithm.joiner unexpected behavior
On Thu, Aug 31, 2017 at 05:37:20PM -0600, Jonathan M Davis via Digitalmars-d-learn wrote: > On Thursday, August 31, 2017 14:09:55 H. S. Teoh via Digitalmars-d-learn [...] > I know. We've had this argument before. I know. Let's not rehash that. :-) [...] > Even copying ranges in generic code does not have defined behavior, > because different types have different semantics even if they follow > the range API and pass isInputRange, isForwardRange, etc. This is actually one of the prime reasons for my stance that we ought to always use .save instead of assigning .front to a local variable. Consider the case where .front returns a subrange. As you state above, copying this subrange does not have defined behaviour. One reason is the difference in semantics between reference types and value types: the assignment operator `=` means different things for each kind of type. `x=y;` for a value type makes a new copy of the value in x, but `x=y;` for a reference type only copies the reference, not the value. So how do you ensure that after assigning .front to a local variable, it will continue to be valid after .popFront is called on the outer range? You can't. If you simply assign it, there's no guarantee it isn't just copying a reference to a buffer reused by .popFront. But if you try to copy it, the result is not defined, as you said. Worse yet, the user can overload opAssign() to do something with side-effects. So this code: auto e = r.front; r.popFront(); userCallback(e); may potentially have a different result from: userCallback(r.front); r.popFront(); The only safe approach is to make as few assumptions as possible, i.e., don't assume that `=` will produce the right result, so avoid saving anything in local variables completely and always use .save instead if you need to refer to a previous value of .front after calling .popFront. Yes this greatly complicates generic code, and I wouldn't impose it on user code. But one would expect that Phobos, at the very least, ought to be of a high enough standard to be able to handle such things correctly. Taking a step back from these nitpicky details, though: this seems to be symptomic of the underlying difficulty of nailing exact range semantics in an imperative language. In a pure functional language without mutation, you wouldn't have such issues, so there you could compose arbitrarily complex ranges of arbitrarily complex behaviours with impunity and not have to worry that something might break if one of the subranges was transient. We can't kill mutation in D, though, so unfortunately we have to live with these complications. T -- Life begins when you can spend your spare time programming instead of watching television. -- Cal Keegan
Re: Valid File Path
On Thursday, August 31, 2017 23:23:17 Vino via Digitalmars-d-learn wrote:
> On Thursday, 31 August 2017 at 21:59:22 UTC, vino wrote:
> > Hi All,
> >
> > Can you help me what is wrong in the below program, as the
> >
> > output is empty.
> >
> > import std.stdio, std.path;
> >
> > void main ()
> > {
> > version (Windows)
> > {
> > auto Path = `C:\#Users\Admin\Desktop\Script\Test1`;
> > if(!Path.isValidPath) { writeln("Path Not Valid");}
> > }
> > }
> >
> > From,
> > Vino.B
>
> Hi,
>
> The path is not valid as it contains the character # after the
> drive name.
And why would that not be valid? isValidPath and isValidFilename are quite
specific about what they think are valid path/file names, and having a # in
a file name is perfectly legitimate. They do have some extra restrictions
for Windows, since Windows is a lot pickier about its filenames than the
rest of the world, but # is not one of the characters listed as invalid:
https://dlang.org/phobos/std_path.html#isValidPath
https://dlang.org/phobos/std_path.html#isValidFilename
It would be invalid to have # as a drive name, but it's perfectly legal in a
filename or directory name.
- Jonathan M Davis
Re: std.algorithm.joiner unexpected behavior
On Thursday, August 31, 2017 14:09:55 H. S. Teoh via Digitalmars-d-learn wrote: > On Thu, Aug 31, 2017 at 01:34:39PM -0600, Jonathan M Davis via > Digitalmars-d-learn wrote: [...] > > > In general, byLine does not work with other range-based algorithms > > precisely because it reuses the buffer. I think that it does manage to > > work for some, but IMHO, it should have just supported foreach via > > opApply and not been a range at all. It's great for efficiency in a > > loop but horrible for range chaining. > > [...] > > Transient ranges are tricky to work with, I agree, but I don't agree > that they're "horrible for range chaining". I argue that many range > algorithms that assume the persistence of .front are inherently wrong, > and ought to be implemented in a way that does *not* make this > assumption. Many std.algorithm algorithms *can* in fact be written in > this way, and those that aren't, are arguably buggy. > > For example, some std.algorithm functions take a forward range but then > tries to save .front to a local variable. Rather, they should use .save > to save the previous position of the range so that they can call .front > on that to access the previous element, instead of making the unfounded > assumption that whatever the local variable refers to will still remain > valid after calling .popFront. It's just sloppy coding IMNSHO. I know. We've had this argument before. Personally, I think that the range spec should require that front _not_ be transient and that any ranges that are be considered non-conformant. Yes, under some set of circumstances, they can be made to work, but IMHO, it's simply not worth it. As it is, simply calling save when it's supposed to be called gets totally botched all the time even if ranges with transient front aren't involved. And adding such ranges just makes it too complicated. What we have currently with the range API allows for a lot of stuff to work correctly as long as certain assumptions are bet, but as soon as folks start doing stuff that doesn't behave the same way that a dynamic array does, things start falling apart. Even copying ranges in generic code does not have defined behavior, because different types have different semantics even if they follow the range API and pass isInputRange, isForwardRange, etc. The status quo works well enough that we get by, but it's a mess when you get into the details - especially if you want code that's actually generic. And IMHO, trying to add ranges with a transient front into the mix is taking it way to far. It's simply not worth it. But I know that you don't agree with that. - Jonathan M Davis
Re: Valid File Path
On Thursday, 31 August 2017 at 21:59:22 UTC, vino wrote:
Hi All,
Can you help me what is wrong in the below program, as the
output is empty.
import std.stdio, std.path;
void main ()
{
version (Windows)
{
auto Path = `C:\#Users\Admin\Desktop\Script\Test1`;
if(!Path.isValidPath) { writeln("Path Not Valid");}
}
}
From,
Vino.B
Hi,
The path is not valid as it contains the character # after the
drive name.
replace switch for mapping
Generally one has to use a switch to map dynamic components.
Given a set X and Y one can form a switch to map X to Y:
switch(X)
{
case x1 : y1; break;
case x1 : y1;
}
Is there any easier way to do this where one simply specifies the
set's rather than having to create a switch directly?
In my specific case, I have to map a two sets of types
A = {Ta1,...,Tan}
B = {Tb1,...,Tbm}
to a template function that takes two types
foo(Tak, Taj)
so, given an arbitrary (a,b) in AxB, it it maps to foo(F(a),G(b)).
Using switches would require n*m cases.
What I actually have is something like
enum X
{
Float,
Int,
`Etc`
}
and X x, y;
and need to call foo!(x,y) but with x and y replaced by their
correct D equivalent types.
e.g., if x = X.Float; y = X.Int; then I need to call
foo!(float,int) rather than foo!(X.Float,x.Int).
This allows me to create a dynamic dispatcher at runtime and use
a templated function rather than having to handle each type
independently. One templated function rather than nxm regular
functions for each type or a nxm switch.
Unfortunately, a complicating factor is that the enum's names do
not directly correspond to the D types through some simple
transformation(e.g., lowerCase). D doesn't seem to support
attributes on enum members for some inane reason and using
strings will complicate
things[https://forum.dlang.org/post/nmgloo$bd1$1...@digitalmars.com]. I think I could use a struct though to solve that.
So, given something like
struct A
{
@("float") enum Float = 0,
@("int") enum Int = 1,
}
struct B
{
@("double") enum Double = 0,
@("short") enum Short = 1,
}
foo(T1,T2)();
create a mapping that takes an A and B and maps AxB to foo that
does something like the following internally.
fooDispatch(A a, B b)
{
switch(a) // Actually needs to be over attributes
{
case "float" :
switch(b) // Actually needs to be over attributes
{
case "double" : return foo!(float, double)();
}
...
}
}
or whatever. I could write a string mixin that generates the
above code but I'm hoping I don't have to and some genius will
find a simple way to do it quickly, efficiently, and performant.
Re: Valid File Path
On Thursday, 31 August 2017 at 21:59:22 UTC, vino wrote:
Hi All,
Can you help me what is wrong in the below program, as the
output is empty.
import std.stdio, std.path;
void main ()
{
version (Windows)
{
auto Path = `C:\#Users\Admin\Desktop\Script\Test1`;
if(!Path.isValidPath) { writeln("Path Not Valid");}
}
}
From,
Vino.B
It doesn't print anything because the path is valid. Why do you
expect otherwise?
Note that isValidPath is only used for syntax check, it does not
checks that the file or directory exists. If that is what you
want use std.file.exists instead.
Valid File Path
Hi All,
Can you help me what is wrong in the below program, as the
output is empty.
import std.stdio, std.path;
void main ()
{
version (Windows)
{
auto Path = `C:\#Users\Admin\Desktop\Script\Test1`;
if(!Path.isValidPath) { writeln("Path Not Valid");}
}
}
From,
Vino.B
Re: Remove all blank lines from a file
On Thursday, 31 August 2017 at 15:48:31 UTC, Rene Zwanenburg
wrote:
On Thursday, 31 August 2017 at 14:44:07 UTC, vino wrote:
Hi All,
Can some provide me a example of how to remove all blank
lines from a file.
From,
Vino.B
This one doesn't read the entire file into memory:
import std.stdio;
import std.array;
import std.algorithm;
import std.uni;
void main(string[] args)
{
auto inputFile = File(args[1]);
auto outputFile = File(args[2], "wb");
inputFile
.byLine
.filter!(line => !line.all!isWhite)
.copy(outputFile.lockingTextWriter);
}
But if you want to replace the input file, you'd have to write
to a temp file, remove the original, then move the temp file.
Hi All,
Thank you very much, was able to resolve the issue.
From,
Vino.B
Re: std.algorithm.joiner unexpected behavior
On Thu, Aug 31, 2017 at 01:34:39PM -0600, Jonathan M Davis via Digitalmars-d-learn wrote: [...] > In general, byLine does not work with other range-based algorithms > precisely because it reuses the buffer. I think that it does manage to > work for some, but IMHO, it should have just supported foreach via > opApply and not been a range at all. It's great for efficiency in a > loop but horrible for range chaining. [...] Transient ranges are tricky to work with, I agree, but I don't agree that they're "horrible for range chaining". I argue that many range algorithms that assume the persistence of .front are inherently wrong, and ought to be implemented in a way that does *not* make this assumption. Many std.algorithm algorithms *can* in fact be written in this way, and those that aren't, are arguably buggy. For example, some std.algorithm functions take a forward range but then tries to save .front to a local variable. Rather, they should use .save to save the previous position of the range so that they can call .front on that to access the previous element, instead of making the unfounded assumption that whatever the local variable refers to will still remain valid after calling .popFront. It's just sloppy coding IMNSHO. T -- MS Windows: 64-bit rehash of 32-bit extensions and a graphical shell for a 16-bit patch to an 8-bit operating system originally coded for a 4-bit microprocessor, written by a 2-bit company that can't stand 1-bit of competition.
Re: writeln() sometimes double prints from main() if I run a thread checking for input?
On Thursday, 31 August 2017 at 14:43:39 UTC, Steven Schveighoffer wrote: Just a thought, but the "double printing" could be a misunderstanding. It could be printing Output\nOutput2, but not getting the 2 out there. No no, it's four lines instead of three. If we change the lines to disjoint sets of letters, the problem persists. Note that DMD 32-bit is using DMC libc. It might be that it gets hung up somehow when expecting input, like it locks the console somehow. I would say that byLineCopy puts the thread to sleep waiting for input, and it doesn't get out of that state. So it could be that the bug only appears when it gets to that state at some point in the output. I'd pepper some sleeps around the outputs to see if you can make the context switches more predictable. Inserting different sleeps into the threads makes the problem go away (cannot reproduce). Inserting identical sleeps produces the error with roughly the same probability. Anyway, I've reported it (https://issues.dlang.org/show_bug.cgi?id=17797), along with a more or less exact version (2.073.2) where the problem was introduced. Ivan Kazmenko.
Re: Deprecation of toUTF16
On 8/31/17 12:12 PM, Andre Pany wrote: Hi, I have some problems to find out what to use instead of the deprecated toUTF16 function. I am calling a shared library written in Delphi. While this coding is working fine (with german ü) import std.utf: toUTF16; wstring ws = toUTF16(s); BSTR bStr = SysAllocStringLen(ws.ptr, cast(UINT) ws.length); This coding fails to display the german ü correctly: import std.utf: encode; wchar[] ws; s.each!(c => encode(ws, c)); BSTR bStr = SysAllocStringLen(ws.ptr, cast(UINT) ws.length); Variable s is of type string. On delphi side WideString is used as type for the string. Where is the error? The error is actually in the compiler -- the toUTF16 function you are calling is NOT being deprecated. But the compiler mistakenly is marking it as deprecated. See here: https://issues.dlang.org/show_bug.cgi?id=17193 -Steve
Re: std.algorithm.joiner unexpected behavior
On Thursday, August 31, 2017 18:43:40 Jesse Phillips via Digitalmars-d-learn wrote: > On Thursday, 31 August 2017 at 18:26:33 UTC, Sergei Degtiarev > > wrote: > > Hi, > > I tried to create a simple range concatenating several files, > > > > something like this: > > File[] files; > > > > foreach(ln; joiner(files.map!(a => a.byLine))) > > > > writeln(ln); > > > > and I see every first line of each file is missing. > > > > However, when I do same thing with separator: > > char[][] separator=[" new file ".dup]; > > foreach(ln; joiner(files.map!(a => a.byLine), separator)) > > > > writeln(ln); > > > > everything works fine, both separator and first lines are > > printed. > > It looks pretty much as a bug for me, but I'm not sure I use it > > correctly. > > Doesn't byLine() reuse a buffer, joiner probably caches the first > line and calls .popFront() In general, byLine does not work with other range-based algorithms precisely because it reuses the buffer. I think that it does manage to work for some, but IMHO, it should have just supported foreach via opApply and not been a range at all. It's great for efficiency in a loop but horrible for range chaining. Folks get bit by this problem all the time. Fortunately, we now have byLineCopy which actually uses a separate dynamic array for each line, but even still, often, someone grabs byLine and then gets weird behavior that they don't understand. - Jonathan M Davis
Re: std.algorithm.joiner unexpected behavior
On Thu, Aug 31, 2017 at 06:26:33PM +, Sergei Degtiarev via Digitalmars-d-learn wrote: [...] > File[] files; > > foreach(ln; joiner(files.map!(a => a.byLine))) > writeln(ln); You probably want to use byLineCopy instead. The problem here is that .byLine returns a transient range (i.e., the value returned by .front changes after .popFront is called), which can cause unexpected behaviour when used with certain algorithms. T -- I don't trust computers, I've spent too long programming to think that they can get anything right. -- James Miller
Re: std.algorithm.joiner unexpected behavior
On Thursday, 31 August 2017 at 18:26:33 UTC, Sergei Degtiarev wrote: Hi, I tried to create a simple range concatenating several files, something like this: File[] files; foreach(ln; joiner(files.map!(a => a.byLine))) writeln(ln); and I see every first line of each file is missing. However, when I do same thing with separator: char[][] separator=[" new file ".dup]; foreach(ln; joiner(files.map!(a => a.byLine), separator)) writeln(ln); everything works fine, both separator and first lines are printed. It looks pretty much as a bug for me, but I'm not sure I use it correctly. Doesn't byLine() reuse a buffer, joiner probably caches the first line and calls .popFront()
std.algorithm.joiner unexpected behavior
Hi, I tried to create a simple range concatenating several files, something like this: File[] files; foreach(ln; joiner(files.map!(a => a.byLine))) writeln(ln); and I see every first line of each file is missing. However, when I do same thing with separator: char[][] separator=[" new file ".dup]; foreach(ln; joiner(files.map!(a => a.byLine), separator)) writeln(ln); everything works fine, both separator and first lines are printed. It looks pretty much as a bug for me, but I'm not sure I use it correctly.
Re: Deprecation of toUTF16
On Thursday, 31 August 2017 at 17:22:51 UTC, Kagamin wrote: import std.conv; auto ws=s.to!wstring; Thank you! Kind regards André
Re: Deprecation of toUTF16
import std.conv; auto ws=s.to!wstring;
Deprecation of toUTF16
Hi, I have some problems to find out what to use instead of the deprecated toUTF16 function. I am calling a shared library written in Delphi. While this coding is working fine (with german ü) import std.utf: toUTF16; wstring ws = toUTF16(s); BSTR bStr = SysAllocStringLen(ws.ptr, cast(UINT) ws.length); This coding fails to display the german ü correctly: import std.utf: encode; wchar[] ws; s.each!(c => encode(ws, c)); BSTR bStr = SysAllocStringLen(ws.ptr, cast(UINT) ws.length); Variable s is of type string. On delphi side WideString is used as type for the string. Where is the error? Kind regards André
Re: Parse tree node allocator
On Thursday, 31 August 2017 at 15:43:05 UTC, Per Nordlöw wrote: Which allocator is best suited for allocating tree nodes (all of equal size around 40-60 bytes in size) in one shot and then delete them all in one go? My use case is parse trees. Region Allocator.
Re: Remove all blank lines from a file
On Thursday, 31 August 2017 at 14:44:07 UTC, vino wrote:
Hi All,
Can some provide me a example of how to remove all blank
lines from a file.
From,
Vino.B
This one doesn't read the entire file into memory:
import std.stdio;
import std.array;
import std.algorithm;
import std.uni;
void main(string[] args)
{
auto inputFile = File(args[1]);
auto outputFile = File(args[2], "wb");
inputFile
.byLine
.filter!(line => !line.all!isWhite)
.copy(outputFile.lockingTextWriter);
}
But if you want to replace the input file, you'd have to write to
a temp file, remove the original, then move the temp file.
Re: Bug in D!!!
On Thursday, 31 August 2017 at 10:34:14 UTC, Kagamin wrote: On Thursday, 31 August 2017 at 00:49:22 UTC, EntangledQuanta wrote: I've already implemented a half ass library solution. It can be improved alot. Then, by all means, genius!
Parse tree node allocator
Which allocator is best suited for allocating tree nodes (all of equal size around 40-60 bytes in size) in one shot and then delete them all in one go? My use case is parse trees.
Re: Remove all blank lines from a file
On Thursday, 31 August 2017 at 14:44:07 UTC, vino wrote:
Hi All,
Can some provide me a example of how to remove all blank
lines from a file.
From,
Vino.B
Super verbose, but:
import std.stdio;
import std.file;
import std.algorithm.iteration;
enum inFilename = "in.txt";
enum outFilename = "out.txt";
void main()
{
immutable lines = readText(inFilename);
char[] outbuffer;
outbuffer.reserve(lines.length);
foreach (line; lines.splitter("\n"))
{
if (!line.length) continue;
outbuffer ~= line;
outbuffer ~= "\n";
}
auto outfile = File(outFilename, "w");
outfile.write(outbuffer);
}
Re: Remove all blank lines from a file
On Thursday, 31 August 2017 at 14:44:07 UTC, vino wrote:
Hi All,
Can some provide me a example of how to remove all blank
lines from a file.
From,
Vino.B
ubyte[] fileData;
ubyte[] writeThis;
uint lastP;
fileData = readRaw(fileName);
foreach(uint p; ubyte b;fileData)
{
if (b == '\n')
{
writeThis ~= fileData[lastP .. p];
lastP = p;
}
}
write(fileName, fileData);
Remove all blank lines from a file
Hi All, Can some provide me a example of how to remove all blank lines from a file. From, Vino.B
Re: writeln() sometimes double prints from main() if I run a thread checking for input?
On 8/30/17 9:33 AM, Ivan Kazmenko wrote:
On Wednesday, 30 August 2017 at 13:24:55 UTC, Ivan Kazmenko wrote:
On Wednesday, 30 August 2017 at 10:55:20 UTC, Timothy Foster wrote:
import std.stdio, core.thread;
void main(){
auto thread = new Thread(&func).start;
writeln("Output");
writeln("Output2");
writeln("Output3");
while(true){}
}
void func(){
foreach(line; stdin.byLineCopy){}
}
I also cannot reproduce this.
My current system is Win7 64-bit. I tried 32-bit dmd 2.072.0 and
2.075.1, with optimizations turned on and off, but it prints correctly
tens of times in each case.
Try running the program in a bare console (cmd.exe on Windows, or some
*sh on Linux). If the problem goes away, your usual environment is
likely at fault. If not,.. well, no idea for now.
Hey, I followed my own advice and do see the same issue! when:
1. compiling with dmd 2.075.1 (optimization switches seem to be
irrelevant but the issue does not reproduce with 2.072.0),
2. running in a bare cmd.exe, and
3. running the program as just "a.exe", so that it waits for console
input (previously I redirected some input to it, like "a.exe "a.exe Interesting. As to what to do with it, no idea for now. At the very
least we can issue a bug report, now that at least two people can
reproduce it, so it is unlikely to be environment-dependent.
Just a thought, but the "double printing" could be a misunderstanding.
It could be printing Output\nOutput2, but not getting the 2 out there.
Note that DMD 32-bit is using DMC libc. It might be that it gets hung up
somehow when expecting input, like it locks the console somehow.
I would say that byLineCopy puts the thread to sleep waiting for input,
and it doesn't get out of that state. So it could be that the bug only
appears when it gets to that state at some point in the output. I'd
pepper some sleeps around the outputs to see if you can make the context
switches more predictable.
-Steve
Re: writeln() sometimes double prints from main() if I run a thread checking for input?
On Wednesday, 30 August 2017 at 13:33:06 UTC, Ivan Kazmenko wrote: Interesting. As to what to do with it, no idea for now. At the very least we can issue a bug report, now that at least two people can reproduce it, so it is unlikely to be environment-dependent. Reported: https://issues.dlang.org/show_bug.cgi?id=17797 .
Re: Output range with custom string type
On Thursday, 31 August 2017 at 07:06:26 UTC, Jacob Carlborg wrote:
On 2017-08-29 19:35, Moritz Maxeiner wrote:
void put(T t)
{
if (!store)
{
// Allocate only once for "small" vectors
store = alloc.makeArray!T(8);
if (!store) onOutOfMemoryError();
}
else if (length == store.length)
{
// Growth factor of 1.5
auto expanded = alloc.expandArray!char(store,
store.length / 2);
if (!expanded) onOutOfMemoryError();
}
assert (length < store.length);
moveEmplace(t, store[length++]);
}
What's the reason to use "moveEmplace" instead of just
assigning to the array: "store[length++] = t" ?
The `move` part is to support non-copyable types (i.e. T with
`@disable this(this)`), such as another owning container
(assigning would generally try to create a copy).
The `emplace` part is because the destination `store[length]` has
been default initialized either by makeArray or expandArray and
it doesn't need to be destroyed (a pure move would destroy
`store[length]` if T has a destructor).
Re: Transitive const and function pointers/delegates
It's const(int delegate(char))
Re: Bug in D!!!
On Thursday, 31 August 2017 at 00:49:22 UTC, EntangledQuanta wrote: I've already implemented a half ass library solution. It can be improved alot.
Re: "Range invalidation" ?
On Wednesday, 30 August 2017 at 15:06:12 UTC, Jonathan M Davis wrote: [...] Thank you for the detailed response.
Re: hijack dub test
On Thursday, 31 August 2017 at 07:04:13 UTC, Jacob Carlborg wrote: On 2017-08-31 08:41, Nicholas Wilson wrote: My project is a library, but I also need to test it and unit tests won't cut it (external hardware). How do you set up the dub.json to build the library normally but when it is invoked with `dub test` it runs a separate configuration that also includes files in the `source/test` folder, but are excluded when not testing. "The configuration name "unittest" has a special meaning - if a configuration with this name is present, it will be used by default when executing dub test." [1] [1] http://code.dlang.org/package-format?lang=json#configurations Thanks!
Re: Output range with custom string type
On 2017-08-29 19:35, Moritz Maxeiner wrote:
void put(T t)
{
if (!store)
{
// Allocate only once for "small" vectors
store = alloc.makeArray!T(8);
if (!store) onOutOfMemoryError();
}
else if (length == store.length)
{
// Growth factor of 1.5
auto expanded = alloc.expandArray!char(store, store.length
/ 2);
if (!expanded) onOutOfMemoryError();
}
assert (length < store.length);
moveEmplace(t, store[length++]);
}
What's the reason to use "moveEmplace" instead of just assigning to the
array: "store[length++] = t" ?
--
/Jacob Carlborg
Re: hijack dub test
On 2017-08-31 08:41, Nicholas Wilson wrote: My project is a library, but I also need to test it and unit tests won't cut it (external hardware). How do you set up the dub.json to build the library normally but when it is invoked with `dub test` it runs a separate configuration that also includes files in the `source/test` folder, but are excluded when not testing. "The configuration name "unittest" has a special meaning - if a configuration with this name is present, it will be used by default when executing dub test." [1] [1] http://code.dlang.org/package-format?lang=json#configurations -- /Jacob Carlborg
