Re: Regarding isSorted

2013-02-06 Thread bearophile
Brad Anderson: Just hit Edit at the top of that link you gave, make the change, and click Proposal File Change and it'll roll up a pull request. Should take less than a minute. Plus naturally 15-30 minutes to write down the missing unittests, that weren't written, so they didn't catch a so

Re: How to read fastly files ( I/O operation)

2013-02-06 Thread monarch_dodra
On Wednesday, 6 February 2013 at 10:43:02 UTC, bioinfornatics wrote: instead to call mmFile opIndex to read ubyte by ubyte i tried to put into a buffer array of length PAGESIZE. code here: http://dpaste.dzfl.pl/25ee34fc and is not faster for 12Go to parse i need 11 minutes. I do not see how

Re: new T[size] vs .reserve - alloca

2013-02-06 Thread Nick Treleaven
On 05/02/2013 22:15, monarch_dodra wrote: On Tuesday, 5 February 2013 at 21:14:32 UTC, Nick Treleaven wrote: On 05/02/2013 21:13, Nick Treleaven wrote: I've just tried it with dmd 2.059 (haven't upgraded yet) sorry, 2.060 Right, it's alias being finicky, because args.length isn't an actual

Re: new T[size] vs .reserve - alloca

2013-02-06 Thread Nick Treleaven
On 05/02/2013 16:39, bearophile wrote: Nick Treleaven: ^ I know you're aware of this, but maybe others might not know the default-argument alloca wrapping trick: For some usages it's an improvement over raw usage of alloca. I did see this in past, but sometimes I forget. Sorry if I sounded

Re: How to read fastly files ( I/O operation)

2013-02-06 Thread monarch_dodra
On Wednesday, 6 February 2013 at 11:15:22 UTC, monarch_dodra wrote: I'm going to try and see with some example files if I can't get something running faster. Benchmarking and tweaking, I was able to find 3 things that speeds up your program: 1) Make the computeLocal a compile time constant.

Re: std.process.system and friends

2013-02-06 Thread HeiHon
On Sunday, 3 February 2013 at 19:05:01 UTC, Peter Sommerfeld wrote: Do you know a way to set it via system/shell calls, notably under windows? Peter You might want to use Tango: module env; // 2013-01-08 // dmd2.061 + Siegelord Tango-D2-d2port import tango.sys.Environment; import

Re: std.process.system and friends

2013-02-06 Thread Peter Sommerfeld
HeiHon heiko.honr...@web.de schrieb: On Sunday, 3 February 2013 at 19:05:01 UTC, Peter Sommerfeld wrote: Do you know a way to set it via system/shell calls, notably under windows? Peter You might want to use Tango: module env; // 2013-01-08 // dmd2.061 + Siegelord Tango-D2-d2port

Re: How to read fastly files ( I/O operation)

2013-02-06 Thread bioinfornatics
i use both gdc / ldc with -w -O -release flags writeln inside loop is never evaluated as computeLocal boolean is always false Thanks in any case i continue to read all your answer :-)

Re: std.process.system and friends

2013-02-06 Thread HeiHon
Hmmm, AFAIK it is outdated, isn't it ? I also hesitate to introduce major dependencies for this small point. It was originally for D1, but SiegeLord ported almost all of it to D2. Can you point me to the sources of this Tango version please. May be I can reuse a small part of it.

Re: std.process.system and friends

2013-02-06 Thread Peter Sommerfeld
Am 06.02.2013, 14:51 Uhr, schrieb HeiHon heiko.honr...@web.de: https://github.com/SiegeLord/Tango-D2 There are still some very useful things in Tango that you don't find in Phobos (e.g. logging) and it plays nicely together with Phobos. Thanks, interesting read. Will have a deeper look in

Re: How to read fastly files ( I/O operation)

2013-02-06 Thread bioinfornatics
On Wednesday, 6 February 2013 at 13:20:58 UTC, bioinfornatics wrote: i use both gdc / ldc with -w -O -release flags writeln inside loop is never evaluated as computeLocal boolean is always false Thanks in any case i continue to read all your answer :-) just to add more information about

Re: How to read fastly files ( I/O operation)

2013-02-06 Thread monarch_dodra
On Wednesday, 6 February 2013 at 15:40:39 UTC, bioinfornatics wrote: It seem in any case is not easy to parse fastly a file in D I don't think that's true. D provides the same FILE primitive you'd get in C, so there is no reason for it to be slower than C. It is the range approach that, as

Re: How to read fastly files ( I/O operation)

2013-02-06 Thread monarch_dodra
On Wednesday, 6 February 2013 at 16:06:20 UTC, monarch_dodra wrote: It correctly takes into account that a sequence can be multiple lines. It does not strip whitespace because according to http://maq.sourceforge.net/fastq.shtml whitespace is not a legal character. Hum, just read your example

Re: How to read fastly files ( I/O operation)

2013-02-06 Thread FG
On 2013-02-04 15:04, bioinfornatics wrote: I am looking to parse efficiently huge file but i think D lacking for this purpose. To parse 12 Go i need 11 minutes wheras fastxtoolkit (written in c++ ) need 2 min. Haven't compared to fastxtoolkit, but I have some code for you. I have processed

Re: How to read fastly files ( I/O operation)

2013-02-06 Thread monarch_dodra
On Wednesday, 6 February 2013 at 19:19:52 UTC, FG wrote: On 2013-02-04 15:04, bioinfornatics wrote: I am looking to parse efficiently huge file but i think D lacking for this purpose. To parse 12 Go i need 11 minutes wheras fastxtoolkit (written in c++ ) need 2 min. Haven't compared to

Re: How to read fastly files ( I/O operation)

2013-02-06 Thread bioinfornatics
Thanks monarch and FG, i will read your code to see where i failing :-) And of course if you are interested with bio format i will really happy to works / review together In any case big thanks that is a very interesting subject

Re: How to read fastly files ( I/O operation)

2013-02-06 Thread FG
On 2013-02-06 21:43, monarch_dodra wrote: On Wednesday, 6 February 2013 at 19:19:52 UTC, FG wrote: I have processed the file SRR077487_1.filt.fastq from ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/data/HG00096/sequence_read/ and expect this syntax (no multiline sequences or whitespace). File takes

Re: How to read fastly files ( I/O operation)

2013-02-06 Thread Ali Çehreli
On 02/06/2013 12:43 PM, monarch_dodra wrote: with dmd, with -release -O -inline Going off topic a little, in a recent experiment, I have noticed that adding -inline made a range solution twice slower. -O -release still helped but -inline was the culprit. Ali

Re: How to read fastly files ( I/O operation)

2013-02-06 Thread Lee Braiden
On 06/02/13 22:21, bioinfornatics wrote: Thanks monarch and FG, i will read your code to see where i failing :-) I wasn't going to mention this as I thought the CPU usage might be trivial, but if both CPU and IO are factors, then it would probably be beneficial to have a separate IO

Re: How to read fastly files ( I/O operation)

2013-02-06 Thread FG
On 2013-02-07 00:41, Lee Braiden wrote: I wasn't going to mention this as I thought the CPU usage might be trivial, but if both CPU and IO are factors, then it would probably be beneficial to have a separate IO thread/task. This wasn't an issue in my version of the program. It took 1m55s to

Re: How to read fastly files ( I/O operation)

2013-02-06 Thread monarch_dodra
On Wednesday, 6 February 2013 at 22:55:14 UTC, FG wrote: On 2013-02-06 21:43, monarch_dodra wrote: On Wednesday, 6 February 2013 at 19:19:52 UTC, FG wrote: I have processed the file SRR077487_1.filt.fastq from ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/data/HG00096/sequence_read/ and expect this