zlib performance

2015-08-07 Thread yawniek via Digitalmars-d-learn
hi, unpacking files is kinda slow, probably i'm doing something wrong. below code is about half the speed of gnu zcat on my os x machine. why? why do i need to .dup the buffer? can i get rid of the casts? the chunk size has only a marginal influence. https://github.com/yannick/zcatd import

Re: zlib performance

2015-08-07 Thread yawniek via Digitalmars-d-learn
On Friday, 7 August 2015 at 08:50:11 UTC, Daniel Kozák wrote: ldc[2] -O -release -boundscheck=off -singleobj app.d ldc 0.15.2 beta2 2.86s user 0.55s system 77% cpu 4.392 total v2.068-devel-8f81ffc 2.86s user 0.67s system 78% cpu 4.476 total v2.067 2.88s user 0.67s system 78% cpu 4.529

Re: zlib performance

2015-08-07 Thread Daniel Kozák via Digitalmars-d-learn
On Fri, 07 Aug 2015 08:42:45 + yawniek dl...@srtnwz.com wrote: On Friday, 7 August 2015 at 08:24:11 UTC, Daniel Kozák wrote: can you try it with ldc? ldc[2] -O -release -boundscheck=off -singleobj app.d ldc 0.15.2 beta2 2.86s user 0.55s system 77% cpu 4.392 total

Re: zlib performance

2015-08-07 Thread yawniek via Digitalmars-d-learn
On Friday, 7 August 2015 at 07:29:15 UTC, Daniel Kozák wrote: Which compiler and version. There has been some performance problem with IO on OSX, it should be fixed in 2.068 release i'm on master. v2.068-devel-8f81ffc also changed file read mode to rb. i don't understand why the program

Re: zlib performance

2015-08-07 Thread Daniel Kozák via Digitalmars-d-learn
On Fri, 07 Aug 2015 08:13:01 + yawniek dl...@srtnwz.com wrote: On Friday, 7 August 2015 at 08:05:01 UTC, Daniel Kozák wrote: import std.zlib, std.file, std.stdio, std.conv; void main(string[] args) { auto f = File(args[1], rb); auto uncompressor = new

Re: zlib performance

2015-08-07 Thread Daniel Kozák via Digitalmars-d-learn
On Fri, 07 Aug 2015 08:13:01 + yawniek dl...@srtnwz.com wrote: On Friday, 7 August 2015 at 08:05:01 UTC, Daniel Kozák wrote: import std.zlib, std.file, std.stdio, std.conv; void main(string[] args) { auto f = File(args[1], rb); auto uncompressor = new

Re: zlib performance

2015-08-07 Thread Daniel Kozák via Digitalmars-d-learn
On Fri, 07 Aug 2015 07:19:43 + yawniek via Digitalmars-d-learn digitalmars-d-learn@puremagic.com wrote: hi, unpacking files is kinda slow, probably i'm doing something wrong. below code is about half the speed of gnu zcat on my os x machine. why? why do i need to .dup the buffer?

Re: zlib performance

2015-08-07 Thread Daniel Kozák via Digitalmars-d-learn
On Fri, 07 Aug 2015 07:36:39 + yawniek dl...@srtnwz.com wrote: On Friday, 7 August 2015 at 07:29:15 UTC, Daniel Kozák wrote: Which compiler and version. There has been some performance problem with IO on OSX, it should be fixed in 2.068 release i'm on master. v2.068-devel-8f81ffc

Re: zlib performance

2015-08-07 Thread Daniel Kozák via Digitalmars-d-learn
On Fri, 07 Aug 2015 07:19:43 + yawniek dl...@srtnwz.com wrote: hi, unpacking files is kinda slow, probably i'm doing something wrong. below code is about half the speed of gnu zcat on my os x machine. why? why do i need to .dup the buffer? It depends. In your case you don't need

Re: zlib performance

2015-08-07 Thread yawniek via Digitalmars-d-learn
On Friday, 7 August 2015 at 07:43:25 UTC, Daniel Kozák wrote: i don't understand why the program crashes when i do not do the .dup This is weird. I would say it should not crash exactely. but try it yourself. the fastest version i could come up so far is below. std.conv slows it down. going

Re: zlib performance

2015-08-07 Thread Daniel Kozák via Digitalmars-d-learn
On Fri, 07 Aug 2015 08:01:27 + yawniek dl...@srtnwz.com wrote: On Friday, 7 August 2015 at 07:48:25 UTC, yawniek wrote: On Friday, 7 August 2015 at 07:43:25 UTC, Daniel Kozák wrote: the fastest version i could come up so far is below. std.conv slows it down. going from a 4kb to a

Re: zlib performance

2015-08-07 Thread yawniek via Digitalmars-d-learn
On Friday, 7 August 2015 at 07:48:25 UTC, yawniek wrote: On Friday, 7 August 2015 at 07:43:25 UTC, Daniel Kozák wrote: the fastest version i could come up so far is below. std.conv slows it down. going from a 4kb to a 4mb buffer helped. now i'm within 30% of gzcat's performance. ok maybe not,

Re: zlib performance

2015-08-07 Thread Daniel Kozák via Digitalmars-d-learn
On Fri, 7 Aug 2015 09:43:25 +0200 Daniel Kozák ko...@dlang.cz wrote: On Fri, 07 Aug 2015 07:36:39 + yawniek dl...@srtnwz.com wrote: On Friday, 7 August 2015 at 07:29:15 UTC, Daniel Kozák wrote: Which compiler and version. There has been some performance problem with IO on OSX,

Re: zlib performance

2015-08-07 Thread yawniek via Digitalmars-d-learn
On Friday, 7 August 2015 at 08:05:01 UTC, Daniel Kozák wrote: import std.zlib, std.file, std.stdio, std.conv; void main(string[] args) { auto f = File(args[1], rb); auto uncompressor = new UnCompress(HeaderFormat.gzip); foreach (buffer; f.byChunk(4096)) { auto

Re: zlib performance

2015-08-07 Thread yawniek via Digitalmars-d-learn
On Friday, 7 August 2015 at 08:24:11 UTC, Daniel Kozák wrote: can you try it with ldc? ldc[2] -O -release -boundscheck=off -singleobj app.d ldc 0.15.2 beta2 2.86s user 0.55s system 77% cpu 4.392 total v2.068-devel-8f81ffc 2.86s user 0.67s system 78% cpu 4.476 total v2.067 2.88s user

Re: zlib performance

2015-08-07 Thread Daniel Kozak via Digitalmars-d-learn
On Friday, 7 August 2015 at 09:12:32 UTC, yawniek wrote: On Friday, 7 August 2015 at 08:50:11 UTC, Daniel Kozák wrote: ldc[2] -O -release -boundscheck=off -singleobj app.d ldc 0.15.2 beta2 2.86s user 0.55s system 77% cpu 4.392 total v2.068-devel-8f81ffc 2.86s user 0.67s system 78% cpu

Re: zlib performance

2015-08-07 Thread yawniek via Digitalmars-d-learn
On Friday, 7 August 2015 at 11:45:00 UTC, Daniel Kozak wrote: On Friday, 7 August 2015 at 09:12:32 UTC, yawniek wrote: [...] Can you try it without write operation (comment out all write)? And than try it without uncompression? // without compression: void main(string[] args) { auto f

Re: zlib performance

2015-08-07 Thread Daniel Kozák via Digitalmars-d-learn
On Fri, 07 Aug 2015 12:29:26 + yawniek dl...@srtnwz.com wrote: On Friday, 7 August 2015 at 11:45:00 UTC, Daniel Kozak wrote: On Friday, 7 August 2015 at 09:12:32 UTC, yawniek wrote: [...] Can you try it without write operation (comment out all write)? And than try it without