On 07.01.18 14:44, Steven Schveighoffer wrote:
> Not from what I'm reading, the C solution is about the same (257 vs.
> 261). Not sure if you have averaged these numbers, especially on a real
> computer that might be doing other things.
yes you are right ... for proper benchmarking proper statistic
On 1/6/18 11:14 AM, Christian Köstlin wrote:
On 05.01.18 23:04, Steven Schveighoffer wrote:
One thing to try, you preallocate the ENTIRE buffer. This only works if
you know how many bytes it will decompress to (not always possible), but
it will take the allocator out of the equation completely.
On 05.01.18 23:04, Steven Schveighoffer wrote:
> On 1/5/18 3:09 PM, Christian Köstlin wrote:
>> On 05.01.18 15:39, Steven Schveighoffer wrote:
>>> Yeah, I guess most of the bottlenecks are inside libz, or the memory
>>> allocator. There isn't much optimization to be done in the main program
>>> its
On 1/5/18 3:09 PM, Christian Köstlin wrote:
On 05.01.18 15:39, Steven Schveighoffer wrote:
Yeah, I guess most of the bottlenecks are inside libz, or the memory
allocator. There isn't much optimization to be done in the main program
itself.
D compiles just the same as C. So theoretically you sho
On 05.01.18 15:39, Steven Schveighoffer wrote:
> Yeah, I guess most of the bottlenecks are inside libz, or the memory
> allocator. There isn't much optimization to be done in the main program
> itself.
>
> D compiles just the same as C. So theoretically you should be able to
> get the same performa
On 1/5/18 1:01 AM, Christian Köstlin wrote:
On 04.01.18 20:46, Steven Schveighoffer wrote:
On 1/4/18 1:57 PM, Christian Köstlin wrote:
Thanks Steve,
this runs now faster, I will update the table.
Still a bit irked that I can't match the C speed :/
But, I can't get your C speed to duplicate o
On 04.01.18 20:46, Steven Schveighoffer wrote:
> On 1/4/18 1:57 PM, Christian Köstlin wrote:
>> Thanks Steve,
>> this runs now faster, I will update the table.
>
> Still a bit irked that I can't match the C speed :/
>
> But, I can't get your C speed to duplicate on my mac even with gcc, so
> I'm
On 1/4/18 1:57 PM, Christian Köstlin wrote:
Thanks Steve,
this runs now faster, I will update the table.
Still a bit irked that I can't match the C speed :/
But, I can't get your C speed to duplicate on my mac even with gcc, so
I'm not sure where to start. I find it interesting that you are n
On 1/4/18 1:57 PM, Christian Köstlin wrote:
Sorry, but I do not get how to cleanup the mallocated memory :)
Could you help me out here?
free(mypipe.window.ptr);
At the moment this works because you haven't released any data from the
front. But eventually, I will need to figure out how to hand
On 04.01.18 16:53, Steven Schveighoffer wrote:
> On 1/3/18 3:28 PM, Steven Schveighoffer wrote:
>
>> Stay tuned, there will be updates to iopipe to hopefully make it as
>> fast in this microbenchmark as the C version :)
>
> v0.0.3 has been released. To take advantage of using malloc/realloc, you
On 1/3/18 3:28 PM, Steven Schveighoffer wrote:
Stay tuned, there will be updates to iopipe to hopefully make it as fast
in this microbenchmark as the C version :)
v0.0.3 has been released. To take advantage of using malloc/realloc, you
can use std.experimental.allocator.mallocator.Mallocator
On 1/4/18 4:47 AM, Christian Köstlin wrote:
I added now a c variant, that does always malloc/memcpy/free. Its much
slower for sure.
Also I put some output in thats shows when a real realloc happens. Its
like you said:
did a real realloc
did a real realloc
did a real realloc
did a real realloc
di
I added now a c variant, that does always malloc/memcpy/free. Its much
slower for sure.
Also I put some output in thats shows when a real realloc happens. Its
like you said:
did a real realloc
did a real realloc
did a real realloc
did a real realloc
did a real realloc
did a real realloc
did not a
On 03.01.18 22:33, Steven Schveighoffer wrote:
> On 1/3/18 3:28 PM, Steven Schveighoffer wrote:
>> 1. The major differentiator between the C and D algorithms is the use
>> of C realloc. This one thing saves the most time. I'm going to update
>> iopipe so you can use it (stand by). I will also be ex
On 1/3/18 3:28 PM, Steven Schveighoffer wrote:
1. The major differentiator between the C and D algorithms is the use of
C realloc. This one thing saves the most time. I'm going to update
iopipe so you can use it (stand by). I will also be examining how to
simulate using realloc when not using C
On 1/3/18 9:42 AM, Steven Schveighoffer wrote:
On 1/3/18 2:47 AM, Christian Köstlin wrote:
On 02.01.18 21:13, Steven Schveighoffer wrote:
Well, you don't need to use appender for that (and doing so is copying a
lot of the data an extra time). All you need is to extend the pipe until
there isn't
On 1/3/18 2:47 AM, Christian Köstlin wrote:
On 02.01.18 21:13, Steven Schveighoffer wrote:
Well, you don't need to use appender for that (and doing so is copying a
lot of the data an extra time). All you need is to extend the pipe until
there isn't any more new data, and it will all be in the bu
On 02.01.18 14:51, Adam D. Ruppe wrote:
> On Tuesday, 2 January 2018 at 10:27:11 UTC, Christian Köstlin wrote:
>> After this I analyzed the first step of the process (gunzipping the
>> data from a file to memory), and found out, that dlangs UnCompress is
>> much slower than java, and ruby and plain
On 02.01.18 21:13, Steven Schveighoffer wrote:
> Well, you don't need to use appender for that (and doing so is copying a
> lot of the data an extra time). All you need is to extend the pipe until
> there isn't any more new data, and it will all be in the buffer.
>
> // almost the same line from y
On 02.01.18 21:48, Steven Schveighoffer wrote:
> On 1/2/18 3:13 PM, Steven Schveighoffer wrote:
>> // almost the same line from your current version
>> auto mypipe = openDev("../out/nist/2011.json.gz")
>> .bufd.unzip(CompressionFormat.gzip);
>
> Would you mind telling me the sou
On 1/2/18 3:13 PM, Steven Schveighoffer wrote:
// almost the same line from your current version
auto mypipe = openDev("../out/nist/2011.json.gz")
.bufd.unzip(CompressionFormat.gzip);
Would you mind telling me the source of the data? When I do get around
to it, I want to hav
On 1/2/18 1:01 PM, Christian Köstlin wrote:
On 02.01.18 15:09, Steven Schveighoffer wrote:
On 1/2/18 8:57 AM, Adam D. Ruppe wrote:
On Tuesday, 2 January 2018 at 11:22:06 UTC, Stefan Koch wrote:
You can make it much faster by using a sliced static array as buffer.
Only if you want data corrup
On 02.01.18 15:09, Steven Schveighoffer wrote:
> On 1/2/18 8:57 AM, Adam D. Ruppe wrote:
>> On Tuesday, 2 January 2018 at 11:22:06 UTC, Stefan Koch wrote:
>>> You can make it much faster by using a sliced static array as buffer.
>>
>> Only if you want data corruption! It keeps a copy of your pointe
On 1/2/18 8:57 AM, Adam D. Ruppe wrote:
On Tuesday, 2 January 2018 at 11:22:06 UTC, Stefan Koch wrote:
You can make it much faster by using a sliced static array as buffer.
Only if you want data corruption! It keeps a copy of your pointer
internally: https://github.com/dlang/phobos/blob/maste
On Tuesday, 2 January 2018 at 11:22:06 UTC, Stefan Koch wrote:
You can make it much faster by using a sliced static array as
buffer.
Only if you want data corruption! It keeps a copy of your pointer
internally:
https://github.com/dlang/phobos/blob/master/std/zlib.d#L605
It also will always
On Tuesday, 2 January 2018 at 10:27:11 UTC, Christian Köstlin
wrote:
After this I analyzed the first step of the process (gunzipping
the data from a file to memory), and found out, that dlangs
UnCompress is much slower than java, and ruby and plain c.
Yeah, std.zlib is VERY poorly written. You
On Tuesday, 2 January 2018 at 10:27:11 UTC, Christian Köstlin
wrote:
Hi all,
over the holidays, I played around with processing some gzipped
json data. First version was implemented in ruby, but took too
long, so I tried, dlang. This was already faster, but not
really satisfactory fast. Then
Hi all,
over the holidays, I played around with processing some gzipped json
data. First version was implemented in ruby, but took too long, so I
tried, dlang. This was already faster, but not really satisfactory fast.
Then I wrote another version in java, which was much faster.
After this I anal
28 matches
Mail list logo
