Stream-Based Processing of Range Chunks in D

2013-12-10 Thread Nordlöw
I'm looking for an elegant way to perform chunk-stream-based processing of arrays/ranges. I'm building a file indexing/search engine in D that calculates various kinds of statistics on files such as histograms and SHA1-digests. I want these calculations to be performed in a single pass with

Re: Stream-Based Processing of Range Chunks in D

2013-12-10 Thread qznc
On Tuesday, 10 December 2013 at 09:57:44 UTC, Nordlöw wrote: I'm looking for an elegant way to perform chunk-stream-based processing of arrays/ranges. I'm building a file indexing/search engine in D that calculates various kinds of statistics on files such as histograms and SHA1-digests. I

Re: Stream-Based Processing of Range Chunks in D

2013-12-10 Thread Philippe Sigaud
On Tue, Dec 10, 2013 at 10:57 AM, Nordlöw per.nord...@gmail.com wrote: I'm looking for an elegant way to perform chunk-stream-based processing of arrays/ranges. I'm building a file indexing/search engine in D that calculates various kinds of statistics on files such as histograms and

Re: range chunks

2010-08-18 Thread Rory Mcguire
Peter Alexander wrote: On 7/08/10 3:06 PM, bearophile wrote: Peter Alexander: (I just Googled for quicksort pseudocode and sure enough, all of the top 10 entries that actually contained code defined a partition function that does what std.algorithm.partition does). Yes, it's named

Re: range chunks

2010-08-07 Thread Peter Alexander
On 7/08/10 2:51 AM, Andrei Alexandrescu wrote: On 08/06/2010 06:52 PM, bearophile wrote: Philippe Sigaud: Yeah, partition, chunk, segment, it a basic thing, worth including in std.range. The most common name is partition(). When I see partition I think it's quicksort's partition

Re: range chunks

2010-08-07 Thread KennyTM~
On Aug 7, 10 07:52, bearophile wrote: Philippe Sigaud: It's better to give it a default chunk size of 2. Why? I'm using partition() for years in various languages, and I've seen that the chunks of size 2 are the most common. And what should the default step be, according to you? If

Re: range chunks

2010-08-07 Thread bearophile
KennyTM~: I've never seen it called Partition[] besides Mathematica. As alternative do you like chunker? Bye, bearophile

Re: range chunks

2010-08-07 Thread KennyTM~
On Aug 7, 10 20:11, bearophile wrote: KennyTM~: I've never seen it called Partition[] besides Mathematica. As alternative do you like chunker? Bye, bearophile I prefer 'chunks', but that conflicts with std.stdio.chunks. It can be resolved by generalizing std.stdio.chunks to

Re: range chunks

2010-08-07 Thread Peter Alexander
On 7/08/10 7:57 AM, bearophile wrote: Peter Alexander: Yeah, partition has always meant that to me as well. Probably you come from a quite different part of computer science (The example I have shown from Mathematica is not the only one). What are the usages of that algorithm, beside being

Re: range chunks

2010-08-07 Thread bearophile
Peter Alexander: It's used a fair amount in geometry. I know the Kirkpatrick-Seidel algorithm uses it and I've seen some Delaunay triangulation implementations use it. I'm pretty sure there's other geometrical applications for it as well. Thank you for your explanations. (I just Googled

Re: range chunks

2010-08-07 Thread Andrei Alexandrescu
On 08/07/2010 01:57 AM, bearophile wrote: Peter Alexander: Yeah, partition has always meant that to me as well. Probably you come from a quite different part of computer science (The example I have shown from Mathematica is not the only one). What are the usages of that algorithm, beside

Re: range chunks

2010-08-06 Thread Philippe Sigaud
2010/8/6 Adrian Matoga e...@atari8.info Hi, Is there any off the shelf solution for iterating over a range by chunks? None that I know of. (should substitute [1, 2, 3], [4, 5, 6], [7, 8, 9], [10] for chunk in subsequent iterations) As a data point, why do you think it should produce [10]

Re: range chunks

2010-08-06 Thread bearophile
Philippe Sigaud: Here is what I cooked, it's still a bit rough around the edges. It has an optional step argument, to see how many elements to jump. It's better to give it a default chunk size of 2. If I have understood this well, then this is the partition() function:

Re: range chunks

2010-08-06 Thread Steven Schveighoffer
On Fri, 06 Aug 2010 13:33:09 -0400, Philippe Sigaud philippe.sig...@gmail.com wrote: Here is what I cooked, it's still a bit rough around the edges. It has an optional step argument, to see how many elements to jump. [snip] ElementType!R[] front() @property { return array(take(range,

Re: range chunks

2010-08-06 Thread Philippe Sigaud
On Fri, Aug 6, 2010 at 19:48, Steven Schveighoffer schvei...@yahoo.comwrote: On Fri, 06 Aug 2010 13:33:09 -0400, Philippe Sigaud philippe.sig...@gmail.com wrote: Here is what I cooked, it's still a bit rough around the edges. It has an optional step argument, to see how many elements to

Re: range chunks

2010-08-06 Thread Philippe Sigaud
On Fri, Aug 6, 2010 at 19:41, bearophile bearophileh...@lycos.com wrote: Philippe Sigaud: Here is what I cooked, it's still a bit rough around the edges. It has an optional step argument, to see how many elements to jump. It's better to give it a default chunk size of 2. Why? And what

Re: range chunks

2010-08-06 Thread Steven Schveighoffer
On Fri, 06 Aug 2010 14:59:17 -0400, Philippe Sigaud philippe.sig...@gmail.com wrote: On Fri, Aug 6, 2010 at 19:48, Steven Schveighoffer schvei...@yahoo.comwrote: On Fri, 06 Aug 2010 13:33:09 -0400, Philippe Sigaud philippe.sig...@gmail.com wrote: Here is what I cooked, it's still a bit

Re: range chunks

2010-08-06 Thread Andrei Alexandrescu
Steven Schveighoffer wrote: On Fri, 06 Aug 2010 14:59:17 -0400, Philippe Sigaud philippe.sig...@gmail.com wrote: On Fri, Aug 6, 2010 at 19:48, Steven Schveighoffer schvei...@yahoo.comwrote: On Fri, 06 Aug 2010 13:33:09 -0400, Philippe Sigaud philippe.sig...@gmail.com wrote: Here is what

Re: range chunks

2010-08-06 Thread Philippe Sigaud
On Fri, Aug 6, 2010 at 22:11, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: Steven Schveighoffer wrote: Doesn't take return a random-access range if the original is a random access range? I would actually expect take(range, n) to return range[0..n] if range supports that. It

Re: range chunks

2010-08-06 Thread Adrian Matoga
On 2010-08-06 19:33, Philippe Sigaud wrote: 2010/8/6 Adrian Matoga e...@atari8.info mailto:e...@atari8.info Hi, Is there any off the shelf solution for iterating over a range by chunks? None that I know of. (should substitute [1, 2, 3], [4, 5, 6], [7, 8, 9], [10] for chunk

Re: range chunks

2010-08-06 Thread bearophile
Philippe Sigaud: It's better to give it a default chunk size of 2. Why? I'm using partition() for years in various languages, and I've seen that the chunks of size 2 are the most common. And what should the default step be, according to you? If chose n (the chunk size), because that's

Re: range chunks

2010-08-06 Thread Andrei Alexandrescu
On 08/06/2010 06:52 PM, bearophile wrote: Philippe Sigaud: It's better to give it a default chunk size of 2. Why? I'm using partition() for years in various languages, and I've seen that the chunks of size 2 are the most common. And what should the default step be, according to you? If

range chunks

2010-08-05 Thread Adrian Matoga
Hi, Is there any off the shelf solution for iterating over a range by chunks? Example: int[] arr = [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ]; int i; foreach (chunk; chunks(arr, 3)) { assert(chunk == arr[i * 3 .. min(i * 3 + 3, arr.length)]); } (should substitute [1, 2, 3], [4, 5, 6], [7, 8,