Re: [Boston.pm] transposing rows and columns in a CSV file

2004-11-15 Thread Ben Tilly
On Sat, 13 Nov 2004 17:43:37 -0500, Uri Guttman [EMAIL PROTECTED] wrote: BT == Ben Tilly [EMAIL PROTECTED] writes: BT How was I confusing issues? What I meant is that calling mmap does BT not use significant amounts of RAM. (The OS needs some to track BT that the mapping exists, but

Re: [Boston.pm] transposing rows and columns in a CSV file

2004-11-15 Thread Dan Sugalski
At 12:12 PM -0800 11/15/04, Ben Tilly wrote: On Sat, 13 Nov 2004 17:43:37 -0500, Uri Guttman [EMAIL PROTECTED] wrote: BT == Ben Tilly [EMAIL PROTECTED] writes: BT In Perl I'd expect it to be possible but fragile. If Parrot could make BT it possible and not fragile, that would be great.

Re: [Boston.pm] transposing rows and columns in a CSV file

2004-11-15 Thread Jason Gloudon
BT I've also heard about Intel's large addressing extensions (keep 2GB BT in normal address space, page around the top 2 GB, you get 64 GB The extension referred to above concerns the 4 gig limit inherent in a 32 bit address space with byte addressable memory. The 4 gig limit is on the

Re: [Boston.pm] transposing rows and columns in a CSV file

2004-11-15 Thread Ben Tilly
On Mon, 15 Nov 2004 15:58:11 -0500, Aaron Sherman [EMAIL PROTECTED] wrote: On Sat, 13 Nov 2004 11:40:25 -0800, Ben Tilly [EMAIL PROTECTED] wrote: On Fri, 12 Nov 2004 23:04:46 -0500, Aaron Sherman [EMAIL PROTECTED] wrote: On Fri, 2004-11-12 at 13:22 -0800, Ben Tilly wrote: [...] Um,

Re: [Boston.pm] transposing rows and columns in a CSV file

2004-11-15 Thread Dan Sugalski
On Mon, 15 Nov 2004, Ben Tilly wrote: On Mon, 15 Nov 2004 15:58:11 -0500, Aaron Sherman [EMAIL PROTECTED] wrote: On Sat, 13 Nov 2004 11:40:25 -0800, Ben Tilly [EMAIL PROTECTED] wrote: On Fri, 12 Nov 2004 23:04:46 -0500, Aaron Sherman [EMAIL PROTECTED] wrote: On Fri, 2004-11-12 at

Re: [Boston.pm] transposing rows and columns in a CSV file

2004-11-15 Thread Ben Tilly
On Mon, 15 Nov 2004 18:46:15 -0500 (EST), Dan Sugalski [EMAIL PROTECTED] wrote: On Mon, 15 Nov 2004, Ben Tilly wrote: On Mon, 15 Nov 2004 15:58:11 -0500, Aaron Sherman [EMAIL PROTECTED] wrote: On Sat, 13 Nov 2004 11:40:25 -0800, Ben Tilly [EMAIL PROTECTED] wrote: On Fri, 12 Nov 2004

Re: [Boston.pm] transposing rows and columns in a CSV file

2004-11-13 Thread Ben Tilly
On Fri, 12 Nov 2004 23:04:46 -0500, Aaron Sherman [EMAIL PROTECTED] wrote: On Fri, 2004-11-12 at 13:22 -0800, Ben Tilly wrote: [...] Um, mmap does not (well should not - Windows may vary) use any RAM You are confusing two issues. using RAM is not the same as allocating process address

Re: [Boston.pm] transposing rows and columns in a CSV file

2004-11-13 Thread Uri Guttman
BT == Ben Tilly [EMAIL PROTECTED] writes: BT How was I confusing issues? What I meant is that calling mmap does BT not use significant amounts of RAM. (The OS needs some to track BT that the mapping exists, but that should be it.) Once you actually use BT the data that you mmapped in,

Re: [Boston.pm] transposing rows and columns in a CSV file

2004-11-12 Thread Ben Tilly
On Fri, 12 Nov 2004 07:38:57 -0500, Gyepi SAM [EMAIL PROTECTED] wrote: On Fri, Nov 12, 2004 at 02:11:37AM -0500, Aaron Sherman wrote: [...] I think mmap would be just as ideal in Perl and a lot less work too. Rather than indexing and parsing a *large* file, you must mmap and parse it. In fact,

Re: [Boston.pm] transposing rows and columns in a CSV file

2004-11-12 Thread Ben Tilly
On Fri, 12 Nov 2004 10:05:27 -0500, Uri Guttman [EMAIL PROTECTED] wrote: GS == Gyepi SAM [EMAIL PROTECTED] writes: [...] this talk about mmap makes little sense to me. it may save some i/o and even some buffering but you still need the ram and mmap still causes disk accesses. Um, mmap does

Re: [Boston.pm] transposing rows and columns in a CSV file

2004-11-12 Thread Aaron Sherman
On Fri, 2004-11-12 at 13:22 -0800, Ben Tilly wrote: On Fri, 12 Nov 2004 10:05:27 -0500, Uri Guttman [EMAIL PROTECTED] wrote: GS == Gyepi SAM [EMAIL PROTECTED] writes: [...] this talk about mmap makes little sense to me. it may save some i/o and even some buffering but you still need the

Re: [Boston.pm] transposing rows and columns in a CSV file

2004-11-11 Thread Aaron Sherman
On Wed, 2004-11-10 at 23:13 -0500, Uri Guttman wrote: sorry i missed the meeting but i have a nasty cold i am fighting off. I just got over that one :-( As for transposing a matrix that won't fit in ram... that's easy. Mail it to someone who has more ram. Aaron Gordian Knot Sherman, at your

Re: [Boston.pm] transposing rows and columns in a CSV file

2004-11-10 Thread Uri Guttman
WR == William Ricker [EMAIL PROTECTED] writes: WR This problem of transposing a matrix bigger than you want to WR consider slurping feels familiar. The techniques used to sort WR big files in the bad old days, disk or tape based sort-merge, WR might be adaptble. The trick is to do it in