Re: [gentoo-user] [OT] - command line read *.csv & create new file

2009-02-25 Thread Etaoin Shrdlu
On Tuesday 24 February 2009, 23:51, Mark Knecht wrote: > Looks like I'm running into one more problem and then I'm ready to > give it a try for real. Unfortunately one vendor platform is putting > quotes around the names in the header row so your _N increment looks > like "High"_4 instead of High_

Re: [gentoo-user] [OT] - command line read *.csv & create new file

2009-02-24 Thread Mark Knecht
On Tue, Feb 24, 2009 at 9:48 AM, Etaoin Shrdlu wrote: > On Tuesday 24 February 2009, 15:41, Mark Knecht wrote: >> On Tue, Feb 24, 2009 at 2:56 AM, Etaoin Shrdlu >> wrote: >> >> > So, in my understanding this is what we want to accomplish so far: >> > >> > given an input of the form >> > >> > D1,

Re: [gentoo-user] [OT] - command line read *.csv & create new file

2009-02-24 Thread Etaoin Shrdlu
On Tuesday 24 February 2009, 15:41, Mark Knecht wrote: > On Tue, Feb 24, 2009 at 2:56 AM, Etaoin Shrdlu > wrote: > > > So, in my understanding this is what we want to accomplish so far: > > > > given an input of the form > > > > D1,T1,a1,b1,c1,d1,...,R1 > > D2,T2,a2,b2,c2,d2,...,R2 > > D3,T3,a3,b

Re: [gentoo-user] [OT] - command line read *.csv & create new file

2009-02-24 Thread Mark Knecht
On Tue, Feb 24, 2009 at 2:56 AM, Etaoin Shrdlu wrote: > > So, in my understanding this is what we want to accomplish so far: > > given an input of the form > > D1,T1,a1,b1,c1,d1,...,R1 > D2,T2,a2,b2,c2,d2,...,R2 > D3,T3,a3,b3,c3,d3,...,R3 > D4,T4,a4,b4,c4,d4,...,R4 > D5,T5,a5,b5,c5,d5,...,R5 > >

Re: [gentoo-user] [OT] - command line read *.csv & create new file

2009-02-24 Thread Etaoin Shrdlu
On Tuesday 24 February 2009, 03:26, Mark Knecht wrote: > If I drop columns - and I do need to - then something like how cut > works would be good, but it needs to repeat across all the rows being > used. For instance, if I'm dropping columns 6 & 12 from a 20 column > wide data set, then I'm droppi

Re: [gentoo-user] [OT] - command line read *.csv & create new file

2009-02-23 Thread Mark Knecht
On Mon, Feb 23, 2009 at 2:18 PM, Etaoin Shrdlu wrote: > On Monday 23 February 2009, 17:05, Mark Knecht wrote: > >> I'm attaching a small (100 line) data file out of TradeStation. Zipped >> it's about 2K. It should expand to about 10K. When I run the command >> to get 10 lines put together it works

Re: [gentoo-user] [OT] - command line read *.csv & create new file

2009-02-23 Thread Etaoin Shrdlu
On Monday 23 February 2009, 17:05, Mark Knecht wrote: > I'm attaching a small (100 line) data file out of TradeStation. Zipped > it's about 2K. It should expand to about 10K. When I run the command > to get 10 lines put together it works correctly and gives me a file > with 91 lines and about 100K

Re: [gentoo-user] [OT] - command line read *.csv & create new file

2009-02-23 Thread Mark Knecht
On Mon, Feb 23, 2009 at 1:57 AM, Etaoin Shrdlu wrote: > On Monday 23 February 2009, 00:31, Mark Knecht wrote: > >> Yeah, that's probably almost usable as it is . I tried it with n=3 and >> n=10. Worked both times just fine. The initial issue might be (as with >> Willie's sed code) that the first l

Re: [gentoo-user] [OT] - command line read *.csv & create new file

2009-02-23 Thread Etaoin Shrdlu
On Monday 23 February 2009, 00:31, Mark Knecht wrote: > Yeah, that's probably almost usable as it is . I tried it with n=3 and > n=10. Worked both times just fine. The initial issue might be (as with > Willie's sed code) that the first line wasn't quite right and required > some hand editing. I'd

Re: [gentoo-user] [OT] - command line read *.csv & create new file

2009-02-22 Thread Paul Hartman
On Sun, Feb 22, 2009 at 5:31 PM, Mark Knecht wrote: > The down side is the output file is 10x larger than the input file - > roughly - and my current input files are 40-60MB so the output files > will be 600MB. Not huge but if they grew too much more I might get > beyond what a single file can be

Re: [gentoo-user] [OT] - command line read *.csv & create new file

2009-02-22 Thread Mark Knecht
On Sun, Feb 22, 2009 at 4:57 PM, Willie Wong wrote: > On Sun, Feb 22, 2009 at 03:15:09PM -0800, Penguin Lover Mark Knecht squawked: >> 1) My actual input data starts with two fields which date & time. For >> lines 2 & 3 I need exclude the 2nd & 3rd date & time from the output >> corresponding to l

Re: [gentoo-user] [OT] - command line read *.csv & create new file

2009-02-22 Thread Willie Wong
On Sun, Feb 22, 2009 at 03:15:09PM -0800, Penguin Lover Mark Knecht squawked: > 1) My actual input data starts with two fields which date & time. For > lines 2 & 3 I need exclude the 2nd & 3rd date & time from the output > corresponding to line 1, so these 3 lines: > > Date1,Time1,A,B,C,D,0 > Date

Re: [gentoo-user] [OT] - command line read *.csv & create new file

2009-02-22 Thread Mark Knecht
On Sun, Feb 22, 2009 at 2:57 PM, Etaoin Shrdlu wrote: > On Sunday 22 February 2009, 23:28, Mark Knecht wrote: > >> > >> > >> > >> > >> >> The above diagram is correct when the lines chosen is 3. I suspect >> that I might chose 10 or 15 lines once I get real data and do some >> testing but

Re: [gentoo-user] [OT] - command line read *.csv & create new file

2009-02-22 Thread Mark Knecht
On Sun, Feb 22, 2009 at 12:59 PM, Willie Wong wrote: > On Sun, Feb 22, 2009 at 11:06:31AM -0800, Penguin Lover Mark Knecht squawked: >>I've got a really big data file in essentially a *.csv format. >> (comma delimited) I need to scan this file and create a new output >> file. I'm wondering if

Re: [gentoo-user] [OT] - command line read *.csv & create new file

2009-02-22 Thread Etaoin Shrdlu
On Sunday 22 February 2009, 23:28, Mark Knecht wrote: > > > > > > > > > > The above diagram is correct when the lines chosen is 3. I suspect > that I might chose 10 or 15 lines once I get real data and do some > testing but that was harder to show in this email. A good design for > me woul

Re: [gentoo-user] [OT] - command line read *.csv & create new file

2009-02-22 Thread Mark Knecht
On Sun, Feb 22, 2009 at 12:15 PM, Etaoin Shrdlu wrote: > On Sunday 22 February 2009, 20:06, Mark Knecht wrote: >> Hi, >>Very off topic other than I'd do this on my Gentoo box prior to >> using R on my Gentoo box. Please ignore if not of interest. >> >>I've got a really big data file in ess

Re: [gentoo-user] [OT] - command line read *.csv & create new file

2009-02-22 Thread Willie Wong
On Sun, Feb 22, 2009 at 11:06:31AM -0800, Penguin Lover Mark Knecht squawked: >I've got a really big data file in essentially a *.csv format. > (comma delimited) I need to scan this file and create a new output > file. I'm wondering if there is a reasonably easy command line way of > doing this

Re: [gentoo-user] [OT] - command line read *.csv & create new file

2009-02-22 Thread Etaoin Shrdlu
On Sunday 22 February 2009, 20:06, Mark Knecht wrote: > Hi, >Very off topic other than I'd do this on my Gentoo box prior to > using R on my Gentoo box. Please ignore if not of interest. > >I've got a really big data file in essentially a *.csv format. > (comma delimited) I need to scan thi

[gentoo-user] [OT] - command line read *.csv & create new file

2009-02-22 Thread Mark Knecht
Hi, Very off topic other than I'd do this on my Gentoo box prior to using R on my Gentoo box. Please ignore if not of interest. I've got a really big data file in essentially a *.csv format. (comma delimited) I need to scan this file and create a new output file. I'm wondering if there is a