Re: [PLUG] reply to suggestions on searching a big file > 1 gig

2012-09-30 Thread Marvin Kosmal
On Sun, Sep 30, 2012 at 9:05 PM, website reader wrote: > Thanks for the suggestions, I will make some comments here. > > I see that I did not provide nearly enough context again, sorry about that. > > To start, almost all records in the file start with ^S. There are > 1,053,062,790 records in the

[PLUG] reply to suggestions on searching a big file > 1 gig

2012-09-30 Thread website reader
Thanks for the suggestions, I will make some comments here. I see that I did not provide nearly enough context again, sorry about that. To start, almost all records in the file start with ^S. There are 1,053,062,790 records in the file so I simply cannot grep for "^S" because over a billion rec

Re: [PLUG] Question on efficiently searching large files for a simple text match

2012-09-30 Thread Scott Bigelow
How about just: grep -f /tmp/sides.txt bigfile.txt Where sides.txt is a list of all items you'd like to search for, one per line. On Sun, Sep 30, 2012 at 12:53 PM, Robert Munro wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On Sat, 29 Sep 2012 19:36:36 -0700 Eric Wilhelm wrote: >

Re: [PLUG] Question on efficiently searching large files for a simple text match

2012-09-30 Thread Robert Munro
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sat, 29 Sep 2012 19:36:36 -0700 Eric Wilhelm wrote: > # from website reader on Saturday 29 September 2012: >>> I have a list of about 2 to 5 thousand items ... > If you don't know Perl, I think this is a really good time to read > the `man perlintr