Hi,
On 29 March 2011 14:30, Hanlie Pretorius <[email protected]> wrote:
> Yes, I was trying to read the file, trim the spaces and write a CSV
> file, but the input file was too big. I found a solution using a
> Python script to split big files
>
I know this is a python list, but for these sort of problems, usual UNIX
shell tools (awk) work very well and are efficient. In your case, if your
file is my_file.txt, the following one liner should work (not tested):
*cat my_file.txt | awk '{if ( (($1 >= -66483.300) && ( $1 <= -33474.900) )
&& (($2 >= -3155672.310) && ( $2 <= -3155672.310) ) ){ print $0}}' >
my_filtered_file.txt*
If you want to debug it (don't take my word for it! :D), you can use head to
examine the first 10 lines:
*head -n 10 my_file.txt | awk '{if ( (($1 >= -66483.300) && ( $1
<= -33474.900) ) && (($2 >= -3155672.310) && ( $2 <= -3155672.310) ) ){
print $0}}' | less*
I have used this approach with 2Gb+ files without any problems. It's fast
and straightforward. Best tool for the job and all that :)
Regards,
Jose
