Re: Is there a faster way to do this?

2008-08-06 Thread Boris Borcic
Is your product ID always the 3rd and last item on the line ? Else your output won't separate IDs. And how does output = open(output_file,'w') for x in set(line.split(',')[2] for line in open(input_file)) : output.write(x) output.close() behave ? [EMAIL PROTECTED] wrote: I have a csv

Is there a faster way to do this?

2008-08-05 Thread [EMAIL PROTECTED]
I have a csv file containing product information that is 700+ MB in size. I'm trying to go through and pull out unique product ID's only as there are a lot of multiples. My problem is that I am appending the ProductID to an array and then searching through that array each time to see if I've seen

Re: Is there a faster way to do this?

2008-08-05 Thread Gary Herron
[EMAIL PROTECTED] wrote: I have a csv file containing product information that is 700+ MB in size. I'm trying to go through and pull out unique product ID's only as there are a lot of multiples. My problem is that I am appending the ProductID to an array and then searching through that array

Re: Is there a faster way to do this?

2008-08-05 Thread Avinash Vora
On Aug 5, 2008, at 10:00 PM, [EMAIL PROTECTED] wrote: I have a csv file containing product information that is 700+ MB in size. I'm trying to go through and pull out unique product ID's only as there are a lot of multiples. My problem is that I am appending the ProductID to an array and then

Re: Is there a faster way to do this?

2008-08-05 Thread Gary Herron
Avinash Vora wrote: On Aug 5, 2008, at 10:00 PM, [EMAIL PROTECTED] wrote: I have a csv file containing product information that is 700+ MB in size. I'm trying to go through and pull out unique product ID's only as there are a lot of multiples. My problem is that I am appending the ProductID to

Re: Is there a faster way to do this?

2008-08-05 Thread RPM1
[EMAIL PROTECTED] wrote: I have a csv file containing product information that is 700+ MB in size. I'm trying to go through and pull out unique product ID's only as there are a lot of multiples. My problem is that I am appending the ProductID to an array and then searching through that array

Re: Is there a faster way to do this?

2008-08-05 Thread Roy H. Han
Why not just use sets? a = set() a.add(1) a.add(2) On Tue, Aug 5, 2008 at 10:14 PM, RPM1 [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] wrote: I have a csv file containing product information that is 700+ MB in size. I'm trying to go through and pull out unique product ID's only as there are a

Re: Is there a faster way to do this?

2008-08-05 Thread Tomasz Rola
On Tue, 5 Aug 2008, [EMAIL PROTECTED] wrote: I have a csv file containing product information that is 700+ MB in size. I'm trying to go through and pull out unique product ID's only as there are a lot of multiples. My problem is that I am appending the ProductID to an array and then searching