Re: csv read _csv.Error: line contains NULL byte

2014-03-21 Thread Mark Lawrence
On 21/03/2014 14:46, chip9m...@gmail.com wrote: On Friday, March 21, 2014 2:39:37 PM UTC+1, Tim Golden wrote: Without disturbing your existing code too much, you could wrap the input_reader in a generator which skips malformed lines. That would look something like this: def unfussy_reader(

Re: csv read _csv.Error: line contains NULL byte

2014-03-21 Thread Tim Golden
On 21/03/2014 14:46, chip9m...@gmail.com wrote: > I am sorry I do not understand how to get to each row in this way. > > Please could you explain also this: > If I define this function, > how do I change my for loop to get each row? Does this help? #!python3 import csv def unfussy_reader(csv_

Re: csv read _csv.Error: line contains NULL byte

2014-03-21 Thread chip9munk
Ok, I have figured it out: for i, row in enumerate(unfussy_reader(input_reader): # and I do something on each row Sorry, it is my first "face to face" with generators! Thank you very much! Best, Chip Munk -- https://mail.python.org/mailman/listinfo/python-list

Re: csv read _csv.Error: line contains NULL byte

2014-03-21 Thread chip9munk
On Friday, March 21, 2014 2:39:37 PM UTC+1, Tim Golden wrote: > Without disturbing your existing code too much, you could wrap the > > input_reader in a generator which skips malformed lines. That would look > > something like this: > > > > def unfussy_reader(reader): > > while True: >

Re: csv read _csv.Error: line contains NULL byte

2014-03-21 Thread Tim Golden
On 21/03/2014 13:29, chip9m...@gmail.com wrote: > Hi all! > > I am reading from a huge csv file (> 20 Gb), so I have to read line by line: > > for i, row in enumerate(input_reader): > # and I do something on each row > > Everything works fine until i get to a row with some strange symbols

csv read _csv.Error: line contains NULL byte

2014-03-21 Thread chip9munk
Hi all! I am reading from a huge csv file (> 20 Gb), so I have to read line by line: for i, row in enumerate(input_reader): # and I do something on each row Everything works fine until i get to a row with some strange symbols "0I`00�^" at that point I get an error: _csv.Error: line contai