On Tue, 16 Jun 2009 10:42:58 -0700, Scott David Daniels wrote: > Dave Angel wrote: >> Jorge wrote: >>> Hi there, >>> I'm making a application that reads 3 party generated ASCII files, >>> but some >>> times >>> the files are corrupted totally or partiality and I need to know if >>> it's a >>> ASCII file with *nix line terminators. In linux I can run the file >>> command but the applications should run in windows. >>> >>> Any help will be great. >>> >>> Thank you in advance. >>> >>> >> So, which is the assignment: >> 1) determine if a file has non-ASCII characters 2) determine whether >> the line-endings are crlf or just lf >> >> In the former case, look at translating the file contents to Unicode, >> specifying ASCII as source. If it fails, you have non-ASCII In the >> latter case, investigate the 'u' attribute of the mode parameter in the >> open() function. >> >> You also need to ask yourself whether you're doing a validation of the >> file, or doing a "best guess" like the file command. >> >> > Also, realize that ASCII is a 7-bit code, with printing characters all > greater than space, and very few people use delete ('\x7F'), so you can > define a function to determine if a file contains only printing ASCII > and a few control characters. This one is False unless some ink would > be printed. > > Python 3.X: > def ascii_file(name, controls=b'\t\n'): > ctrls = set(controls + b' ') > with open(name, 'rb') as f: > chars = set(f.read()) > return min(chars) >= min(ctrls) ord('~') >= max(chars) > ) and min(chars - ctrls) > ord(' ') > > Python 2.X: > def ascii_file(name, controls='\t\n'): > ctrls = set(controls + ' ') > with open(name, 'rb') as f: > chars = set(f.read()) > return min(chars) >= min(ctrls) and '~' >= max(chars > ) and min(chars - ctrls) > ' ' > > For potentially more performance (at least on 2.X), you could do min and > max on the data read, and only do the set(data) if the min and max are > OK.
You're suggesting that running through the entire data three times instead of once is an optimization? Boy, I'd hate to see what you consider a pessimation! *wink* I think the best solution will probably be a lazy function which stops processing as soon as it hits a character that isn't ASCII. # Python 2.5, and untested def ascii_file(name): from string import printable with open(name, 'rb') as f: for c in f.read(1): if c not in printable: return False return True This only reads the entire file if it needs to, and only walks the data once if it is ASCII. In practice, you may actually get better performance by reading in a block at a time, rather than a byte at a time: # Python 2.5, and still untested def ascii_file(name, bs=65536): # 64K default blocksize from string import printable with open(name, 'rb') as f: text = f.read(bs) while text: for c in text: if c not in printable: return False text = f.read(bs) return True -- Steven -- http://mail.python.org/mailman/listinfo/python-list