Re: Decimating Excel files
I tried to open the file with Kate, trust me, it's an Excel file.
I'm using xlrd, it works beautifully (although come to think of it, I
haven't tried writing to an .xls file yet... hmmm)
> To clear up the doubts, I'd suggest that the OP do something like this
> at the Python interactive prompt:
>
> print repr(open('nasty_file.xls', 'rb').read(512))
>
> If that produces recognisable stuff, then it's a CSV file (or a tab
> separated file) masquerading as an XLS file.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Decimating Excel files
Excellent suggestion. I'm going with xlrd! Thanks > I've had good luck with xlrd. It does not require using COM, Excel, or even > Windows! > http://www.lexicon.net/sjmachin/xlrd.htm > Robert Kern -- http://mail.python.org/mailman/listinfo/python-list
Re: Decimating Excel files
Yeah, it definitely an Excel file (so says Kate). > Is the file format really native Excel, or is a CSV or TSV file? I've > seen apps (one is a data acquisition program, as a matter of fact) > that create "Excel" files that are just CSV or TSV files. Try opening > the file with a text editor to see if it's plain ASCII text. -- http://mail.python.org/mailman/listinfo/python-list
Re: Decimating Excel files
Thanks, but I was looking for a python solution. > Excel has VBA and can do this easily. One thing about > Excel's VBA is that it already understands Excel. -- http://mail.python.org/mailman/listinfo/python-list
Decimating Excel files
No, I don't want to destroy them (funny how the word 'decimate' has changed definition over the years) :). We have a data acquisition program that saves its output to Excel's .xls format. Unfortunately, the programmer was too stupid to write files the average user can read. I'd like some advice on how to go about: 1. Reading a large Excel file and chop it into many Excel files (with only 65535 lines per file) or 2. Decimate an Excel file & write... say every other line (user selectable)... to a new file. I'm pretty experienced at reading and writing simple text files, but this is my first foray into using COM. I would imagine either choice 1 or 2 is pretty simple once I can get the file open. Thanks in advance. -- http://mail.python.org/mailman/listinfo/python-list
Two dimensional lists
I might get an answer since I didn't call them arrays. :^) Ok, I have 2 lists that I need to process individually, then merge them into a 2x list and fill with data. arinc429 = ['ab', '2b', '0b', '21', 'c1', '61', '11', 'db', '9b', '5b', 'eb', '6b', '1b', '6e', '3e'] iPIDs = [300, 301, 320, 321] merged[arinc429, iPIDs] # PID & a429 are defined elsewhere a_idx = iPIDs.index[PID] p_idx = [arinc429.index[a429] # shouldn't I be able to fill the lists simply by pointing to a location? matrix[a_idx, p_idx] = 0x219 # and so on? tia -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Windows Editors
I prefer PyScripter too, but would like to know if I can have 'indentation guides' enabled like PythonWin allows. On 1/21/07, Stef Mientki <[EMAIL PROTECTED]> wrote: > I tried 2 of them, and only was stable enough: PyScripter > http://mmm-experts.com/Products.aspx?ProductID=4 -- http://mail.python.org/mailman/listinfo/python-list
Re: OT Annoying Habits (Was: when format strings attack)
I should write a python script to read this. :) >.snoitnevnoc >hsilgnE tpada )ylbissop revenehw( dluohs ew os dna ,naitraM ton ,puorgswen >egaugnal hsilgnE na no er'ew ,segaugnal hcus era ereht fi neve tuB -- http://mail.python.org/mailman/listinfo/python-list
Re: How to comment code?
If it's hard to write, it should be hard to read! :) On 1/19/07, Martin P. Hellwig <[EMAIL PROTECTED]> wrote: > Hi all, (snip) > However since I'm learning more of python I've struggled with > commenting, how should I've comment my code (snip) -- http://mail.python.org/mailman/listinfo/python-list
variable scope
Greetings,
I've been using Python to successfully parse files. When the entire
program was smaller, the variable firstMsg worked fine, but now
doesn't because it's used in function PID_MinMax. I know it's a result
of variables and their scope.
I declare the variable 'firstMsg = 0' in the main loop, pass it to the
function 'PID_MinMax(firstMsg, PID)'. When the function increments the
variable, it doesn't pass it back to the main program. What am I doing
wrong?
major snippage) ---
firstMsg = 0
skipHeader = 13
pPIDs = ['0321'] # hundreds more
pLen = len(pPIDs)
pMsgNum = pLen * [0]
pMax = pLen * [0]
pMin = pLen * [10]
pLast = pLen * [0]
def PID_MinMax(firstMsg, PID):
idx = pPIDs.index(PID)
pMsgNum[idx] += 1
# Need to have 2 samples to determine Delta
if firstMsg != 0:
tDelta = tCurrent - pLast[idx]
if tDelta > pMax[idx]:
pMax[idx] = tDelta
if tDelta < pMin[idx]:
pMin[idx] = tDelta
elif firstMsg == 0:
firstMsg = 1
pLast[idx] = tCurrent
print pMin, pMax
return firstMsg
## main ##
bf_file = file('bf_data/sByteflightLog_wISS.txt', 'r')
for line in bf_file:
# skip header
if skipHeader != 0:
skipHeader -= 1
# skip footer
elif line == '\n':
break
else:
raw_msg = line.split()
tCurrent = int(raw_msg[0], 16) * 0.0001
PID = raw_msg[2]
if PID in pPIDs:
PID_MinMax(firstMsg, PID)
--
http://mail.python.org/mailman/listinfo/python-list
Python cheatsheets
Curious if anyone has a python cheatsheet* published? I'm looking for something that summarizes all commands/functions/attributes. Having these printed on a 8" x 11" double-sided laminated paper is pretty cool. * cheatsheet probably isn't the right word, but you get the idea. :) -- http://mail.python.org/mailman/listinfo/python-list
Writing more efficient code
Greetings, and happyNewYear to all. I picked up Python a few weeks ago, and have been able to parse large files and process data pretty easily, but I believe my code isn't too efficient. I'm hoping dictionaries will help out, but I'm not sure the best way to implement them. I've been using a bunch of nested if/elif/else statements to select slices (0317 & 03de) from a file, then parse the data (aa, hh, bb, d2-d9) into parameters (a = airspeed, h = heading) & flags. #sample file contents 007d 03 0317 aa aa aa aa aa hh hh hh bb bb 007e 06 03de d2 d3 d4 d5 d6 d7 d8 d9 10 11 # some pseudo code if PID == '03de': flapsCmd = int(d3, 16) if flapsCmd == 0xc0: elif flapsCmd == 0x03: if PID == '0317': airspeed == 'combine aa for airspeed & multiply by 0.1' heading == 'combine hh for heading' mach == 'combine bb for mach & multiply by 0.01' Might dictionaries help in this case... say Label0317(parameterName, slice (d3), scaleFactor(0.1))... I'd like to use them if they'll replace the dozens of nested conditionals. I have roughly 75 different parameters to decode from a file containing ~2.5 million lines of data. I know my pseudo code lacks details, but hopefully I'm getting my point across... (I suppose switch/select/case statements would help a bit, but python doesn't seem to use them... not to start a religious war or anything). Any help (or encouragement) is appreciated. -- http://mail.python.org/mailman/listinfo/python-list
Re: Noobie: Open file -> read characters & multiply
Thanks to all that responded. I chose a modified version of Scott's second recommendation: time = line[:8] decoded_File.write( '%00.4f' % (int(time, 16) * .0001) + ', ') 'print >>' added a CRLF that I didn't need, so I went with '.print' (I need to process about 20 values from the remaining bytes). Thank you. On 12/26/06, Scott David Daniels <[EMAIL PROTECTED]> wrote: ... > > Any help is really appreciated. > for line in AP_file: > print >>decoded_File, '%s.%04d' % divmod(int(line[:8], 16), 1 > ), line[9:].rstrip() > > or: > > for line in AP_file: > print >>decoded_File, '%.4f' % (int(line[:8], 16) * .0001 > ), line[9:].rstrip() ... -- http://mail.python.org/mailman/listinfo/python-list
Noobie: Open file -> read characters & multiply
I've been using Python for a few days. It's such the perfect language
for parsing data!
I really like it so far, but I'm having a hard time reading a file,
reading the first few hex characters & converting them to an integer.
Once the characters are converted to an integer, I'd like to write the
data to another file.
Here's the code snipped to the bare minimum:
---
# Open File
AP_File= open("AP.txt", "r")
decoded_File= open("decoded.txt", "w")
# read & process data line by line
for line in AP_File:
time = int(hex(line[0:8]), 16) * 0.0001 # this line is completely hosed!
decodedFile.write(time)
#close files
AP_File.close()
decoded_File.close()
---
AP.txt
00d5 26 0600 80 00 ec 80 02 03 7d db 02 33
00d5 26 0601 80 00 80 00 02 37 fe 54 01 09
00d5 06 0602 80 00 e0 00 01 29 fe d2 69 99
00d5 06 0603 80 00 e0 00 02 29 fe d2 6a 99
00d5 26 0604 80 00 fe 54 02 09 80 00 01 5d
00d5 06 0605 80 00 e0 00 02 15 fc 71 ca 0b
00d5 4a 0610 81 00 86 00 02 26 12 00 02 a6
00d5 4f 0611 00 00 00 50 00 00 00 00 07 00
00d5 06 0612 80 00 e0 00 01 15 fc 71 c9 0b
00d5 0a 0613 08 5c 04 88 08 98 00 00 00 00
00d5 06 0614 80 00 e0 00 02 01 60 79 82 2b
00d5 0a 0615 08 00 00 00 00 00 00 00 00 00
00d5 26 0616 80 00 80 00 02 5d 04 22 3a 88
(actual files are 250MB!)
decodedTime.txt (should be)
0.0213 26 0600 80 00 ec 80 02 03 7d db 02 33
...
My boss and I are trying to complete the same task (he figured out how
to do it, but his code uses a while != "" loop and doesn't look
pythony (it looks too 'c'). Not that there's anything wrong with that!
Any help is really appreciated.
--
http://mail.python.org/mailman/listinfo/python-list
