[Numpy-discussion] A bug in loadtxt and how to convert a string array (hex data) to decimal?
Hi, All, I have found a bug in the loadtxt function. Here is the example. The file name is test.txt and contains: Thist is test 3FF 3fE 3Ef 3e8 3Df 3d9 3cF 3c7 In the Python 2.5.2, I type: test=loadtxt('test.txt',comments='',dtype='string',converters={0:lambda s:int(s,16)}) test will contain array([['102', '3fE'], ['100', '3e8'], ['991', '3d9'], ['975', '3c7']], dtype='|S3') The first two values 102 and 100 are wrong. The reason I am doing this because that I have to process a large amount of data from a file. The data is in hex format. This is only way I found that I can efficiently convert the hex to decimal. Anyone has a good solution? Thanks Frank _ Get more out of the Web. Learn 10 hidden secrets of Windows Live. http://windowslive.com/connect/post/jamiethomson.spaces.live.com-Blog-cns!550F681DAD532637!5295.entry?ocid=TXT_TAGLM_WL_domore_092008___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] A bug in loadtxt and how to convert a string array (hex data) to decimal?
frank wang wrote: Hi, All, I have found a bug in the loadtxt function. Here is the example. The file name is test.txt and contains: Thist is test 3FF 3fE 3Ef 3e8 3Df 3d9 3cF 3c7 In the Python 2.5.2, I type: test=loadtxt('test.txt',comments='',dtype='string',converters={0:lambda s:int(s,16)}) test will contain array([['102', '3fE'], ['100', '3e8'], ['991', '3d9'], ['975', '3c7']], dtype='|S3') It's because of how numpy handles strings arrays (which I admit I don't understand very well.) Basically, it's converting the numbers properly, but truncating them to 3 characters. Try this, which just forces it to expand to strings 4 characters wide: test=loadtxt('test.txt',comments='',dtype='|S4',converters={0:lambda s:int(s,16)}) HTH, Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] A bug in loadtxt and how to convert a string array (hex data) to decimal?
Hi, Ryan, Thank you very much. It solves my problem. I have struggled with this for long time. Frank Date: Thu, 18 Sep 2008 16:39:47 -0500 From: [EMAIL PROTECTED] To: numpy-discussion@scipy.org Subject: Re: [Numpy-discussion] A bug in loadtxt and how to convert a string array (hex data) to decimal? frank wang wrote: Hi, All,I have found a bug in the loadtxt function. Here is the example. The file name is test.txt and contains: Thist is test 3FF 3fE 3Ef 3e8 3Df 3d9 3cF 3c7In the Python 2.5.2, I type: test=loadtxt('test.txt',comments='',dtype='string',converters={0:lambda s:int(s,16)})test will contain array([['102', '3fE'], ['100', '3e8'], ['991', '3d9'], ['975', '3c7']], dtype='|S3')It's because of how numpy handles strings arrays (which I admit I don't understand very well.) Basically, it's converting the numbers properly, but truncating them to 3 characters. Try this, which just forces it to expand to strings 4 characters wide: test=loadtxt('test.txt',comments='',dtype='|S4',converters={0:lambda s:int(s,16)}) HTH, Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion _ See how Windows connects the people, information, and fun that are part of your life. http://clk.atdmt.com/MRT/go/msnnkwxp1020093175mrt/direct/01/___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion