Re: [Tutor] genfromtxt and dtype to convert data to correct format
Thanks again Alan for your help and patience. Your earlier suggestion works; i just now realized that the output was indeed correct. Thanks again--EK ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] genfromtxt and dtype to convert data to correct format
On 19/05/16 12:51, Ek Esawi wrote:
> Thanks again!
>
> I tried a combination of the suggestions and still not getting what i want.
>
> Here are the original code, file, and output.
>
> CODE
>
> mport csv; import numpy as np; from datetime import datetime, date, time
>
> CF = lambda date: datetime.strptime(bytes.decode(date),
> '%m/%d/%Y').strftime('%Y-%m-%d')
Again you have strftime at the end so it will result in a string
output. You need to remove that if you want a date/time/datetime
object as a result.
That's assuming that it is a datetime style object that you want.
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] genfromtxt and dtype to convert data to correct format
Thanks again!
I tried a combination of the suggestions and still not getting what i want.
Here are the original code, file, and output.
CODE
mport csv; import numpy as np; from datetime import datetime, date, time
CF = lambda date: datetime.strptime(bytes.decode(date),
'%m/%d/%Y').strftime('%Y-%m-%d')
CF1 = lambda time: datetime.strptime(bytes.decode(time),
'%H:%M').strftime('%H:%M:%S')
MyFile='c:/Users/EK Esawi/My Documents/Temp/GOSA-3.csv'
CRNs={'Date':
CF,'Time':CF1,'Interval':CF1,'Duration':CF1,'Preplay':CF1,'Prediction':CF1}
data = np.genfromtxt(MyFile,
names=True,delimiter=',',converters=CRNs,dtype=None)
print(data)
INPUT
O Date Geyser Time Interval Duration Preplay Heght Prediction 312171
7/1/1995 Position 13:37 1:43 4:42 13:16 162 13:19 358237 5/25/1993 Position
12:22 1:31 4:16 12:03 160 12:13 339971 7/17/1994 Position 15:54 1:23 4:36
15:43 160 15:51
OUTPUT
[ (312171, '1995-07-01', b'Old Faithful', '13:37:00', '01:43:00',
'04:42:00', '13:16:00', 162, '13:19:00')
(358237, '1993-05-25', b'Old Faithful', '12:22:00', '01:31:00',
'04:16:00', '12:03:00', 160, '12:13:00')
(339971, '1994-07-17', b'Old Faithful', '15:54:00', '01:23:00',
'04:36:00', '15:43:00', 160, '15:51:00')
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] genfromtxt and dtype to convert data to correct format
On 19/05/16 01:31, Ek Esawi wrote:
> Thanks Alan!
>
> Taking the strtime off did not work either and printed dattime.dattime(very
> long date format).
But isn't that just the representation of a datetime object (which is
what strptime produces)? In other words it's an object not a string.
Can you show us what you get back?
Or try printing the type() of the object:
eg.
print (type( row[1])
print (type( row[3])
You may be getting confused between the representation of the
object and the actual data. But until we see some actual code
and the actual output it produces it is hard to be sure.
Can you send us a cut 'n paste of an actual python session
so that we can see the parameters to genfromtxt and how
you are producing the output?
For example I'd expect something like:
>>> import datetime as dt
>>> d = dt.datetime.strptime("07/05/2015","%m/%d/%Y")
>>> t = (1,d,'foo')
>>> print(t)
(1, datetime.datetime(2015, 7, 5, 0, 0), 'foo')
>>>
Notice the middle entry is a datetime object, which is,
I think, what you want? You can get just the date (or time)
portion if you want by calling the appropriate method.
You could do that in your lambdas:
>>> CF = lambda datestr: dt.datetime.strptime(datestr,
'%m/%d/%Y').date()
>>> d2 = CF('11/08/2012')
>>> data = (1,d2,'My string')
>>> print(data)
(1, datetime.date(2012, 11, 8), 'My string')
>>>
hth
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] genfromtxt and dtype to convert data to correct format
Thanks Alan! Taking the strtime off did not work either and printed dattime.dattime(very long date format). I was able to get the integer and string parts fixed via dtype, but could not do the same for date and time. I have a feeling that structured arrays might do the trick but don't know yet how. Thanks again--EK ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] genfromtxt and dtype to convert data to correct format
On 18/05/16 14:14, Ek Esawi wrote: > OPS! Sorry, I made a mistake on the posting. My output is in a string > format as shown below. I want the 1st and 8th integers, the 3rd string > which is OK as, the 2nd date and the rest time. You will need to convert the integers manually I suspect with int(row[0]) and int(row[7]). For the dates and times, what happens if you simply remove the strftime() functions from your lambdas? Things are complicated slightly by your use of genfromtxt since it's part of numpy and not in the standard library. I have no idea what it does so there may be tricks you can do with it. > [ (b'312171', '1995-07-01', b'Old', '13:37:00', '01:43:00', '04:42:00', > '13:16:00', b'162', '13:19:00') -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] genfromtxt and dtype to convert data to correct format
Thanks Sydney. The print statement prints items w/o quotation marks, but doesn't change the type. I want the datatype to be changed. I am thinking it can eb done with astructured array dtyep but have not been able to get to wort. Thanks again--EK ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] genfromtxt and dtype to convert data to correct format
On 18/05/2016 14:14, Ek Esawi wrote: OPS! Sorry, I made a mistake on the posting. My output is in a string format as shown below. I want the 1st and 8th integers, the 3rd string which is OK as, the 2nd date and the rest time. I tried several combinations of dtype but could not get the date and time data to without the single quote -string format. [ (b'312171', '1995-07-01', b'Old', '13:37:00', '01:43:00', '04:42:00', '13:16:00', b'162', '13:19:00') (b'358237', '1993-05-25', b'Old', '12:22:00', '01:31:00', '04:16:00', '12:03:00', b'160', '12:13:00') ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor Try the following; line1 = [ (b'312171', '1995-07-01', b'Old', '13:37:00', '01:43:00', '04:42:00','13:16:00', b'162', '13:19:00')] line2 = [(b'358237', '1993-05-25', b'Old', '12:22:00', '01:31:00','04:16:00','12:03:00', b'160', '12:13:00')] item1A = line1[0][0] item1B = line1[0][7] item2A = line2[0][0] item2B = line2[0][7] print item1A, item1B print item2A, item2B -- Sydney ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] genfromtxt and dtype to convert data to correct format
OPS! Sorry, I made a mistake on the posting. My output is in a string format as shown below. I want the 1st and 8th integers, the 3rd string which is OK as, the 2nd date and the rest time. I tried several combinations of dtype but could not get the date and time data to without the single quote -string format. [ (b'312171', '1995-07-01', b'Old', '13:37:00', '01:43:00', '04:42:00', '13:16:00', b'162', '13:19:00') (b'358237', '1993-05-25', b'Old', '12:22:00', '01:31:00', '04:16:00', '12:03:00', b'160', '12:13:00') ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] genfromtxt and dtype to convert data to correct format
On 17/05/16 18:11, Ek Esawi wrote:
> output comes out in the correct format (the desired output as shown below).
> I used converters to covert time and date values, but all came out in
> string format (output below).
What makes you think they are strings? I would expect to see quote signs
if they were strings but there are no quotes...
> CF = lambda date: datetime.strptime(bytes.decode(date),
> %m/%d/%Y).strftime(%Y-%m-%d) # convert data
>
> CF1 = lambda time: datetime.strptime(bytes.decode(time),
> %H:%M).strftime(%H:%M:%S) # convert time
But both of these lambdas return strings (strftime)
You parse the date/time from the bytes then call strftime
on the result which returns a string.
But also you don't have quotes around your format strings
so you should get a syntax error. Please post the actual
code you are using.
>
> MyFile=New_file
> CRNs={Date: CF,Time:CF1,In:CF1,Dur:CF1,PP:CF1,Pre:CF1}
> data = np.genfromtxt(MyFile,
>names=True,delimiter=,,converters=CRNs,dtype=None)
>
> Input
> ODateName TimeInt
> Dur PP H Pred
>
> 312171 7/1/1995 Old 13:37 1:434:42
> 13:16 162 13:19
>
> Output
> [ (312171, 1995-07-01, bOld, 13:37:00, 01:43:00, 04:42:00, 13:16:00, 162,
> 13:19:00)
>
>
> Desired output
> [ (312171, 1995-07-01, bOld, 13:37:00, 01:43:00, 04:42:00, 13:16:00, 162,
> 13:19:00)
Your desired output looks exactly like the real output so it's
not clear what exactly you want to be different?
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
[Tutor] genfromtxt and dtype to convert data to correct format
Hi All—
I am reading data from a file using genfromtxt. A part of my data (input,
output, and desired output) is shown below which consists of string,
integer, time, and date type data. I want to read the data so that the
output comes out in the correct format (the desired output as shown below).
I used converters to covert time and date values, but all came out in
string format (output below).
I tried to use structured dtype but without success and I don’t want to use
pandas. What is the best remedy to achieve the desired output shown below?
Thanks in advance—EK
++
Code
CF = lambda date: datetime.strptime(bytes.decode(date),
%m/%d/%Y).strftime(%Y-%m-%d) # convert data
CF1 = lambda time: datetime.strptime(bytes.decode(time),
%H:%M).strftime(%H:%M:%S) # convert time
MyFile=New_file
CRNs={Date: CF,Time:CF1,In:CF1,Dur:CF1,PP:CF1,Pre:CF1}
#converters
data = np.genfromtxt(MyFile,
names=True,delimiter=,,converters=CRNs,dtype=None)
Input
ODateName TimeInt
Dur PP H Pred
312171 7/1/1995 Old 13:37 1:434:42
13:16 162 13:19
358237 5/25/1993New 12:22 1:314:16
12:03 160 12:13
Output
[ (312171, 1995-07-01, bOld, 13:37:00, 01:43:00, 04:42:00, 13:16:00, 162,
13:19:00)
(358237, 1993-05-25, bNew, 12:22:00, 01:31:00, 04:16:00, 12:03:00, 160,
12:13:00)]
Desired output
[ (312171, 1995-07-01, bOld, 13:37:00, 01:43:00, 04:42:00, 13:16:00, 162,
13:19:00)
(358237, 1993-05-25, bNew, 12:22:00, 01:31:00, 04:16:00, 12:03:00, 160,
12:13:00)]
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
