[web2py] Re: comparing datetime objects in db query fails

2011-09-14 Thread nick name
Yes. I'm saying I can't find my test case that was failing without it (although I'm still trying to recreate one). > I thought this was a performance issue as well as db(...).select() returns parsed types and it is best if the driver does it as opposed if web2py does it. It probably is bette

[web2py] Re: comparing datetime objects in db query fails

2011-09-14 Thread Massimo Di Pierro
Are you telling me that the line driver_args['detect_types'] = self.driver.PARSE_DECLTYPES which I added yesterday can be safely removed? I thought this was a performance issue as well as db(...).select() returns parsed types and it is best if the driver does it as opposed if web2py does it. On

[web2py] Re: comparing datetime objects in db query fails

2011-09-13 Thread nick name
Ok, so I've just spent an hour trying to recreate the old problems I have (without the PARSE_DECLTYPES), and I am unable to. I know for sure that I was occasionally getting strings back when I expected a datetime() object, because I had a test in the code that would report that, and it stopped

[web2py] Re: comparing datetime objects in db query fails

2011-09-13 Thread nick name
Oh, and regarding backward compatibility - it will break anything that was expecting a string returned from selecting a date or timestamp field. Such code wouldn't work on other databases.

[web2py] Re: comparing datetime objects in db query fails

2011-09-13 Thread nick name
by default, the sqlite driver will return everything as a string, regardless of what the real type of the field is -- because sqlite actually stores everything as a string (as the original poster describes here https://groups.google.com/d/msg/web2py/5ABy8XH4ujU/6xtq0xQB8nEJ ; ). I can't find my

[web2py] Re: comparing datetime objects in db query fails

2011-09-13 Thread Massimo Di Pierro
I added this in trunk so we can test it but I do not fully understand what this does. Do you? Can this break backward compatibility?

[web2py] Re: comparing datetime objects in db query fails

2011-09-02 Thread nick name
Two patches that solve this in different ways (both against trunk, but would probably apply cleanly to any recent version): 1 - patch to welcome application diff -r 1462bcd5ea7a applications/welcome/models/db.py --- a/applications/welcome/models/db.pyFri Sep 02 20:41:54 2011 -0500 +++ b/appl

[web2py] Re: comparing datetime objects in db query fails

2011-09-01 Thread Massimo Di Pierro
Want to try send me a patch? On Sep 1, 6:27 pm, nick name wrote: > If I understand correctly, Massimo was unaware of this option when he > originally implemented the DAL for sqlite, and making it default now is not > backward compatible.

[web2py] Re: comparing datetime objects in db query fails

2011-09-01 Thread nick name
If I understand correctly, Massimo was unaware of this option when he originally implemented the DAL for sqlite, and making it default now is not backward compatible.

[web2py] Re: comparing datetime objects in db query fails

2011-09-01 Thread Luis Goncalves
Thanks for the solution. Sorry for the slow reply - I wanted to try it out, but haven't since I had already fixed the problem by explicitly converting from strings to datetimes, and moved on with my work. One question: Why isn't the use of PARSE_COLTYPES the default behavior? It seems DAL/sq

[web2py] Re: comparing datetime objects in db query fails

2011-08-19 Thread nick name
This thread might be relevant: https://groups.google.com/d/topic/web2py/1N0TwMOgp3o/discussion If you do use the PARSE_COLTYPES feature, datetimes work perfectly.

[web2py] Re: comparing datetime objects in db query fails

2011-08-19 Thread Luis Goncalves
I think I have figured it out, with the help of a python expert at work. I am using sqlite3, which internally stores data as strings. When creating my real database (not the toy example), I read raw date and time data as an isoformat string, and relied on DAL/web2py to convert. But perhaps in s

[web2py] Re: comparing datetime objects in db query fails

2011-08-19 Thread Luis Goncalves
Ciao Massimo, I wasn't clear in my post : It's not a web app, I'm using DAL as part of a data analysis program I am writing. I debug with ipython, but the script/programs I write can be called like : python test_412_determine_open_or_closed_lane.py , which outputs results to a file. t

[web2py] Re: comparing datetime objects in db query fails

2011-08-19 Thread Massimo Di Pierro
Perhaps not the problem but this: sys.path.append("/mnt/data1/web2py") is not thread safe. It will be executed at every request and in the best of cases you will get a memory leak. On Aug 19, 12:51 pm, Luis Goncalves wrote: > Here is a simple example ... but it seems to behave properly: > >   "

[web2py] Re: comparing datetime objects in db query fails

2011-08-19 Thread Luis Goncalves
Here is a simple example ... but it seems to behave properly: """ test datetime in db queries """ import sys sys.path.append("/mnt/data1/web2py") from gluon import * import datetime import random db = DAL('sqlite://debug_datetime.sqlite', folder='/home/goncalves/matlab/objr

[web2py] Re: comparing datetime objects in db query fails

2011-08-19 Thread Luis Goncalves
I think you are right, Anthony! In the original query, first_search (or gar with), first_search[270] is the first entry on the new date!!! (and also first_search[269] is the same entry as broken_query[269], so the search results are the same up to that point). I will still make a simple examp

[web2py] Re: comparing datetime objects in db query fails

2011-08-19 Thread Luis Goncalves
Apologies again - all typos. The first query was built by typing (no longer in my ipython history), and full of mistakes. The proper way to do this is to create a simple sample that everyone can replicate (too tired late last night to do so..). Will do so now, constrained by work interruption

[web2py] Re: comparing datetime objects in db query fails

2011-08-19 Thread Anthony
Are all the results (2012, 12, 10) and earlier? Maybe it's only comparing the dates and not the times for some reason (don't know why). In that case, since you're using strict < (rather than <=), it would make sense that no results would be later than (2012, 12, 10). Anthony On Friday, August

[web2py] Re: comparing datetime objects in db query fails

2011-08-19 Thread Jose C
Hmm, nothing stands out as wrong in the logic to me. Could `store_num` or `lane_num` have changed between runs? Is it a shared db, i.e. is it possible someone else is accessing the database (adding/removing records) while you're testing? You have some other inconsistencies in the output above. Fo

Re: [web2py] Re: comparing datetime objects in db query fails

2011-08-19 Thread Luis Goncalves
Cut and paste, sorry! Wanted to rename broken_search for clarity and failed. Late night... —« sent from my Droid tablet »— On Aug 19, 2011 12:12 AM, "Jose C" wrote: >> In [183]: broken_search = db( (db.transEntry.dateTime < ex) & >> (db.transEntry.storeNumber==store_num) & >> (db.transEntry.term

[web2py] Re: comparing datetime objects in db query fails

2011-08-19 Thread Jose C
> In [183]: broken_search = db( (db.transEntry.dateTime < ex) & > (db.transEntry.storeNumber==store_num) & > (db.transEntry.terminalNumber==lane_num) ).select( > orderby=db.transEntry.dateTime) > > In [184]: len(gar2) > Out[184]: 270 In your exmaple above, in [183] you're assigning to variable `br