open a ticket, this can be done. I like the idea of passing a processor.
On Feb 9, 3:14 pm, Anthony <abasta...@gmail.com> wrote: > I've been thinking about something like this as well. Instead of a separate > select_raw() method, maybe we can just add a raw=True|False argument to the > existing select() method. I like the namedtuple idea as well (I think some > adapters already provide that as an option -- e.g., psycopg2). > > Anthony > > > > > > > > On Thursday, February 9, 2012 3:04:41 PM UTC-5, nick name wrote: > > > Yes, that is the basis of what I am suggesting. > > > There is not currently such a thing; there is something called > > 'select_raw' implemented in the GoogleDataStore adapter, but not in > > anything else, and it isn't exactly what I am proposing. > > > To elaborate: > > > Assume the table is defined as follows: > > > reftable = db.define_table('reftable', Field('a', string)) > > table = db.define_table('table', Field('b', reftable)) > > > In my case, I need to pull all the records (60,000) from the database to > > compute some aggregation which I cannot compute using sql. There are two > > alternatives here: > > > r1 = db().select(table.ALL) # takes > 6 seconds > > > r2 = db.executesql(db._select(table.ALL)) # takes ~0.1sec > > > The records returned in the first instance are much richer; they have > > record chasing (e.g. I can do r1[0].b.a to select through the foreign key), > > they have methods like r1[0].update_record() and r1[0].delete_record(), and > > other nice stuff. > > > However, for this use, I don't need the additional records, and I do need > > the speed, so I would rather use r2. However, r2 is not a direct > > replacement -- it doesn't have the column names. If I use > > > r3 = db.executesql(db._select(table.ALL), as_dict=True) # still takes > > ~0.1sec > > > I can do r3[0]['b'] but I cannot do r3[0].b; and it takes a lot more > > memory than r2. > > > A suggestion: add another parameter, processor=... which, if available, > > will be called with the db.connection.cursor, returning a function, through > > which each routine will be passed; example > > > def named_tuple_process(name, description): > > from collections import namedtuple > > fields = ' '.join([x[0] for x in description]) > > return namedtuple(name, fields) > > > r4 = db.executesql(db._select(table.ALL), process=lambda x: > > named_tuple_process('tablerec', x)) > > > r4[0].b # will now work; not a full replacement, but good enough for many > > uses. > > > In fact, you can do that externally - > > > r4 = db.executesql(db._select(table.ALL)) > > f = named_tuple_process('tablerec', db._adapter.cursor.description) > > r4 = [f(x) for x in r4] > > > But this requires reaching into the internals of the db adapter. > > > Finally, I propose to define x.raw_select(*args) to do: > > db.executesql(x._select(*args)) > > > which would make this a relatively clean replacement.