[sqlalchemy] Re: Result set slice (session.query)
On Aug 28, 2008, at 12:18 AM, PyDevler wrote: I am using the ORM query strategy using session.query. I vaguely remember seeing in the past that doing a slice on the result set, i.e. result[:10] used to perform a Limit/Offset query, it may not have been so. Nevertheless, with SA 0.5beta3 this is definitely not the case. It seems that limit offset queries are only issued when I index the result set, e.g. result[0] which does a limit 1 offset 0. The question is, how would I be able to extend the slicing to use Limit Offset. e.g: result[:10] = Limit 10 offset 0 I have a large database, and I quickly run out of memory with queries that do not utilize the Limit Offset. Anything better than [result[i] for i in range(0,10)] would be greatly appreciated (since that would run 10 different queries). query[start:end] works just great, I've been using every 0.5 version for a soon-to-be production app here. There may be some edge cases that are not supported, such as result[-5:-10], and we also now have checks which prevent subsequent filtering applied to an already limited query, since this is an ambiguous use case (i.e., do you want to filter from the limited results, or apply the filter first? SQLA would not like to guess. If you were expecting the latter, then I'm glad I decided to implement it this way since the former is more correct). Additionally limit/offset can be applied directly using limit() and offset().Do you have a test case illustrating your specific issue ? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Result set slice (session.query)
On Aug 28, 2008, at 12:40 PM, PyDevler wrote: Hi Michael, (Sorry for the repost at sqlalchemy-devel I was not able to find this post, so I thought it didnt go through) I can write up a small sample later. When I turn echoing on the DB engine, I can see that doing: query[start:end] issues the full query, not using limit, offset. Unlike: query[start] which does issue a limit 1 offset `start` query I forgot about using limit and offset directly, which is what i need to do. But I would have thought that it was more usable to do a limit/offset if a user is requesting a slice of the result set. I undrestand that edge slices may not be supported, especially in reverse, or when they overlap a boundary. I cant reproduce query[start:end] not producing LIMIT/OFFSET. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Result set slice (session.query)
Hi Michael, (Sorry for the repost at sqlalchemy-devel I was not able to find this post, so I thought it didnt go through) I can write up a small sample later. When I turn echoing on the DB engine, I can see that doing: query[start:end] issues the full query, not using limit, offset. Unlike: query[start] which does issue a limit 1 offset `start` query I forgot about using limit and offset directly, which is what i need to do. But I would have thought that it was more usable to do a limit/offset if a user is requesting a slice of the result set. I undrestand that edge slices may not be supported, especially in reverse, or when they overlap a boundary. Thanks, -- Hatem On Aug 28, 8:57 am, Michael Bayer [EMAIL PROTECTED] wrote: On Aug 28, 2008, at 12:18 AM, PyDevler wrote: I am using the ORM query strategy using session.query. I vaguely remember seeing in the past that doing a slice on the result set, i.e. result[:10] used to perform a Limit/Offset query, it may not have been so. Nevertheless, with SA 0.5beta3 this is definitely not the case. It seems that limit offset queries are only issued when I index the result set, e.g. result[0] which does a limit 1 offset 0. The question is, how would I be able to extend the slicing to use Limit Offset. e.g: result[:10] = Limit 10 offset 0 I have a large database, and I quickly run out of memory with queries that do not utilize the Limit Offset. Anything better than [result[i] for i in range(0,10)] would be greatly appreciated (since that would run 10 different queries). query[start:end] works just great, I've been using every 0.5 version for a soon-to-be production app here. There may be some edge cases that are not supported, such as result[-5:-10], and we also now have checks which prevent subsequent filtering applied to an already limited query, since this is an ambiguous use case (i.e., do you want to filter from the limited results, or apply the filter first? SQLA would not like to guess. If you were expecting the latter, then I'm glad I decided to implement it this way since the former is more correct). Additionally limit/offset can be applied directly using limit() and offset().Do you have a test case illustrating your specific issue ? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Result set slice (session.query)
On Aug 28, 10:50 am, Michael Bayer [EMAIL PROTECTED] wrote: I cant reproduce query[start:end] not producing LIMIT/OFFSET. Actually that is right: query[start:end] does produce limit/offset. What I was doing was: query[:end] Since in lists start defaults to 0, that does not produce a Limit/ Offset. Thanks, -- Hatem --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Result set slice (session.query)
On Aug 28, 2008, at 2:59 PM, PyDevler wrote: On Aug 28, 10:50 am, Michael Bayer [EMAIL PROTECTED] wrote: I cant reproduce query[start:end] not producing LIMIT/OFFSET. Actually that is right: query[start:end] does produce limit/offset. What I was doing was: query[:end] Since in lists start defaults to 0, that does not produce a Limit/ Offset. we'll try to look into slices lke [:y] and [x:] producing something reasonable. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
