I've been working on a migration script, and turned to my trusty friend the WindowedRangeQuery recipe:
https://bitbucket.org/zzzeek/sqlalchemy/wiki/UsageRecipes/WindowedRangeQuery I ran into a small problem and hoping someone else may have done this and saved me some work. I'm trying to work out some edge cases on a large table, and the default performance is pretty bad in Postgres. The two big issues are: 1. The initial query to create an iterable range is painful. It takes me about 2 minutes. This seems to be due to: a - loading a lot of data into memory b - sequentially scanning the 'id' column. this function won't use the primary key, or any other index, for id. it seq scans the whole table. 2. Window ranges are created for all ids in the table. If I only want to work on half the records, with a min/max, the 'iterable' query doesn't use the min/max filter values; it generates 100% of potential ranges based on the id, and the filters are only used for the 'where' clause. Has anyone worked out an improved version of this technique, or am I cobbling something together myself? -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.