Re: [sqlalchemy] convert subset to dictionary

2020-07-07 Thread 'Jonathan Vanasco' via sqlalchemy
Based on what you shared above: * The "Subject" table is: `StudentId, SubjectCode, SubjectName` * There are 181 subjects It looks like you don't have a "Subject" table, but a "StudentAndSubject" table. I think you'd have a bigger performance improvement by normalizing that data into two

Re: [sqlalchemy] convert subset to dictionary

2020-07-07 Thread Justvuur
Nope, I'm running it on a Xubuntu VM. The VM only has 4 gigs of RAM and 4 cores assigned to it. Nope, the students table has created_on, updated_on, id, 3 integer columns and a boolean column. The subjects table has student_id, code and name columns. The code column is character varying(100)

Re: [sqlalchemy] Attaching a second database to a connection

2020-07-07 Thread Mike Bayer
On Tue, Jul 7, 2020, at 8:11 AM, Richard Damon wrote: > Ok, I guess I knew you could execute explicit SQL but wasn't thinking > about it or coming across it in my searches. In my case I wouldn't want > to automatically connect, as it will be done at a specific time for a > specific operation, so

Re: [sqlalchemy] convert subset to dictionary

2020-07-07 Thread Simon King
2:25 minutes, even for 1,267,000 rows, sounds like quite a long time to me. Are you physically a long way from the database, querying across the internet? Do the tables contain many columns, or perhaps large BLOB or TEXT fields? If there are large fields that you don't need very often, you could

Re: [sqlalchemy] "Virtual models" for JSONB data?

2020-07-07 Thread Burak Arslan
On 03/07/2020 16:21, Pedro Ferreira wrote: > Hi, > > I was wondering if there's some way to create some sort of a "virtual" > model which is not based on an actual table? > > Use case: I have a model (e.g. `Room`) which contains a `data` JSONB > attribute/column which is supposed to store

Re: [sqlalchemy] Attaching a second database to a connection

2020-07-07 Thread Richard Damon
Ok, I guess I knew you could execute explicit SQL but wasn't thinking about it or coming across it in my searches. In my case I wouldn't want to automatically connect, as it will be done at a specific time for a specific operation, so I could do the ATTACH specifically. Will I need to explicitly

Re: [sqlalchemy] convert subset to dictionary

2020-07-07 Thread Justvuur
I'm currently testing with 7000 students with 181 subjects. I first went over to the DB to run the query directly on there. I did an innerjoin between the two tables. It took about 2:25 minutes. I then selected specific columns from the two tables and the time dropped to about 2 minutes or so. I

Re: [sqlalchemy] convert subset to dictionary

2020-07-07 Thread Simon King
How long is it taking? You mentioned 2000 students with 100 subjects each, so there are something like 200,000 rows in the Subjects table, and you need to load all of it. I wouldn't expect that to take longer than a couple of seconds though. Simon On Mon, Jul 6, 2020 at 7:34 PM Justvuur wrote: