Re: [sqlalchemy] constructing `Mapper` objects is slow (≈4 seconds) when running a local test

2020-12-01 Thread Mike Bayer
On Tue, Dec 1, 2020, at 3:16 PM, Bill Finn wrote: > On Tue, Dec 1, 2020 at 7:42 PM Mike Bayer wrote: do your "tests" each run in separate processes? >>> >>> Heh, is "tests" surrounding in quotes because we're constructing our entire >>> data model for each test? FWIW, we call them

Re: [sqlalchemy] constructing `Mapper` objects is slow (≈4 seconds) when running a local test

2020-12-01 Thread Bill Finn
On Tue, Dec 1, 2020 at 7:42 PM Mike Bayer wrote: > do your "tests" each run in separate processes? > > > Heh, is "tests" surrounding in quotes because we're constructing our > entire data model for each test? FWIW, we call them integration tests, and > we aren't under the illusion that they are

Re: [sqlalchemy] constructing `Mapper` objects is slow (≈4 seconds) when running a local test

2020-12-01 Thread Mike Bayer
On Tue, Dec 1, 2020, at 11:03 AM, Bill Finn wrote: > Hi Mike, > > Thanks again for the response. > >> do your "tests" each run in separate processes? > > Heh, is "tests" surrounding in quotes because we're constructing our entire > data model for each test? FWIW, we call them integration

Re: [sqlalchemy] constructing `Mapper` objects is slow (≈4 seconds) when running a local test

2020-12-01 Thread Bill Finn
Hi Mike, Thanks again for the response. > do your "tests" each run in separate processes? > Heh, is "tests" surrounding in quotes because we're constructing our entire data model for each test? FWIW, we call them integration tests, and we aren't under the illusion that they are unit tests. To

Re: [sqlalchemy] constructing `Mapper` objects is slow (≈4 seconds) when running a local test

2020-12-01 Thread Mike Bayer
On Mon, Nov 30, 2020, at 7:37 PM, Bill Finn wrote: > Hi Mike, > > Thanks very much for the helpful response. I'll include a few follow-up notes > below, but I suspect the solution to our problem is either to: > > 1. partition our tests such that any given test only imports the models that >

Re: [sqlalchemy] constructing `Mapper` objects is slow (≈4 seconds) when running a local test

2020-11-30 Thread Bill Finn
Hi Mike, Thanks very much for the helpful response. I'll include a few follow-up notes below, but I suspect the solution to our problem is either to: 1. partition our tests such that any given test only imports the models that it depends on (rather than blanketly importing all of the model

Re: [sqlalchemy] constructing `Mapper` objects is slow (≈4 seconds) when running a local test

2020-11-30 Thread Mike Bayer
On Mon, Nov 30, 2020, at 5:17 PM, Bill Finn wrote: > Hello! > > We're running a unit test that includes constructing ≈330 tables & ≈2,000 > columns total in a local PostgreSQL 10.13 database (on disk, not in memory). > After profiling the test, we found that it takes ≈4 seconds to create all

[sqlalchemy] constructing `Mapper` objects is slow (≈4 seconds) when running a local test

2020-11-30 Thread Bill Finn
Hello! We're running a unit test that includes constructing ≈330 tables & ≈2,000 columns total in a local PostgreSQL 10.13 database (on disk, not in memory). After profiling the test, we found that it takes ≈4 seconds to create all of the SQLAlchemy `Mapper` objects [0], which are currently a