Re: Invoke commands with custom env.py from library
On Thu, Mar 5, 2020, at 1:59 PM, Daniel Cardin wrote:
> 1. I always expected that, with any reasonably non-trivial app, I would
> inevitably need to fall back to using their env.py. My hope was that for
> simpler cases, since this is a testing tool, there would be a way to not
> require any changes to their env.py by default. Ultimately I can document
> what the recommended changes to a typical env.py would be, and that's not the
> end of the world.
>
> 2. I did try to change version_locations, but unless I'm mistaken what I'm
> asking for maybe seems like an unsupported usecase. Attempting to target the
> library env with "script_location" and the local versions/ with
> "version_locations" still seems to look at the wrong location. Given how
> ScriptDirectory is constructed and and then `run_env` directly looks for
> "env.py", it seems like what im asking for is just not an expected usecase.
it's true, it is an assumption that env.py is co-located with the versioning
files, although looking at the source, I don't know that i see why it would
look in the wrong location, ScriptDirectory pretty clearly looks in "dir" for
env.py and "version_locations" for the version files.
I'm wondering why these users would not have an env.py set up? or they would,
and your tool is attempting to do some magic thing on an existing installation
without them making any changes? If there is truly something blocking
ScriptDirectory from working this way, maybe I'd suggest monkeypatching, but
script_directory and version_locations look separate to me and are handled
separately all the way from ScriptDirectory.from_config().
>
> On Thursday, March 5, 2020 at 12:21:54 PM UTC-5, Mike Bayer wrote:
>>
>>
>> On Thu, Mar 5, 2020, at 12:02 PM, Daniel Cardin wrote:
>>> So, yes I mean "commands" as in `alembic.command.upgrade()`.
>>>
>>> The idea would be that the library defines an env.py (e.g. the important
>>> portion of which would look something like:)
>>> ...
>>>
>>> connectable = context.config.attributes.get("connection", None)
>>> with self.connection.connect() as connection:
>>> alembic.context.configure(connection=connection)
>>>
>>> with alembic.context.begin_transaction():
>>> alembic.context.run_migrations()
>>>
>>> which lives at `src/library/foo/env.py`. If the migrations were colocated
>>> with the env.py, then afaik setting "script_location" to "library:foo"
>>> would enable me to run `alembic.command.upgrade()` and it would just work.
>>> However in this case (if possible), the migrations/versions/* would live at
>>> an arbitrary other location (e.g. the code which has installed this
>>> library).
>>
>> Ok two thoughts on that:
>>
>> 1. usually the way this goes is that the project that has the migrations
>> does everything normally, it has its own env.py, and inside that env.py,
>> that's where it imports *your* env.py. that is, instaed of having the usual
>> env.py it would only have:
>>
>> # env.py
>>
>> from daniel_cardins.library.env import run_migrations
>>
>> run_migrations()
>>
>> 2. the locations of the version files vs. the env.py script are actually
>> separate in the config. there is a script_location that it uses to find the
>> home base where env.py is, however there is also a version_locations config
>> that can have any number of other directories in it and these supersede
>> script_location for the version files themselves.
>>
>>
>>
>> I think in general, if the end-user has a bunch of version files set up,
>> it's probably not a big deal that they have a stub "env.py" right there that
>> just calls out to your library. that's something very clear that people
>> looking at a project that uses your library can understand quickly if they
>> already know alembic.
>>
>>
>>
>>
>>>
>>> General workflow:
>>> * person working on project "foo" invokes some cli/command/whatever which
>>> requests an upgrade
>>> * library does whatever requisite setup
>>> * library invokes `alembic.command.upgrade()`
>>> * upgrade() ends up routing the code through the library's `env.py`
>>> * the context of the migration command is targeting the project "foo"'s
>>> local migrations/versions folder
>>>
>>> The specifics of the above are just based on my knowledge of alembic, so if
>>> there's another process i could be doing where env.py isn't "invoked" so
>>> much as the above code block is just called normally, then that's ideal.
>>>
>>> On Thursday, March 5, 2020 at 9:36:09 AM UTC-5, Mike Bayer wrote:
On Thu, Mar 5, 2020, at 8:08 AM, Daniel Cardin wrote:
> I am attempting to write a library which invokes alembic commands, while
> referencing the migrations of a separate package which has installed said
> library.
>
> The intent here, is for the library to invoke the alembic commands with
> an env.py defined in that package. This seems to work through
> config.get("script_location",
Re: Invoke commands with custom env.py from library
1. I always expected that, with any reasonably non-trivial app, I would
inevitably need to fall back to using their env.py. My hope was that for
simpler cases, since this is a testing tool, there would be a way to not
require any changes to their env.py by default. Ultimately I can document
what the recommended changes to a typical env.py would be, and that's not
the end of the world.
2. I did try to change version_locations, but unless I'm mistaken what I'm
asking for maybe seems like an unsupported usecase. Attempting to target
the library env with "script_location" and the local versions/ with
"version_locations" still seems to look at the wrong location. Given how
ScriptDirectory is constructed and and then `run_env` directly looks for
"env.py", it seems like what im asking for is just not an expected usecase.
On Thursday, March 5, 2020 at 12:21:54 PM UTC-5, Mike Bayer wrote:
>
>
>
> On Thu, Mar 5, 2020, at 12:02 PM, Daniel Cardin wrote:
>
> So, yes I mean "commands" as in `alembic.command.upgrade()`.
>
> The idea would be that the library defines an env.py (e.g. the important
> portion of which would look something like:)
> ...
>
> connectable = context.config.attributes.get("connection", None)
> with self.connection.connect() as connection:
> alembic.context.configure(connection=connection)
>
> with alembic.context.begin_transaction():
> alembic.context.run_migrations()
>
> which lives at `src/library/foo/env.py`. If the migrations were colocated
> with the env.py, then afaik setting "script_location" to "library:foo"
> would enable me to run `alembic.command.upgrade()` and it would just work.
> However in this case (if possible), the migrations/versions/* would live at
> an arbitrary other location (e.g. the code which has installed this
> library).
>
>
> Ok two thoughts on that:
>
> 1. usually the way this goes is that the project that has the migrations
> does everything normally, it has its own env.py, and inside that env.py,
> that's where it imports *your* env.py. that is, instaed of having the
> usual env.py it would only have:
>
> # env.py
>
> from daniel_cardins.library.env import run_migrations
>
> run_migrations()
>
> 2. the locations of the version files vs. the env.py script are actually
> separate in the config.there is a script_location that it uses to find
> the home base where env.py is, however there is also a version_locations
> config that can have any number of other directories in it and these
> supersede script_location for the version files themselves.
>
>
>
> I think in general, if the end-user has a bunch of version files set up,
> it's probably not a big deal that they have a stub "env.py" right there
> that just calls out to your library. that's something very clear that
> people looking at a project that uses your library can understand quickly
> if they already know alembic.
>
>
>
>
>
> General workflow:
> * person working on project "foo" invokes some cli/command/whatever which
> requests an upgrade
> * library does whatever requisite setup
> * library invokes `alembic.command.upgrade()`
> * upgrade() ends up routing the code through the library's `env.py`
> * the context of the migration command is targeting the project "foo"'s
> local migrations/versions folder
>
> The specifics of the above are just based on my knowledge of alembic, so
> if there's another process i could be doing where env.py isn't "invoked" so
> much as the above code block is just called normally, then that's ideal.
>
> On Thursday, March 5, 2020 at 9:36:09 AM UTC-5, Mike Bayer wrote:
>
>
>
> On Thu, Mar 5, 2020, at 8:08 AM, Daniel Cardin wrote:
>
> I am attempting to write a library which invokes alembic commands, while
> referencing the migrations of a separate package which has installed said
> library.
>
> The intent here, is for the library to invoke the alembic commands with an
> env.py defined in that package. This seems to work through
> config.get("script_location", "package_name:foldername")
> but then obviously expects the actual migrations to be colocated at the
> same location.
>
> My guess would be, if this is possible at all, that there'd be something I
> could put in the env.py which would reconfigure it to execute the
> `context.run_migrations()` migration context (and therefore search path,
> back at the original call site.
>
> I realize that this won't always work, given that env.py is often likely
> customized enough such that a generic one wouldn't be able to execute them,
> but per your cookbook suggestions about programmatic invocation of
> commands, this sort of thing requires the user to opt into changing their
> env.py to use
> connectable = context.config.attributes.get("connection", None)
> in order to make use of a connection handed to it.
>
>
> I'm not following all the moving pieces here, like when you say
> "commands", i assume you mean the commands in alembic.command, , like
>
Re: Invoke commands with custom env.py from library
On Thu, Mar 5, 2020, at 12:02 PM, Daniel Cardin wrote:
> So, yes I mean "commands" as in `alembic.command.upgrade()`.
>
> The idea would be that the library defines an env.py (e.g. the important
> portion of which would look something like:)
> ...
>
> connectable = context.config.attributes.get("connection", None)
> with self.connection.connect() as connection:
> alembic.context.configure(connection=connection)
>
> with alembic.context.begin_transaction():
> alembic.context.run_migrations()
>
> which lives at `src/library/foo/env.py`. If the migrations were colocated
> with the env.py, then afaik setting "script_location" to "library:foo" would
> enable me to run `alembic.command.upgrade()` and it would just work. However
> in this case (if possible), the migrations/versions/* would live at an
> arbitrary other location (e.g. the code which has installed this library).
Ok two thoughts on that:
1. usually the way this goes is that the project that has the migrations does
everything normally, it has its own env.py, and inside that env.py, that's
where it imports *your* env.py. that is, instaed of having the usual env.py it
would only have:
# env.py
from daniel_cardins.library.env import run_migrations
run_migrations()
2. the locations of the version files vs. the env.py script are actually
separate in the config. there is a script_location that it uses to find the
home base where env.py is, however there is also a version_locations config
that can have any number of other directories in it and these supersede
script_location for the version files themselves.
I think in general, if the end-user has a bunch of version files set up, it's
probably not a big deal that they have a stub "env.py" right there that just
calls out to your library. that's something very clear that people looking at a
project that uses your library can understand quickly if they already know
alembic.
>
> General workflow:
> * person working on project "foo" invokes some cli/command/whatever which
> requests an upgrade
> * library does whatever requisite setup
> * library invokes `alembic.command.upgrade()`
> * upgrade() ends up routing the code through the library's `env.py`
> * the context of the migration command is targeting the project "foo"'s local
> migrations/versions folder
>
> The specifics of the above are just based on my knowledge of alembic, so if
> there's another process i could be doing where env.py isn't "invoked" so much
> as the above code block is just called normally, then that's ideal.
>
> On Thursday, March 5, 2020 at 9:36:09 AM UTC-5, Mike Bayer wrote:
>>
>>
>> On Thu, Mar 5, 2020, at 8:08 AM, Daniel Cardin wrote:
>>> I am attempting to write a library which invokes alembic commands, while
>>> referencing the migrations of a separate package which has installed said
>>> library.
>>>
>>> The intent here, is for the library to invoke the alembic commands with an
>>> env.py defined in that package. This seems to work through
>>> config.get("script_location", "package_name:foldername")
>>> but then obviously expects the actual migrations to be colocated at the
>>> same location.
>>>
>>> My guess would be, if this is possible at all, that there'd be something I
>>> could put in the env.py which would reconfigure it to execute the
>>> `context.run_migrations()` migration context (and therefore search path,
>>> back at the original call site.
>>>
>>> I realize that this won't always work, given that env.py is often likely
>>> customized enough such that a generic one wouldn't be able to execute them,
>>> but per your cookbook suggestions about programmatic invocation of
>>> commands, this sort of thing requires the user to opt into changing their
>>> env.py to use
>>> connectable = context.config.attributes.get("connection", None)
>>> in order to make use of a connection handed to it.
>>
>> I'm not following all the moving pieces here, like when you say "commands",
>> i assume you mean the commands in alembic.command, , like
>> alembic.command.upgrade(). when you say "an env.py defined in that package",
>> I guess you mean in the separate package that is using your library.
>>
>> this doesn't seem different from what Alembic itself does, "that package"
>> would have a directory where the env.py and its migration files are present?
>> So... I'm not really sure what you're trying to do beyond call an Alembic
>> command that is against a particular directory.
>>
>> if OTOH by "commands" you mean migration operations, like you want to call
>> op.create_table() yourself, OK, but I don't really understand enough to
>> formulate an answer for you.
>>
>>
>>
>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "sqlalchemy-alembic" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to sqlalchemy-alembic+unsubscr...@googlegroups.com.
>>> To view this discussion on
Re: Invoke commands with custom env.py from library
So, yes I mean "commands" as in `alembic.command.upgrade()`.
The idea would be that the library defines an env.py (e.g. the important
portion of which would look something like:)
...
connectable = context.config.attributes.get("connection", None)
with self.connection.connect() as connection:
alembic.context.configure(connection=connection)
with alembic.context.begin_transaction():
alembic.context.run_migrations()
which lives at `src/library/foo/env.py`. If the migrations were colocated
with the env.py, then afaik setting "script_location" to "library:foo"
would enable me to run `alembic.command.upgrade()` and it would just work.
However in this case (if possible), the migrations/versions/* would live at
an arbitrary other location (e.g. the code which has installed this
library).
General workflow:
* person working on project "foo" invokes some cli/command/whatever which
requests an upgrade
* library does whatever requisite setup
* library invokes `alembic.command.upgrade()`
* upgrade() ends up routing the code through the library's `env.py`
* the context of the migration command is targeting the project "foo"'s
local migrations/versions folder
The specifics of the above are just based on my knowledge of alembic, so if
there's another process i could be doing where env.py isn't "invoked" so
much as the above code block is just called normally, then that's ideal.
On Thursday, March 5, 2020 at 9:36:09 AM UTC-5, Mike Bayer wrote:
>
>
>
> On Thu, Mar 5, 2020, at 8:08 AM, Daniel Cardin wrote:
>
> I am attempting to write a library which invokes alembic commands, while
> referencing the migrations of a separate package which has installed said
> library.
>
> The intent here, is for the library to invoke the alembic commands with an
> env.py defined in that package. This seems to work through
> config.get("script_location", "package_name:foldername")
> but then obviously expects the actual migrations to be colocated at the
> same location.
>
> My guess would be, if this is possible at all, that there'd be something I
> could put in the env.py which would reconfigure it to execute the
> `context.run_migrations()` migration context (and therefore search path,
> back at the original call site.
>
> I realize that this won't always work, given that env.py is often likely
> customized enough such that a generic one wouldn't be able to execute them,
> but per your cookbook suggestions about programmatic invocation of
> commands, this sort of thing requires the user to opt into changing their
> env.py to use
> connectable = context.config.attributes.get("connection", None)
> in order to make use of a connection handed to it.
>
>
> I'm not following all the moving pieces here, like when you say
> "commands", i assume you mean the commands in alembic.command, , like
> alembic.command.upgrade(). when you say "an env.py defined in that
> package", I guess you mean in the separate package that is using your
> library.
>
> this doesn't seem different from what Alembic itself does, "that package"
> would have a directory where the env.py and its migration files are
> present? So... I'm not really sure what you're trying to do beyond
> call an Alembic command that is against a particular directory.
>
> if OTOH by "commands" you mean migration operations, like you want to call
> op.create_table() yourself, OK, but I don't really understand enough to
> formulate an answer for you.
>
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy-alembic" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy-alembic+unsubscr...@googlegroups.com .
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sqlalchemy-alembic/5612f1d1-9e93-46ed-9be7-0db362b815a8%40googlegroups.com
>
>
> .
>
>
>
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy-alembic" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy-alembic+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/sqlalchemy-alembic/598f480c-e977-4ce2-a315-8563aa9944fc%40googlegroups.com.
Re: Invoke commands with custom env.py from library
On Thu, Mar 5, 2020, at 8:08 AM, Daniel Cardin wrote:
> I am attempting to write a library which invokes alembic commands, while
> referencing the migrations of a separate package which has installed said
> library.
>
> The intent here, is for the library to invoke the alembic commands with an
> env.py defined in that package. This seems to work through
> config.get("script_location", "package_name:foldername")
> but then obviously expects the actual migrations to be colocated at the same
> location.
>
> My guess would be, if this is possible at all, that there'd be something I
> could put in the env.py which would reconfigure it to execute the
> `context.run_migrations()` migration context (and therefore search path, back
> at the original call site.
>
> I realize that this won't always work, given that env.py is often likely
> customized enough such that a generic one wouldn't be able to execute them,
> but per your cookbook suggestions about programmatic invocation of commands,
> this sort of thing requires the user to opt into changing their env.py to use
> connectable = context.config.attributes.get("connection", None)
> in order to make use of a connection handed to it.
I'm not following all the moving pieces here, like when you say "commands", i
assume you mean the commands in alembic.command, , like
alembic.command.upgrade(). when you say "an env.py defined in that package", I
guess you mean in the separate package that is using your library.
this doesn't seem different from what Alembic itself does, "that package" would
have a directory where the env.py and its migration files are present? So...
I'm not really sure what you're trying to do beyond call an Alembic command
that is against a particular directory.
if OTOH by "commands" you mean migration operations, like you want to call
op.create_table() yourself, OK, but I don't really understand enough to
formulate an answer for you.
>
> --
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy-alembic" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy-alembic+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sqlalchemy-alembic/5612f1d1-9e93-46ed-9be7-0db362b815a8%40googlegroups.com
>
> .
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy-alembic" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy-alembic+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/sqlalchemy-alembic/68640cf0-9297-4a55-9c26-7c54d7edae62%40www.fastmail.com.
Invoke commands with custom env.py from library
I am attempting to write a library which invokes alembic commands, while
referencing the migrations of a separate package which has installed said
library.
The intent here, is for the library to invoke the alembic commands with an
env.py defined in that package. This seems to work through
config.get("script_location", "package_name:foldername")
but then obviously expects the actual migrations to be colocated at the
same location.
My guess would be, if this is possible at all, that there'd be something I
could put in the env.py which would reconfigure it to execute the
`context.run_migrations()` migration context (and therefore search path,
back at the original call site.
I realize that this won't always work, given that env.py is often likely
customized enough such that a generic one wouldn't be able to execute them,
but per your cookbook suggestions about programmatic invocation of
commands, this sort of thing requires the user to opt into changing their
env.py to use
connectable = context.config.attributes.get("connection", None)
in order to make use of a connection handed to it.
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy-alembic" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy-alembic+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/sqlalchemy-alembic/5612f1d1-9e93-46ed-9be7-0db362b815a8%40googlegroups.com.
