Re: Do i want to use bases or branches or something else?
If you look at the pyramid-cookiecutter-starter [1] the integration modifies the env.py file to load the url from the app settings instead of the alembic section of the ini. This allows you to put a copy-pasta [alembic] section into the file which points at the migrations/slug format/etc and sources the url from elsewhere. Maybe someone else has something better, but that's what we're using right now. [1] https://github.com/Pylons/pyramid-cookiecutter-starter/blob/41a23afa7e05958f53100a31dfd8aaabaeab67e9/%7B%7Bcookiecutter.repo_name%7D%7D/%7B%7Bcookiecutter.repo_name%7D%7D/sqlalchemy_alembic/env.py On Wed, Feb 5, 2020 at 11:43 AM Jonathan Vanasco wrote: > I'd like to integrate Alembic into a project that is mostly driven by the > Pyramid framework. > > In my use-case, Pyramid is driven by two or more different configuration > files, such as: > > example_development.ini > example_production.ini > > The relevant difference between them is the sqlalchemy url - which can > point to different databases and even different database drivers. Then > there are the test systems. > > What is the most-correct/easiest/best way of integrating alembic in this > situation? > > Is it possible to somehow declare the sqlalchemy url in the my pyramid > .ini files, then have a single simple alembic file reference that? > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy-alembic" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sqlalchemy-alembic+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/sqlalchemy-alembic/9258400f-d23b-4c92-856e-cba8c74dacf6%40googlegroups.com > <https://groups.google.com/d/msgid/sqlalchemy-alembic/9258400f-d23b-4c92-856e-cba8c74dacf6%40googlegroups.com?utm_medium=email_source=footer> > . > -- Michael -- You received this message because you are subscribed to the Google Groups "sqlalchemy-alembic" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy-alembic/CAKdhhwESaod2-bfbS7As0r-S2hruio3JANPS74CB%3Dofx0VDaHg%40mail.gmail.com.
Re: merging old versions
I think the basic idea is to create a database and codebase in the state of the target revision. Then autogenerate a migration from nothing to that revision - just like you would do when starting to use alembic from an existing schema. From there you can change the slug on it so that it works as the down_revision of later migrations and clear out the old unused migrations that you're replacing. - Michael On Thu, Jun 20, 2019 at 2:37 AM Chris Withers wrote: > Hi All, > > I have some versions that make use of the third party package I no > longer use, how do I collapse down alembic revisions that have already > been executed everywhere? > > I found > > https://stackoverflow.com/questions/34491914/alembic-how-to-merge-all-revision-files-to-one-file > but that doesn't feel right... > > Chris > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy-alembic" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sqlalchemy-alembic+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/sqlalchemy-alembic/933cd6aa-0e35-8716-3725-56947157103b%40withers.org > . > For more options, visit https://groups.google.com/d/optout. > -- Michael -- You received this message because you are subscribed to the Google Groups "sqlalchemy-alembic" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy-alembic/CAKdhhwFSmC4yBgCuYSNDS%2BLXUSBUYbbE8tCnjQdYyp_NWfHo8Q%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: selective migration options (e.g. create and add only)
On Friday, June 9, 2017 at 9:00:00 AM UTC-4, ktang wrote: > > Hi, > > I have an application migrating existing mysql dbs which may have tables > only in some system. > I am trying to use alembic to handle the migration. > > When I migrate the db to a newer version, if I don't want to drop any > existing tables or columns, how can I do this (just adding stuff)? > I'm assuming you're talking about autogenerate. If you are just writing migration scripts by hand, you just put whatever commands you want in them and that does not impact anything else that's already in the database. If you are using autogenerate, and you are comparing a model that's only a subset of the real database, you can have autogenerate exclude other tables by writing an "include_object" function. Documentation for this is here: http://alembic.zzzcomputing.com/en/latest/api/runtime.html?highlight=include_object#alembic.runtime.environment.EnvironmentContext.configure.params.include_object . You probably want to look at objects of type "table" and include based on a list of names you care about. > > And a minor issue, when I first setup alembic in my application, I > generated an initial migration script which autoincrement is not set in > only one of the tables while all other tables are created correctly. > Is this a known issue? > not really, it depends on the specifics of how your Table metadata is set up. If it's just one table then there is likely something different about it in your code. -- You received this message because you are subscribed to the Google Groups "sqlalchemy-alembic" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Migration fails when changing to single table inheritance
Yes, I'll try Plan A first(do the INSERT FROM first) But mainly out of academic curiosity, if I were to start with a blank slate as you say, I'd still need to import the existing data that is currently in the non-inheritance table schema? So I guess I would just drop the database, upgrade to head. At this point I have the latest inheritance-based schema but no data. Say I dumped the db to a file before I dropped it, does alembic have facilities to take a dumped db and insert it into a new/slightly different schema? On Wednesday, June 7, 2017 at 7:12:27 PM UTC-4, mike bayer wrote: > > > > On 06/07/2017 07:01 PM, Michael wrote: > > Thanks for the insight Mike. I guess the best way to go about that would > > be to just call the raw insert sql statemen in the migration? like in > > > https://stackoverflow.com/questions/23206562/sqlalchemy-executing-raw-sql-with-parameter-bindings/23206636#23206636 > > > i'd start with that just to make sure it gets it working.From there, > we do support insert().from_select(select()) if you have the inclination > to build a Core SQL statement out of it but if you are only targeting > Postgresql it's not critical. > > > http://docs.sqlalchemy.org/en/latest/core/dml.html?highlight=insert%20from_select#sqlalchemy.sql.expression.Insert.from_select > > > > Since this app is not in production yet, would it be easier to make the > > current schema the initial schema and just insert the data that is in > > the MediaChapter table into the new schema just once? I guess could use > > bulk_insert() to do that? > > if the data is not there yet, sure it's easier to start with a blank > slate if that's what you mean > > > > > On Wednesday, June 7, 2017 at 6:14:39 PM UTC-4, mike bayer wrote: > > > > > > > > On 06/07/2017 04:44 PM, Michael wrote: > > > Hi all, I have a class called MediaChapter(Base), which I've > > refactored > > > into MediaBase(Base) and MediaChapter(MediaBase) When I run the > > > migration, I see: > > > > > > | > > > psycopg2.IntegrityError:insert orupdate on table > > "mediachapter"violates > > > foreign key constraint "fk_mediachapter_id_mediabase" > > > DETAIL:Key(id)=(570)isnotpresent intable "mediabase". > > > > > > here's the real error with the SQL: > > > > sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) insert or > > update on table "mediachapter" violates foreign key constraint > > "fk_mediachapter_id_mediabase" > > DETAIL: Key (id)=(570) is not present in table "mediabase". > >[SQL: 'ALTER TABLE mediachapter ADD CONSTRAINT > > fk_mediachapter_id_mediabase FOREIGN KEY(id) REFERENCES mediabase > > (id)'] > > > > the error means that your "mediachapter" table contains an id, > "570", > > which is not present in the "mediabase" table. > > > > it looks like you are starting with a populated "mediachapter" table > > then adding a new table "mediabase". Before you create the > constraint, > > you need to run an INSERT on "mediabase" that selects from > > "mediachapter", like: > > > > INSERT INTO mediabase (id, col1, col2, ...) SELECT id, col1, col2, > .. > > FROM mediachapter > > > > > > > -- You received this message because you are subscribed to the Google Groups "sqlalchemy-alembic" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Migration fails when changing to single table inheritance
Thanks for the insight Mike. I guess the best way to go about that would be to just call the raw insert sql statemen in the migration? like in https://stackoverflow.com/questions/23206562/sqlalchemy-executing-raw-sql-with-parameter-bindings/23206636#23206636 Since this app is not in production yet, would it be easier to make the current schema the initial schema and just insert the data that is in the MediaChapter table into the new schema just once? I guess could use bulk_insert() to do that? On Wednesday, June 7, 2017 at 6:14:39 PM UTC-4, mike bayer wrote: > > > > On 06/07/2017 04:44 PM, Michael wrote: > > Hi all, I have a class called MediaChapter(Base), which I've refactored > > into MediaBase(Base) and MediaChapter(MediaBase) When I run the > > migration, I see: > > > > | > > psycopg2.IntegrityError:insert orupdate on table "mediachapter"violates > > foreign key constraint "fk_mediachapter_id_mediabase" > > DETAIL:Key(id)=(570)isnotpresent intable "mediabase". > > > here's the real error with the SQL: > > sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) insert or > update on table "mediachapter" violates foreign key constraint > "fk_mediachapter_id_mediabase" > DETAIL: Key (id)=(570) is not present in table "mediabase". > [SQL: 'ALTER TABLE mediachapter ADD CONSTRAINT > fk_mediachapter_id_mediabase FOREIGN KEY(id) REFERENCES mediabase (id)'] > > the error means that your "mediachapter" table contains an id, "570", > which is not present in the "mediabase" table. > > it looks like you are starting with a populated "mediachapter" table > then adding a new table "mediabase". Before you create the constraint, > you need to run an INSERT on "mediabase" that selects from > "mediachapter", like: > > INSERT INTO mediabase (id, col1, col2, ...) SELECT id, col1, col2, .. > FROM mediachapter > > > > -- You received this message because you are subscribed to the Google Groups "sqlalchemy-alembic" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Migration fails when changing to single table inheritance
Hi all, I have a class called MediaChapter(Base), which I've refactored
into MediaBase(Base) and MediaChapter(MediaBase) When I run the migration,
I see:
psycopg2.IntegrityError: insert or update on table "mediachapter" violates
foreign key constraint "fk_mediachapter_id_mediabase"
DETAIL: Key (id)=(570) is not present in table "mediabase".
I'm not sure exactly what this means because I create a mediabase.id.
I did
ws-alembic -c kjvrvg/conf/development.ini -x packages=all revision --auto -m
"MediaBase-MediaChapter subclassing"
-- SUCCESS
then
ws-alembic -c kjvrvg/conf/development.ini -x packages=all upgrade head
-- FAILED
Here are my models, MediaBase:
class MediaBase(Base):
#: The table in the database
__tablename__ = "mediabase"
#: Database primary key for the row (running counter)
id = Column(Integer, autoincrement=True, primary_key=True)
# table inheritance
media_type = Column(String(32), nullable=False)
#: Publicly exposed non-guessable
uuid = Column(UUID(as_uuid=True), default=uuid4)
localizedname = Column(Unicode(128), default=None)
#: url
url = Column(Unicode(384), default=None)
# full iso language-locale identifier i.e. zh-Hans-US
language_id = Column(String(16), default=None)
# name of the person presenting the material
presenter_name = Column(Unicode(64), default=None)
source_material = Column(Unicode(128), default=None)
# table inheritance
__mapper_args__ = {'polymorphic_on': media_type}
def __repr__(self):
"""Shell and debugger presentation."""
return '{} ({}) {} <{}>'.format(self.localizedname, self.language_id
, str(self.uuid), self.url)
def __str__(self):
"""Python default and admin UI string presentation."""
return '{} ({}) presenter: {} source: {} <{}>'.format(self.
localizedname, self.language_id, self.presenter_name, self.source_material,
self.url)
and MediaChapter:
class MediaChapter(MediaBase):
#: The table in the database
__tablename__ = "mediachapter"
__mapper_args__ = {'polymorphic_identity': 'chapter'}
id = Column(Integer, ForeignKey('mediabase.id'), primary_key=True)
#: Which chapter this media is part of
chapter_id = Column(Integer, ForeignKey('chapter.id'))
chapter = relationship("Chapter", back_populates="mediachapter")
and finally here is the auto-generated migration. I put all files in a
github Gist.
https://gist.github.com/mazz/7d63e521316859f4ae852e5cea5d84eb
Any suggestions?
Mazz
"""MediaBase-MediaChapter subclassing
Revision ID: a00980918d75
Revises: e74ba4203098
Create Date: 2017-06-07 16:10:29.807437
"""
# revision identifiers, used by Alembic.
revision = 'a00980918d75'
down_revision = 'e74ba4203098'
branch_labels = None
depends_on = None
import datetime
import websauna.system.model.columns
from sqlalchemy.types import Text # Needed from proper creation of JSON
fields as Alembic inserts astext_type=Text() row
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects import postgresql
def upgrade():
# ### commands auto generated by Alembic - please adjust! ###
op.create_table('mediabase',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('media_type', sa.String(length=32), nullable=False),
sa.Column('uuid', postgresql.UUID(as_uuid=True), nullable=True),
sa.Column('localizedname', sa.Unicode(length=128), nullable=True),
sa.Column('url', sa.Unicode(length=384), nullable=True),
sa.Column('language_id', sa.String(length=16), nullable=True),
sa.Column('presenter_name', sa.Unicode(length=64), nullable=True),
sa.Column('source_material', sa.Unicode(length=128), nullable=True),
sa.PrimaryKeyConstraint('id', name=op.f('pk_mediabase'))
)
op.alter_column('group', 'created_at',
existing_type=postgresql.TIMESTAMP(timezone=True),
type_=websauna.system.model.columns.UTCDateTime(),
existing_nullable=True)
op.alter_column('group', 'updated_at',
existing_type=postgresql.TIMESTAMP(timezone=True),
type_=websauna.system.model.columns.UTCDateTime(),
existing_nullable=True)
op.create_foreign_key(op.f('fk_mediachapter_id_mediabase'),
'mediachapter', 'mediabase', ['id'], ['id'])
op.drop_column('mediachapter', 'source_material')
op.drop_column('mediachapter', 'presenter_name')
op.drop_column('mediachapter', 'localizedname')
op.drop_column('mediachapter', 'url')
op.drop_column('mediachapter', 'uuid')
op.drop_column('mediachapter', 'language_id')
op.alter_column('user_activation', 'created_at',
existing_type=postgresql.TIMESTAMP(timezone=True),
type_=websauna.system.model.columns.UTCDateTime(),
existing_nullable=True)
op.alter_column('user_activation', 'expires_at',
existing_type=postgresql.TIMESTAMP(timezone=True),
Re: Generate code to generate db
Thanks Mike, I will tinker around with your example. -- You received this message because you are subscribed to the Google Groups "sqlalchemy-alembic" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
SQLAlchemy 1.0.0b4 released
SQLAlchemy release 1.0.0b4 is now available. This release contains a handful of fixes and enhancements mostly to address issues that were reported by beta testers. Things are looking very good and it's hoped that *maybe* this will be the last beta, unless a host of new regressions are reported. In preparation for 1.0.0, production installations that haven't yet been tested in the 1.0 series should be making sure that their requirements files are capped at 0.9.99, to avoid surprise upgrades. Changelog for 1.0.0b4 is at: http://docs.sqlalchemy.org/en/latest/changelog/changelog_10.html#change-1.0.0b4 SQLAlchemy 1.0.0b4 is available on the Download Page at http://www.sqlalchemy.org/download.html. -- You received this message because you are subscribed to the Google Groups sqlalchemy-alembic group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Alembic 0.7.5 released
Hey all - Alembic 0.7.5 is now available. This release has a handful of bug fixes and some new features. Changelog is available at: http://alembic.readthedocs.org/en/latest/changelog.html#change-0.7.5 Download Alembic 0.7.5 at: https://pypi.python.org/pypi/alembic/0.7.5.post1 Note there is no “0.7.5” release, because I failed to upload the PGP file and Pypi has a new policy that filenames can never be used again once uploaded.So 0.7.5.post1 is 0.7.5. -- You received this message because you are subscribed to the Google Groups sqlalchemy-alembic group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Creating a gist index on a new colmn
the “GIST” identifier for USING is available using the “postgresql_using” keyword argument: http://docs.sqlalchemy.org/en/rel_0_9/dialects/postgresql.html#index-types Jay Payne lett...@gmail.com wrote: I've been looking through the docs and I cannot find out how to create a gist index on a geometry column that's I've added. Any ideas? Thanks --J -- You received this message because you are subscribed to the Google Groups sqlalchemy-alembic group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups sqlalchemy-alembic group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: stack trace with key error from: alembic revision --autogenerate -m first rev
Dewey Gaedcke de...@pathoz.com wrote:
Is it likely that this construct:
@event.listens_for(Column, before_parent_attach)
def attach(target, cls):
# cls is the table obj, not the declarative class
target.name = %s%s % (cls.name[0:4], target.name)
is interfering with Alembic??
yes.the above process will definitely break table reflection which is what
you are using when you make use of autogenerate.
I'm getting this stack trace and it changes if I disable the event listener
above.
I don't actually have any columns named cat_cat_sit_id so it looks like
that prefix is being double attached???
this is the table that’s being reflected from your database, first of all, so
you definitely don’t want to be changing .name in that case, because it is set
to exactly what the name is in the DB.
the order of steps is:
1. reflection loads the name of the column from the DB. In this case I’m going
to say it’s “cat_sit_id”. which makes sense because the scheme you’re going
for here is tablename_colname.
2. reflection creates a Column(name) object, name “sit_id”.
3. Column copies the “.name” field to that of “.key”.
4. reflection attaches the Column to the Table, I’m going to guess it’s named
“cat”.
5. your event fires off. Renames .name to “cat_cat_sit_id”. Does not change
the .key.
6. The column is placed into table.c with the key “sit_id”. So
table.c.sit_id.name == “cat_cat_sit_id”.
7. Alembic, assuming this reflected table is just a plain table reflected from
the DB, is making the slight misjudgment that .name and .key are the same, this
could be improved, but generally it’s looking at column.name and then in that
error, requesting that name from table.c.
if it were me, I’d probably limit the use of that event to Table objects that
you know are being created from programmatic code. Table accepts a parameter
“listeners” so that you can create events that are local to that Table as it is
being created:
http://docs.sqlalchemy.org/en/rel_0_9/core/metadata.html#sqlalchemy.schema.Table.params.listeners.
Or you can, in your event, look at table.metadata and make sure its the
“metadata” object that you know is associated with your setup (eg.
table.metadata is Base.metadata, something like that). Or put a flag in
table.info, or metadata.info (OK metadata.info is only in 1.0, but something
like that), and check that.
Thanks for any suggestions to fix this.
Dewey
INFO [alembic.autogenerate.compare] Detected added foreign key
(ctr_gem_id)(ctr_gem_id) on table cat_ctr_att
INFO [alembic.autogenerate.compare] Detected added foreign key
(dtw_id)(dtw_id) on table cat_ctr_att
INFO [alembic.autogenerate.compare] Detected added foreign key
(sit_id)(sit_id) on table cat_ctr_att
Traceback (most recent call last):
File /Users/dgaedcke/Virtualenvs/paysys/bin/alembic, line 9, in module
load_entry_point('alembic==0.7.2', 'console_scripts', 'alembic')()
File
/Users/dgaedcke/Virtualenvs/paysys/lib/python2.7/site-packages/alembic/config.py,
line 399, in main
CommandLine(prog=prog).main(argv=argv)
File
/Users/dgaedcke/Virtualenvs/paysys/lib/python2.7/site-packages/alembic/config.py,
line 393, in main
self.run_cmd(cfg, options)
File
/Users/dgaedcke/Virtualenvs/paysys/lib/python2.7/site-packages/alembic/config.py,
line 376, in run_cmd
**dict((k, getattr(options, k)) for k in kwarg)
File
/Users/dgaedcke/Virtualenvs/paysys/lib/python2.7/site-packages/alembic/command.py,
line 113, in revision
script.run_env()
File
/Users/dgaedcke/Virtualenvs/paysys/lib/python2.7/site-packages/alembic/script.py,
line 382, in run_env
util.load_python_file(self.dir, 'env.py')
File
/Users/dgaedcke/Virtualenvs/paysys/lib/python2.7/site-packages/alembic/util.py,
line 241, in load_python_file
module = load_module_py(module_id, path)
File
/Users/dgaedcke/Virtualenvs/paysys/lib/python2.7/site-packages/alembic/compat.py,
line 79, in load_module_py
mod = imp.load_source(module_id, path, fp)
File alembic/env.py, line 74, in module
run_migrations_online()
File alembic/env.py, line 67, in run_migrations_online
context.run_migrations()
File string, line 7, in run_migrations
File
/Users/dgaedcke/Virtualenvs/paysys/lib/python2.7/site-packages/alembic/environment.py,
line 742, in run_migrations
self.get_context().run_migrations(**kw)
File
/Users/dgaedcke/Virtualenvs/paysys/lib/python2.7/site-packages/alembic/migration.py,
line 296, in run_migrations
for step in self._migrations_fn(heads, self):
File
/Users/dgaedcke/Virtualenvs/paysys/lib/python2.7/site-packages/alembic/command.py,
line 95, in retrieve_migrations
autogen._produce_migration_diffs(context, template_args, imports)
File
/Users/dgaedcke/Virtualenvs/paysys/lib/python2.7/site-packages/alembic/autogenerate/api.py,
Re: stack trace with key error from: alembic revision --autogenerate -m first rev
Michael Bayer mike...@zzzcomputing.com wrote: Dewey Gaedcke de...@pathoz.com wrote: this is the table that’s being reflected from your database, first of all, so you definitely don’t want to be changing .name in that case, because it is set to exactly what the name is in the DB. the order of steps is: 1. reflection loads the name of the column from the DB. In this case I’m going to say it’s “cat_sit_id”. which makes sense because the scheme you’re going for here is tablename_colname. 2. reflection creates a Column(name) object, name “sit_id”. grr, “cat_sit_id” 3. Column copies the “.name” field to that of “.key”. 4. reflection attaches the Column to the Table, I’m going to guess it’s named “cat”. 5. your event fires off. Renames .name to “cat_cat_sit_id”. Does not change the .key. 6. The column is placed into table.c with the key “sit_id”. So table.c.sit_id.name == “cat_cat_sit_id”. grr, table.c.cat_sit_id.name == “cat_cat_sit_id” 7. Alembic, assuming this reflected table is just a plain table reflected from the DB, is making the slight misjudgment that .name and .key are the same, this could be improved, but generally it’s looking at column.name and then in that error, requesting that name from table.c. if it were me, I’d probably limit the use of that event to Table objects that you know are being created from programmatic code. Table accepts a parameter “listeners” so that you can create events that are local to that Table as it is being created: http://docs.sqlalchemy.org/en/rel_0_9/core/metadata.html#sqlalchemy.schema.Table.params.listeners. Or you can, in your event, look at table.metadata and make sure its the “metadata” object that you know is associated with your setup (eg. table.metadata is Base.metadata, something like that). Or put a flag in table.info, or metadata.info (OK metadata.info is only in 1.0, but something like that), and check that. -- You received this message because you are subscribed to the Google Groups sqlalchemy-alembic group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Using a 3rd Party Dialect
alembic doesn’t have a formally published extension API as of yet, to get that
to work you’d need to make an Impl:
from alembic.ddl.postgresql import PostgresqlImpl
class RedshiftImpl(PostgresqlImpl):
__dialect__ = ‘redshift’
that will register the name “redshift” into alembic’s lookup.
On Oct 31, 2014, at 4:34 PM, Daniel Son daniel@knewton.com wrote:
Hi,
I'm new to SQLalchemy and alembic and I'm doing a proof of concept for
migrations in Redshift.
Initially, using the regular postgresql dialect everything is fine.
But when I try to use the redshift_sqlalchemy dialect for the last bit of
Redshift functionality, it goes caput.
And I've looked over SQLAlchemy and Alembic docs for any help on using 3rd
Party dialects and I can't seem to get anywhere.
Environment:
The basic setup as told by the Tutorial
The following are installed in a virtualenv:
pip install alembic
pip install psycopg2
pip install redshift_sqlalchemy
I've tried to manually register the dialect:
from sqlalchemy.dialects import registry
registry.register(redshift, redshift_sqlalchemy.dialect,
RedshiftDialect)
registry.register(redshift.psycopg2, redshift_sqlalchemy.dialect,
RedshiftDialect)
And tried the following url setups in my alembic.ini (via alembic init)
sqlalchemy.url = redshift+psycopg2://user:pass@host:port/dbname
sqlalchemy.url = redshift://user:pass@host:port/dbname
Both error out the same as this trace out.
Traceback (most recent call last):
File /opt/virtualenvs/alembic/bin/alembic, line 9, in module
load_entry_point('alembic==0.6.7', 'console_scripts', 'alembic')()
File
/opt/virtualenvs/alembic/lib/python2.7/site-packages/alembic/config.py,
line 306, in main
CommandLine(prog=prog).main(argv=argv)
File
/opt/virtualenvs/alembic/lib/python2.7/site-packages/alembic/config.py,
line 300, in main
self.run_cmd(cfg, options)
File
/opt/virtualenvs/alembic/lib/python2.7/site-packages/alembic/config.py,
line 286, in run_cmd
**dict((k, getattr(options, k)) for k in kwarg)
File
/opt/virtualenvs/alembic/lib/python2.7/site-packages/alembic/command.py,
line 129, in upgrade
script.run_env()
File
/opt/virtualenvs/alembic/lib/python2.7/site-packages/alembic/script.py,
line 208, in run_env
util.load_python_file(self.dir, 'env.py')
File
/opt/virtualenvs/alembic/lib/python2.7/site-packages/alembic/util.py, line
230, in load_python_file
module = load_module_py(module_id, path)
File
/opt/virtualenvs/alembic/lib/python2.7/site-packages/alembic/compat.py,
line 63, in load_module_py
mod = imp.load_source(module_id, path, fp)
File dson_test/env.py, line 92, in module
run_migrations_online()
File dson_test/env.py, line 80, in run_migrations_online
target_metadata=target_metadata
File string, line 7, in configure
File
/opt/virtualenvs/alembic/lib/python2.7/site-packages/alembic/environment.py,
line 672, in configure
opts=opts
File
/opt/virtualenvs/alembic/lib/python2.7/site-packages/alembic/migration.py,
line 158, in configure
return MigrationContext(dialect, connection, opts, environment_context)
File
/opt/virtualenvs/alembic/lib/python2.7/site-packages/alembic/migration.py,
line 103, in __init__
self.impl = ddl.DefaultImpl.get_by_dialect(dialect)(
File
/opt/virtualenvs/alembic/lib/python2.7/site-packages/alembic/ddl/impl.py,
line 55, in get_by_dialect
return _impls[dialect.name]
KeyError: 'redshift'
What am I missing?
Thanks,
Daniel
--
You received this message because you are subscribed to the Google Groups
sqlalchemy-alembic group.
To unsubscribe from this group and stop receiving emails from it, send an
email to sqlalchemy-alembic+unsubscr...@googlegroups.com
mailto:sqlalchemy-alembic+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout
https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy-alembic group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy-alembic+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: MySQL 1071 Error when using alembic autogenerate from SQLAlchemy models
the first hit on google for this is a stack overflow answer that points to the relevant details for this message: http://stackoverflow.com/questions/1814532/1071-specified-key-was-too-long-max-key-length-is-767-bytes the size of the index is being impacted by the character encoding in use; see the links above for more details/workarounds. On Oct 8, 2014, at 1:32 PM, Lance Blais la...@eventmobi.com wrote: I have an existing Flask based Python 3.3 project and I'm trying to set it up to use Alembic. I've followed the instructions and modified env.py as directed in various places. When I run `alembic revision --autogenerate -m 'test' `, I get a proper list of all my models. After that, I run `alembic upgrade head` and I get the following MySQL error: sqlalchemy.exc.InternalError: (InternalError) (1071, 'Specified key was too long; max key length is 767 bytes') b'\nCREATE TABLE hashed_links (\n\t`key` VARCHAR(256) NOT NULL, \n\tlink TEXT NOT NULL, \n\tPRIMARY KEY (`key`)\n)\n\n' () Can anyone tell me if there's something I've misconfigured or what to do to look deeper into this issue? The database is MySQL 5.6, the default engine is InnoDB and I've ensured that all permissions are correctly set. The database itself is fresh (drop database; create database) whenever I try something new, so I'm hoping that gets all the trivial debug steps out of the way. Thank you! -- You received this message because you are subscribed to the Google Groups sqlalchemy-alembic group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups sqlalchemy-alembic group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Alembic 0.6.6 released
Alembic 0.6.6 Released This is a bug fix release. For a list of changes please see: http://alembic.readthedocs.org/en/latest/changelog.html#change-0.6.6 As always, download Alembic on pypi at: https://pypi.python.org/pypi/alembic/ -- You received this message because you are subscribed to the Google Groups sqlalchemy-alembic group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: TypeError on updating DB with new date
Thanks, am using SQLite3. I think it's better to set everything to offset-naive too. I did so by doing this: rss.py: class RssFeed(Base): ... latestpost = Column(DateTime) tasks.py: pub_time = parse(post.published) *pub_time_naive = pub_time.replace(tzinfo=None)* And things seem to take just fine now. On Saturday, August 2, 2014 10:33:43 PM UTC-4, Michael Bayer wrote: On Aug 2, 2014, at 5:00 PM, Michael taoma...@gmail.com javascript: wrote: *TypeError: can't compare offset-naive and offset-aware datetimes* I'm comparing the date of the rss feed items published date with the latestpost of the rss feed. I ensure that the dates generated have pytz.UTC() added. Note the actual issue happens in line 61 of fetch_rss_posts task. Here is the py source of the tasks: http://pastebin.com/FPNvHLp4 note on line 61 I'm simply doing feed.latestpost = utc_pub_time I generate latestpost in an alembic migration script, of which I'm sure I'm adding offset information with pytz.UTC() here is how I generate the RssFeed member, latest_post on line 32: http://pastebin.com/eN2kipVs Here is some logging: http://pastebin.com/Vb31bCP6 Another interesting thing is, the issue doesn't actually occur until after the rss feed completely parsed. I don’t see information here on what database backend this is but to my knowledge only the postgresql backend has any ability at all to deal with offset-aware datetime objects, and you’d need to ensure timezone=True on the Python side and “TIMESTAMP WITH TIMEZONE” on the DB side. Within the flush here, using echo=‘debug’ will show what kinds of values are being sent out as well as what kind are being returned from the Python DBAPI. Typically the date times coming back as result sets don’t have timezone information added. I’m not sure what psycopg2’s behavior is here, you might need a custom type object that adds the tz info appropriately. (if it were me, I’d enforce all offset-naive UTC datetimes everywhere). -- You received this message because you are subscribed to the Google Groups sqlalchemy-alembic group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: how to handle exceptions in alembic migrations?
On Jun 8, 2014, at 9:12 AM, Ofir Herzas herz...@gmail.com wrote:
I have an alembic migration script and I want to add some exception handling
but not sure what is the best practice.
Basically, I have several issues to handle:
A change was already made and not needed (e.g. if I try to add_column, and
this column already exists, I want it to continue)
A table is locked (if I try to perform some operation on a table and it is
locked, I want to raise an exception)
other exceptions?
def upgrade():
engine = op.get_bind().engine
op.add_column('t_break_employee', sa.Column('auto', sa.Boolean()))
op.add_column('t_employee', sa.Column('settings', sa.Text()))
I thought about adding a class to be used with the 'with' statement on every
change. Does it sound reasonable?
for example:
def upgrade():
engine = op.get_bind().engine
with my_test():
op.add_column('t_break_employee', sa.Column('auto', sa.Boolean()))
with my_test():
op.add_column('t_employee', sa.Column('settings', sa.Text()))
In that case, what are the exceptions I need to handle and how do I know if a
table is locked?
seems a little risky, if your application wants to add a column, but someone
else added it in some other way, how can you be sure what else has been done to
that DB?Skipping individual ops might address small issues but not big
ones. If you really have a workflow where the target DB might have manual
changes applied, maybe instead you want to check for the existence of the
column?
def upgrade():
inspector = inspect(engine)
cols = inspector.get_columns(t_break_employee)
if auto not in [c['name'] for c in cols]:
op.add_column(...)
just a thought.
as far as table is locked, if it's persistently locked then your script would
just hang, just like every other application trying to get to that table, until
the deadlock elsewhere is resolved.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy-alembic group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy-alembic+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: fix pull request for fk constraint rendering with metadata schema
the fix looks fine and we use git on both bitbucket and github. Will try to test it out soon, thanks! On Jun 4, 2014, at 6:05 AM, Andreas Zeidler a...@zitc.de wrote: hi, i ran into an error while using a metadata schema and tried to come up with a fix: https://github.com/zzzeek/alembic/pull/14 i’m not all too familiar with mercurial, so i hope it’s acceptable to also use git(hub) for this… :) cheers, andi -- pgp key at http://zitc.de/pgp - http://wwwkeys.de.pgp.net/ upgrade to plone 4! -- http://plone.org/4 signature.asc Description: Message signed with OpenPGP using GPGMail
Re: DBSession.add() has no effect in tweepy callback in pyramid 1.5
Presumably you do not have the pyramid_tm tween active in your pyramid
configuration which will perform commits at the end of requests:
config.include('pyramid_tm')
If you are running a script, you'd want to do your database actions inside
of a tm block:
with transaction.manager:
...
On Sun, Jun 1, 2014 at 10:27 PM, Michael taomaili...@gmail.com wrote:
Hi all,
I'm trying to store tweets into an sqlite db and when I call
DBSession.add() to store it, it seems to do nothing.
Here is a trace of me setting up the db:
http://pastebin.com/tB4KLLXj
Here is how I set up the model in __init__.py
http://pastebin.com/DVS78C5b
Here is the Tweet class:
http://pastebin.com/nt2zPWrU
Here is the pyramid view and tweepy class that initiates the stream
listening and DBSession.add() call. I was going to use celery but want to
get the db call working first..
http://pastebin.com/yH2krrfE
finally, a log of the code running but missing the SQLite log success
messages. I've confirmed that the .sqlite file doesn't contain the new data.
http://pastebin.com/kJwizsbS
Any ideas why there is no successful db call?
--
You received this message because you are subscribed to the Google Groups
sqlalchemy-alembic group.
To unsubscribe from this group and stop receiving emails from it, send an
email to sqlalchemy-alembic+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy-alembic group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy-alembic+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Using alembic in a plugin pattern
On May 11, 2014, at 9:03 AM, Steeve C steevechaill...@gmail.com wrote: hi, I'm looking for writing plugins (stevedore) for my application, all plugins are new python modules and can create some entry|columns|table in the application database, I want to manage migration by plugins|modules is it possible using alembic? is there some documentation on that purpose? Currently, a single Alembic directory maintains a linear list of migrations, that is, A, B, C, D in a sequential pattern. So if the idea here is that you have multiple, independent apps with their own migration streams, at the moment a workaround is to maintain separate Alembic directories per app, and then use different version_table entries for each. This is a little verbose but it can be done using different named sections in alembic.ini, e.g.: [alembic_app1] version_table = app1_migration_version script_location = path/to/app1_migrations/ [alembic_app2] version_table = app2_migration_version script_location = path/to/app2_migrations/ ... then in env.py: context.configure( connection=connection, target_metadata=target_metadata, version_table=config.get_main_option(version_table) ) then when you run alembic: alembic upgrade head --name alembic_app2 For now, that's it. But later, a better approach will be available, when we will add support for multiple version directories and multiple independent branches. I hope to have funded support for these features within the next six months, see https://bitbucket.org/zzzeek/alembic/issues?status=newstatus=openmilestone=tier%201. -- You received this message because you are subscribed to the Google Groups sqlalchemy-alembic group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Alembic 0.6.4 released
Hi list - Alembic 0.6.4 is released. This release has a bunch of fixes that have been piling up; in particular, a whole bunch of fixes to work with the new naming_convention feature in SQLAlchemy 0.9, and ever more fixes regarding autogenerate with indexes and unique constraints. As always, the release is at https://pypi.python.org/pypi/alembic . Changelog is at: https://alembic.readthedocs.org/en/latest/changelog.html#change-0.6.4 . -- You received this message because you are subscribed to the Google Groups sqlalchemy-alembic group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Delete class in model and downgrade, bulk_insert
On Mar 15, 2014, at 10:29 PM, Tom Haulitschke
thomas.haulitsc...@googlemail.com wrote:
I really have problems with the downgrade() part:
def downgrade():
### commands auto generated by Alembic - please adjust! ###
dog_table = op.create_table('dog',
sa.Column('id', sa.INTEGER(), primary_key=True,
autoincrement=True),
create_table() currently does not have a return value. So you need to just
create a regular Table object, emit dog_table.create(op.get_bind()), then use
that Table for your insert(). So the first call you bulk_insert() is failing,
because dog_table is None:
op.bulk_insert(dog_table, new_dogs)
then for some reason, you're putting that in a try/except and then calling
bulk_insert again with a string object, which is also not how bulk_insert
works, so that fails too:
op.bulk_insert('dog', new_dogs)
So, create a Table, use that, make sure you're sending the right kinds of
objects to bulk_insert().
--
You received this message because you are subscribed to the Google Groups
sqlalchemy-alembic group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy-alembic+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: pyc files left in version folder create ghost history
well, for most releases we only read the .py files, until someone reported it as a bug that we *don't* read the .pyc files. because that person is doing sourceless installs. this is issue 163: https://bitbucket.org/zzzeek/alembic/issue/163 so in 623c7e76ef04c5656 you now need to add the flag sourceless=true to alembic.ini for this behavior to take effect. On Mar 14, 2014, at 1:51 AM, Marco Falcioni marcofalci...@gmail.com wrote: Hi, Recently I found out that pyc files left in the version folder are loaded by alembic when computing the history. This happened after some trial and error and git rm business. Alembic thought that I had a branch and refused to upgrade. I have reproduced the basic behavior in the archive below. (Python 2.7.2, Mac OS X) https://dl.dropboxusercontent.com/u/1980224/alembic_bug.tar.gz Cheers! Marco -- You received this message because you are subscribed to the Google Groups sqlalchemy-alembic group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups sqlalchemy-alembic group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Enum name gets lost.
Sorry, I should have noted the versions earlier.
SQLAlchemy: 0.7.4
Alembic: 0.6.3
Python: 2.7.3
Mike
On Tuesday, February 4, 2014 1:56:57 PM UTC, Michael Mulqueen wrote:
I'm hoping someone can help with this. My code is all built on SA's
declarative base.
What's happening is that from my code:
disposition = Column(Enum(Release, Rework, name=disposition_enum))
Alembic autogenerate is producing this:
sa.Column('disposition', sa.Enum('Release', 'Rework'), nullable=True),
The name is getting lost somewhere and this makes the upgrade fail because
Postgres requires a name. This is the first revision, so no altering is
going on, just creating new tables etc.
Does anyone have any ideas? Am I doing something wrong? I could go and
manually set the names in the generated code, but that kind of defeats the
point.
Thanks,
Mik
--
You received this message because you are subscribed to the Google Groups
sqlalchemy-alembic group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy-alembic+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Alembic 0.6.3 released
Hey lists - Alembic 0.6.3 is now available. 0.6.3 has a handful of bug fixes, the vast majority geared towards the autogenerate feature, including getting schemas to work with index and unique constraint detection as well as improved handling of schema attributes on Table objects within autogenerate. We’ve also added support for a so-called “sourceless” mode of usage, for the case where you’re distributing a software package that has only .pyc or .pyo files. Not something I’m too interested in myself but there you go. Changelog is at: http://alembic.readthedocs.org/en/latest/changelog.html#change-0.6.3 Alembic 0.6.3 is available on pypi: https://pypi.python.org/pypi/alembic signature.asc Description: Message signed with OpenPGP using GPGMail
