[sqlalchemy] Re: running parallel migrations using sharded/partioned/spaced queries?

Crap. That's way simpler. Thanks! I could even use Postgres as the orchestrator -- just create a table of ids and loop through those based on migration status. (new table would avoid the re-write on a full record from Postgres). -- You received this message because you are subscribed to the

[sqlalchemy] running parallel migrations using sharded/partioned/spaced queries?

I have to run a script on 2MM objects to update the database. Not really a schema migration, more like changing the internal data representation in the fields. There's a bit of post-processing and bottlenecks involved, so doing everything one-at-a-time will take a few days. I'd like to split

[sqlalchemy] Re: Exception in logger while using query options

BTW: sqlalchemy 1.0.8, python 3.3.4 On Wednesday, October 14, 2015 at 12:34:04 PM UTC+3, Ofir Herzas wrote: > > An exception pertaining to the logger is raised when using query.options > > Here is the relevant part of the exception: > > > query =

[sqlalchemy] Exception in logger while using query options

An exception pertaining to the logger is raised when using query.options Here is the relevant part of the exception: query = query.options(sqlalchemy.orm.load_only(*properties)) File "/opt/enigmai/ve/python-3.3.4/lib/python3.3/site-packages/sqlalchemy/orm/query.py", line 1100, in

Re: [sqlalchemy] Problem with hybrid_property

Works for me: Python 2.7.10 (default, Aug 22 2015, 20:33:39) [GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.1)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from modelosCaja import * >>> c = se.query(Caja).get(1) >>> c

Re: [sqlalchemy] running parallel migrations using sharded/partioned/spaced queries?

On 10/14/15 12:55 PM, jason kirtland wrote: > If you can partition the rows numerically, this is trivially easily to > implement using redis as the orchestrator. > > For example if you have integer PKs, you might have a loop like: > > offset = 0 > while offset < tablesize: >

Re: [sqlalchemy] Problem with hybrid_property

Yes this is the result of printing: c = se.query(Caja).get(1) c.__dict__ {'Entidad': u'Eprueba', '_sa_instance_state': , 'Salida': 10.0, 'Fecha': datetime.date(2015, 10, 12), 'id_Conceptos': 1, 'Grupo': u'grp', 'id': 1, 'Entrada': 50.0} type(c) type(c).__dict__ dict_proxy({'Entidad': ,

Re: [sqlalchemy] Exception in logger while using query options

On 10/14/15 5:34 AM, Ofir Herzas wrote: > An exception pertaining to the logger is raised when using query.options > > Here is the relevant part of the exception: issue 3539 fixed for 1.0.9: https://bitbucket.org/zzzeek/sqlalchemy/issues/3539 > > > query =

Re: 'AutogenContext' object has no attribute '__getitem__'

Does it no longer support the autogen_context['imports'] list, like here? http://alembic.readthedocs.org/en/latest/autogenerate.html#affecting-the-rendering-of-types-themselves On Wed, Oct 14, 2015 at 6:57 PM, Mike Bayer wrote: > > > On 10/14/15 12:41 AM, Fayaz Yusuf

Re: [sqlalchemy] running parallel migrations using sharded/partioned/spaced queries?

If you can partition the rows numerically, this is trivially easily to implement using redis as the orchestrator. For example if you have integer PKs, you might have a loop like: offset = 0 while offset < tablesize: for row in query[offset:batchsize]: migrate(row)

Re: 'AutogenContext' object has no attribute '__getitem__'

On 10/14/15 12:41 AM, Fayaz Yusuf Khan wrote: > Hi, > Looks like the autogen_context parameter has changed type recently. indeed it has: http://alembic.readthedocs.org/en/latest/api/autogenerate.html#alembic.autogenerate.api.AutogenContext > > I'm getting this exception: > > Traceback