I'm currently working on moving on the MySQL for savanna-ci
On Wed, Feb 5, 2014 at 3:53 PM, Sergey Lukjanov
<slukja...@mirantis.com <mailto:slukja...@mirantis.com>> wrote:
Agreed, let's move on to the MySQL for savanna-ci to run
integration tests against production-like DB.
On Wed, Feb 5, 2014 at 1:54 AM, Andrew Lazarev
<alaza...@mirantis.com <mailto:alaza...@mirantis.com>> wrote:
Since sqlite is not in the list of "databases that would be
used in production", CI should use other DB for testing.
Andrew.
On Tue, Feb 4, 2014 at 1:13 PM, Alexander Ignatov
<aigna...@mirantis.com <mailto:aigna...@mirantis.com>> wrote:
Indeed. We should create a bug around that and move our
savanna-ci to mysql.
Regards,
Alexander Ignatov
On 05 Feb 2014, at 01:01, Trevor McKay <tmc...@redhat.com
<mailto:tmc...@redhat.com>> wrote:
> This brings up an interesting problem:
>
> In https://review.openstack.org/#/c/70420/ I've added a
migration that
> uses a drop column for an upgrade.
>
> But savann-ci is apparently using a sqlite database to
run. So it can't
> possibly pass.
>
> What do we do here? Shift savanna-ci tests to non sqlite?
>
> Trevor
>
> On Sat, 2014-02-01 at 18:17 +0200, Roman Podoliaka wrote:
>> Hi all,
>>
>> My two cents.
>>
>>> 2) Extend alembic so that op.drop_column() does the
right thing
>> We could, but should we?
>>
>> The only reason alembic doesn't support these
operations for SQLite
>> yet is that SQLite lacks proper support of ALTER
statement. For
>> sqlalchemy-migrate we've been providing a work-around
in the form of
>> recreating of a table and copying of all existing rows
(which is a
>> hack, really).
>>
>> But to be able to recreate a table, we first must have
its definition.
>> And we've been relying on SQLAlchemy schema reflection
facilities for
>> that. Unfortunately, this approach has a few drawbacks:
>>
>> 1) SQLAlchemy versions prior to 0.8.4 don't support
reflection of
>> unique constraints, which means the recreated table
won't have them;
>>
>> 2) special care must be taken in 'edge' cases (e.g.
when you want to
>> drop a BOOLEAN column, you must also drop the
corresponding CHECK (col
>> in (0, 1)) constraint manually, or SQLite will raise an
error when the
>> table is recreated without the column being dropped)
>>
>> 3) special care must be taken for 'custom' type columns
(it's got
>> better with SQLAlchemy 0.8.x, but e.g. in 0.7.x we had
to override
>> definitions of reflected BIGINT columns manually for each
>> column.drop() call)
>>
>> 4) schema reflection can't be performed when alembic
migrations are
>> run in 'offline' mode (without connecting to a DB)
>> ...
>> (probably something else I've forgotten)
>>
>> So it's totally doable, but, IMO, there is no real
benefit in
>> supporting running of schema migrations for SQLite.
>>
>>> ...attempts to drop schema generation based on models
in favor of migrations
>>
>> As long as we have a test that checks that the DB
schema obtained by
>> running of migration scripts is equal to the one
obtained by calling
>> metadata.create_all(), it's perfectly OK to use model
definitions to
>> generate the initial DB schema for running of
unit-tests as well as
>> for new installations of OpenStack (and this is
actually faster than
>> running of migration scripts). ... and if we have
strong objections
>> against doing metadata.create_all(), we can always use
migration
>> scripts for both new installations and upgrades for all
DB backends,
>> except SQLite.
>>
>> Thanks,
>> Roman
>>
>> On Sat, Feb 1, 2014 at 12:09 PM, Eugene Nikanorov
>> <enikano...@mirantis.com
<mailto:enikano...@mirantis.com>> wrote:
>>> Boris,
>>>
>>> Sorry for the offtopic.
>>> Is switching to model-based schema generation is
something decided? I see
>>> the opposite: attempts to drop schema generation based
on models in favor of
>>> migrations.
>>> Can you point to some discussion threads?
>>>
>>> Thanks,
>>> Eugene.
>>>
>>>
>>>
>>> On Sat, Feb 1, 2014 at 2:19 AM, Boris Pavlovic
<bpavlo...@mirantis.com <mailto:bpavlo...@mirantis.com>>
>>> wrote:
>>>>
>>>> Jay,
>>>>
>>>> Yep we shouldn't use migrations for sqlite at all.
>>>>
>>>> The major issue that we have now is that we are not
able to ensure that DB
>>>> schema created by migration & models are same
(actually they are not same).
>>>>
>>>> So before dropping support of migrations for sqlite &
switching to model
>>>> based created schema we should add tests that will
check that model &
>>>> migrations are synced.
>>>> (we are working on this)
>>>>
>>>>
>>>>
>>>> Best regards,
>>>> Boris Pavlovic
>>>>
>>>>
>>>> On Fri, Jan 31, 2014 at 7:31 PM, Andrew Lazarev
<alaza...@mirantis.com <mailto:alaza...@mirantis.com>>
>>>> wrote:
>>>>>
>>>>> Trevor,
>>>>>
>>>>> Such check could be useful on alembic side too. Good
opportunity for
>>>>> contribution.
>>>>>
>>>>> Andrew.
>>>>>
>>>>>
>>>>> On Fri, Jan 31, 2014 at 6:12 AM, Trevor McKay
<tmc...@redhat.com <mailto:tmc...@redhat.com>> wrote:
>>>>>>
>>>>>> Okay, I can accept that migrations shouldn't be
supported on sqlite.
>>>>>>
>>>>>> However, if that's the case then we need to fix up
savanna-db-manage so
>>>>>> that it checks the db connection info and throws a
polite error to the
>>>>>> user for attempted migrations on unsupported
platforms. For example:
>>>>>>
>>>>>> "Database migrations are not supported for sqlite"
>>>>>>
>>>>>> Because, as a developer, when I see a sql error
trace as the result of
>>>>>> an operation I assume it's broken :)
>>>>>>
>>>>>> Best,
>>>>>>
>>>>>> Trevor
>>>>>>
>>>>>> On Thu, 2014-01-30 at 15:04 -0500, Jay Pipes wrote:
>>>>>>> On Thu, 2014-01-30 at 14:51 -0500, Trevor McKay wrote:
>>>>>>>> I was playing with alembic migration and
discovered that
>>>>>>>> op.drop_column() doesn't work with sqlite. This
is because sqlite
>>>>>>>> doesn't support dropping a column (broken imho,
but that's another
>>>>>>>> discussion). Sqlite throws a syntax error.
>>>>>>>>
>>>>>>>> To make this work with sqlite, you have to copy
the table to a
>>>>>>>> temporary
>>>>>>>> excluding the column(s) you don't want and delete
the old one,
>>>>>>>> followed
>>>>>>>> by a rename of the new table.
>>>>>>>>
>>>>>>>> The existing 002 migration uses op.drop_column(),
so I'm assuming
>>>>>>>> it's
>>>>>>>> broken, too (I need to check what the migration
test is doing). I
>>>>>>>> was
>>>>>>>> working on an 003.
>>>>>>>>
>>>>>>>> How do we want to handle this? Three good
options I can think of:
>>>>>>>>
>>>>>>>> 1) don't support migrations for sqlite (I think
"no", but maybe)
>>>>>>>>
>>>>>>>> 2) Extend alembic so that op.drop_column() does
the right thing
>>>>>>>> (more
>>>>>>>> open-source contributions for us, yay :) )
>>>>>>>>
>>>>>>>> 3) Add our own wrapper in savanna so that we have
a drop_column()
>>>>>>>> method
>>>>>>>> that wraps copy/rename.
>>>>>>>>
>>>>>>>> Ideas, comments?
>>>>>>>
>>>>>>> Migrations should really not be run against SQLite
at all -- only on
>>>>>>> the
>>>>>>> databases that would be used in production. I
believe the general
>>>>>>> direction of the contributor community is to be
consistent around
>>>>>>> testing of migrations and to not run migrations at
all in unit tests
>>>>>>> (which use SQLite).
>>>>>>>
>>>>>>> Boris (cc'd) may have some more to say on this topic.
>>>>>>>
>>>>>>> Best,
>>>>>>> -jay
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> OpenStack-dev mailing list
>>>>>>> OpenStack-dev@lists.openstack.org
<mailto:OpenStack-dev@lists.openstack.org>
>>>>>>>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> OpenStack-dev mailing list
>>>>>> OpenStack-dev@lists.openstack.org
<mailto:OpenStack-dev@lists.openstack.org>
>>>>>>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev@lists.openstack.org
<mailto:OpenStack-dev@lists.openstack.org>
>>>>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev@lists.openstack.org
<mailto:OpenStack-dev@lists.openstack.org>
>>>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
<mailto:OpenStack-dev@lists.openstack.org>
>>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
<mailto:OpenStack-dev@lists.openstack.org>
>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
<mailto:OpenStack-dev@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
<mailto:OpenStack-dev@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Sincerely yours,
Sergey Lukjanov
Savanna Technical Lead
Mirantis Inc.
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
<mailto:OpenStack-dev@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
