Hi,

Well, seems that your deployment process needs fixing.

Simplest option is to stop serving site (put it to maintenance mode to show for example static maintenance page), update codebase and run migrations. Restart servers and put django serving back.

If you're looking zero downtime migrations you have to plan model modifications very carefully.

Here is pretty nice article how it could be done:
http://lucasroesler.com/2017/02/zero-downtime-deploys-a-tale-of-django-migrations/


On 06.07.2017 04:14, tay...@cedar.com wrote:
I am sorry everyone. I guess I am not very good at explaining the problem. I will try explain it better. Please ignore everything I have written before because it seems to be confusing.

I have perfectly working code in production. Later, I want to make a model change to remove a field and update the code to not use the field. I use makemigrations to create a migration with a RemoveField operation. When it comes time to release my new code, if I run the migration first in production (before pushing the code), for a few seconds I will get errors in queries. The old code will try to query the deleted field and will error.

Your immediate response may be "just run the migration after the code release". Well that could work if there are only RemoveField operations. If there is a AddField operation somewhere, the migrations must be run before the code release. So there is the dilemma. Depending upon the operation in the migration, a migration
may need to be run before or after.

My solution would be to always run migrations before a release, but all migrations must be backwards compatible. I am looking for a way to remove fields from the model, but keep the field in the DB, so the old code doesn't break.


On Wednesday, July 5, 2017 at 3:11:17 PM UTC-4, Alceu Rodrigues de Freitas Junior wrote:

    It seems to me more a organization/project management isssue that
    something that should be handled by Django itself. Well, if I got
    the problem correctly, that is.

    It seems you can't remove the field from the Model class because
    that would remove the column from the underline table itself as
    soon as the migration is executed.

    If your developers are doing development on their own computers
    (like they should), it would be a matter to leave them working on
    a different branch of the SCM where you're not applying that.
    Later is just a matter to provide comunication that they should
    update their own local repositories after you finished the merge
    process.

    In the case you need to keep the column around for a while, you
    might want to add some abstraction, like a view with a harcoded
    value for that column.


    Em 05/07/2017 11:19, tay...@cedar.com <javascript:> escreveu:
    Avraham,
    Thanks for your answer again. When you say planning for the long
    term is not ideal, what do you mean? I am interested in learning
    better processes. I do currently work with 20 developers, if that
    is what you meant.

    Your approach with a property was my first attempt. That didn't
    work because Django still queries for the column by default,
    unless every single queryset in the project we put
    .only(<fields>) excluding the deprecated columns, which can
    introduce other issues if we add new fields. I tried overriding
    get_queryset to do that globally on the model, but Django doesn't
    respect .only when doing inserts, only updates and retrieves.

    We do have very good test coverage, so warnings would be a great
    option if I could find a way to have Django not try to write or
    query that field by default.

    In regards to running migrations after deploying, that would be
    very difficult. Some migrations need to be run before a code
    release (adding fields), so people would have to be very careful
    not to put an AddField and a RemoveField in the same code
    release. Even if that was possible, we don't release manually (we
    use continuous deployment), so we would have to write some kind
    of migration reader that could determine whether to run a
    migration before or after the release.

    With your question about temporary, it may or may not be
    temporary, it depends on the field. Certain fields can later be
    removed safely, but there will be cases where we keep deprecated
    fields forever for reporting reasons, etc.


    On Wednesday, July 5, 2017 at 8:41:11 AM UTC-4, Avraham Serour
    wrote:

        From what you are describing it seems that you are planning
        for the long term. which I don't believe is ideal
        You may mark the whole model as not managed, but I don't
        think you can mark just one field and unmanaged.

        Another simple approach would be to transform the attribute
        to a property and print deprecation warnings or raise an
        exception whenever someone tries to write to it (or read)

        I hope that if you have tests it will be enough to remove all
        use of the deprecated field

        Is that temporary? That would influence the approach
        From what I understood from your original post this is just a
        matter of deploying the new app version that removes a field,
        so you would just need to run migrations after deploying


        On Wed, Jul 5, 2017 at 3:18 PM, Jani Tiainen
        <red...@gmail.com> wrote:

            Hi,

            Sounds like your all developers do use same database if
            you have such a problems.

            It's usually good practice to have per developer
            development database. That will allow individual
            developers to do changes to database and migrate others
            as they please. Also it doesn't "matter" if one developer
            breaks their database for example by accidentally running
            migrations that are not in the repo yet.

            Of course, it requires that you have either database
            creation script, or like we do, we clone our staging
            database for development basis.


            On 05.07.2017 15:09, tay...@cedar.com wrote:
            Thanks for responding Avraham.

            That would be a good option if I was developing by
            myself, but I am working with a team of 20 developers.
            The process needs to be the same whether there are
            deprecated fields or not. I can't realistically expect
            20 people to not apply one migration (or a few specific
            ones). There may be other migrations after the
            deprecation that need to be applied, so it's very
            difficult to apply certain ones and ignore others. It
            would be similarly difficult to get everyone to apply
            one of the migrations with --fake, but do a different
            process with every other migration. Also, --fake applies
            to the entire migration (not just specific operations),
            so people would be forced to make separate migrations
            for other model changes and hopefully remember not to
            run --fake on those.

            My current best attempt was to create a custom DB
            migration operation
            
https://docs.djangoproject.com/en/1.11/ref/migration-operations/#writing-your-own
            
<https://docs.djangoproject.com/en/1.11/ref/migration-operations/#writing-your-own>
            , that removes the field in state_forwards, but doesn't
            do anything to the db in database_forwards. That works,
            but the person that deprecates the field need to
            remember to change the RemoveField operation into the
            custom DeprecateField operation. It would be great if
            makemigrations created the correct operations
            automatically. Is there a way to do that?

            On Wednesday, July 5, 2017 at 7:27:22 AM UTC-4, Avraham
            Serour wrote:

                you can remove the field and don't run migrations
                until you are ready to actually remove the column,
                or you may run migrations fake and leave the column
                there forever

                
https://docs.djangoproject.com/en/1.11/ref/django-admin/#cmdoption-migrate-fake
                
<https://docs.djangoproject.com/en/1.11/ref/django-admin/#cmdoption-migrate-fake>

                On Wed, Jul 5, 2017 at 6:39 AM, <tay...@cedar.com>
                wrote:

                    I am having some trouble figuring out the best
                    way to remove model fields in Django. If I
                    remove a field from a model and then run
                    makemigrations, it creates a RemoveField
                    operation in a migration. That is great, but if
                    I decide to run the migration before releasing
                    the new code, the existing code will break (for
                    a short time between running the migration and
                    releasing the new code) because the old code is
                    still querying for the removed column (Django
                    queries for all columns by default). I could run
                    the migration after the release, but that won't
                    work if I also have an AddField operation
                    because the new code needs the new column, so it
                    needs to be run before. I am wondering if anyone
                    has solved this issue?

                    My best solution (I don't think Django supports
                    this) would be to have a special type of field
                    called a DeprecatedField. It would delete the
                    field from Django's perspective, but keep the
                    column in the DB. Django would no longer query
                    for the column, but the column would still be in
                    the DB. On the next release, I could remove the
                    column completely (with a RemoveField operation)
                    and the existing code would not error because it
                    has no knowledge of the column.

                    I noticed Django has an idea of a private field,
                    which is on a model but not in the DB. Is there
                    a way to create a field that is in the DB, but
                    Django model doesn't query for it or allow it to
                    be used in creates and updates? Very similar to
                    the managed=False on the Model, but on the Field
                    level. If anyone has other approaches to the
                    problem, I would be very excited to find
                    alternative methods.

                    Thanks,
                    Taylor
-- You received this message because you are
                    subscribed to the Google Groups "Django users"
                    group.
                    To unsubscribe from this group and stop
                    receiving emails from it, send an email to
                    django-users...@googlegroups.com.
                    To post to this group, send email to
                    django...@googlegroups.com.
                    Visit this group at
                    https://groups.google.com/group/django-users
                    <https://groups.google.com/group/django-users>.
                    To view this discussion on the web visit
                    
https://groups.google.com/d/msgid/django-users/a52ae01a-1a7d-43ce-a94f-fb00c4e1b7d1%40googlegroups.com
                    
<https://groups.google.com/d/msgid/django-users/a52ae01a-1a7d-43ce-a94f-fb00c4e1b7d1%40googlegroups.com?utm_medium=email&utm_source=footer>.
                    For more options, visit
                    https://groups.google.com/d/optout
                    <https://groups.google.com/d/optout>.


-- You received this message because you are subscribed to
            the Google Groups "Django users" group.
            To unsubscribe from this group and stop receiving emails
            from it, send an email to django-users...@googlegroups.com.
            To post to this group, send email to
            django...@googlegroups.com.
            Visit this group at
            https://groups.google.com/group/django-users
            <https://groups.google.com/group/django-users>.
            To view this discussion on the web visit
            
https://groups.google.com/d/msgid/django-users/e1f61e62-a6cc-44a4-ba35-7fa8b28c5549%40googlegroups.com
            
<https://groups.google.com/d/msgid/django-users/e1f61e62-a6cc-44a4-ba35-7fa8b28c5549%40googlegroups.com?utm_medium=email&utm_source=footer>.
            For more options, visit
            https://groups.google.com/d/optout
            <https://groups.google.com/d/optout>.

-- Jani Tiainen

-- You received this message because you are subscribed to
            the Google Groups "Django users" group.
            To unsubscribe from this group and stop receiving emails
            from it, send an email to django-users...@googlegroups.com.
            To post to this group, send email to
            django...@googlegroups.com.
            Visit this group at
            https://groups.google.com/group/django-users
            <https://groups.google.com/group/django-users>.
            To view this discussion on the web visit
            
https://groups.google.com/d/msgid/django-users/d4ba64c5-0dac-bcb4-520f-78835d049ad4%40gmail.com
            
<https://groups.google.com/d/msgid/django-users/d4ba64c5-0dac-bcb4-520f-78835d049ad4%40gmail.com?utm_medium=email&utm_source=footer>.


            For more options, visit
            https://groups.google.com/d/optout
            <https://groups.google.com/d/optout>.


-- You received this message because you are subscribed to the
    Google Groups "Django users" group.
    To unsubscribe from this group and stop receiving emails from it,
    send an email to django-users...@googlegroups.com <javascript:>.
    To post to this group, send email to django...@googlegroups.com
    <javascript:>.
    Visit this group at https://groups.google.com/group/django-users
    <https://groups.google.com/group/django-users>.
    To view this discussion on the web visit
    
https://groups.google.com/d/msgid/django-users/f8891294-3be5-41d0-a6a7-9740d7b8b3bb%40googlegroups.com
    
<https://groups.google.com/d/msgid/django-users/f8891294-3be5-41d0-a6a7-9740d7b8b3bb%40googlegroups.com?utm_medium=email&utm_source=footer>.
    For more options, visit https://groups.google.com/d/optout
    <https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com <mailto:django-users+unsubscr...@googlegroups.com>. To post to this group, send email to django-users@googlegroups.com <mailto:django-users@googlegroups.com>.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/a4d711f4-7996-47ea-b903-c73698bdac29%40googlegroups.com <https://groups.google.com/d/msgid/django-users/a4d711f4-7996-47ea-b903-c73698bdac29%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

--
Jani Tiainen

--
You received this message because you are subscribed to the Google Groups "Django 
users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/ad797cfc-7a4b-b41e-302a-7ad13855e9e3%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to