I should clarify one thing... On Thu, 2 May 2019 at 20:28, Shaheed Haque <shaheedha...@gmail.com> wrote:
> On Thu, 2 May 2019 at 19:37, <dansm...@gmail.com> wrote: > >> What gold! >> >> The essential piece of your logic is here: >> >>> Enforce a development process that ensures that (roughly speaking) all >>> database changes result in a new column, and where the old cannot be >>> removed until a later update cycle. All migrations populate the new column. >> >> >> I assume that "enforce" is a software engineering management thing, not >> clever CI/CD code. >> > > Ack. > This response appears to contradict my original note, but what I actually think is that *some* automation along the lines of Django's existing migration capability is possible, and that careful software engineering management would likely be needed to (a) maximise the chance the automation will work, and (b) detect the cases when it won't. As for the posited automation itself, one could imagine that the existing Django logic to analyse the need for migrations would work as-is. What would likely be needed is a different "backend" that targets blue-green upgrades. I would guess the overall level of automation possible is less than the near-magical current system, but I'm hopeful it might make the problem manageable. (Of course, I'm still crossing my fingers that some real Django migration and SQL gurus will solve this before I have to). Thanks, Shaheed > To change a column, you essentially do it in two steps: >> >> - Create new column with migrations, and do a release to the >> environment. >> - Drop old column with migrations, and do a release to the >> environment. >> >> Just so. > > >> If you are only dropping an old column, it might go like this: >> >> - Drop the old column from the model, but not from the database >> (e.g. assure that makemigrations has not been run), test and deploy. >> - Add the migration that does away with the old column - test and >> deploy. >> >> Indeed. > > Personally, I have a similar career trajectory, but from systems >> programming in C/C++ to software architect for webapps. Understanding >> C/C++ and Linux gave me a capability of working with both developers and >> system guys. I think it was in 2013 that I did the presentation that >> started to kill Adobe ColdFusion (which I didn't want to learn). I >> instead the next 6 years getting all of our applications moved to Django, >> now we are in transition to Python 3.6 and Django 2.2, with our first cloud >> system coming up. >> > > LOL.I bet we could both tell some tales. > > On your slow cloud builds, I have an architecture that may help. The basic >> idea is easy to explain: >> >> - Do system provisioning through ansible roles >> - Install all the roles needed for all of your stacks when you build >> the AMI, but only run some of them, e.g. the basics. >> - If your build time in the cloud takes too long, the architecture >> assures you can easily pivot to prebaking more into the AMI, but you are >> not assuming you will have to. >> >> Of course, you still cannot go to milliseconds. But it allows you to >> trade building ahead of time and building during bring-up to nearly your >> hearts content. Even the role that knows how to install a Python webapp is >> already on the AMI. >> > > From the previous experience I alluded to, I appreciate that using Puppet > (or Chef or Ansible) to pre-bake a VM image and such an image would, as you > note, significantly reduce my re-spin "system down" window. Now, I hesitate > to say the next bit out loud, because I don't yet know if I am right, or > the received wisdom about freezing dependencies is right, but here > goes...comments welcome! > > My current thinking is that the notion of trying to keep a public website > secure by freezing dependencies (think virtualenv AND apt) is an Sisiphyian > task given the implied need to track the transitive fanout of ALL > dependencies down to the kernel. So, given that we have decent test > coverage, and are frequently running those "live-to-test" upgrades, I can > trade the security vulnerability tracking for compatibility tracking by > having each upgrade cycle rebuild from the latest apt and pypi > repositories. That's a win because we have to do the compatibility tracking > anyway. > > Thus, I get early exposure to occasional issues (e.g. pgAdmin was broken > recently by pyscopg2 2.8, and django-polymorphic is broken by Django 2.2, > both of which I discovered within about a day of the issue arising) while > ditching the need to continuously track the security perimeter of the whole > shooting match. Another way to look at it is that I leverage everybody > else's tracking of their security issues (fixes for functional issues are a > side benefit). > > Assuming this analysis proves itself, the benefits trump the advantages of > a pre-baked VM image over rebuilds from live repos, at least for me (I > might think differently if we had more human bandwidth to burn). > > And anyway, I'll fix the downtime with a cluster. One day. :-) > > > -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to django-users+unsubscr...@googlegroups.com. >> To post to this group, send email to django-users@googlegroups.com. >> Visit this group at https://groups.google.com/group/django-users. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-users/0799c1b8-04cc-424e-a08d-f124eab03ce7%40googlegroups.com >> <https://groups.google.com/d/msgid/django-users/0799c1b8-04cc-424e-a08d-f124eab03ce7%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAHAc2jePajQxpf3ZgR5Ys9iJhWykz%3DbJcAGg-bmpC%2Bk4t%2B47xA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
