Ah, yes and it is an answer to Mike's initial question indeed. It is not sufficient for what Miguel describes however.
On Thu, Mar 6, 2014 at 10:43 AM, Miguel Ferreira <mferre...@schubergphilis.com> wrote: > Hi all, > > I've been looking at the way DB related code (SQL scripts and Java classes) > are updated and what is the impact of those updates on a live cloudstack DB. > By the way, my intention is to support a per-commit DB upgrade of a running > system. On Thu, Mar 6, 2014 at 2:02 PM, Donal Lafferty <donal.laffe...@citrix.com> wrote: > The thread started with a discussion of problems with out of date databases. > Mike pulled from master, rebuilt, and found out his testbed's database was > out of date and no longer works. > > We've tried to find a database solution, but out of databases are caused by > more than schema changes. The data in the tables and other pieces of > CloudStack can change after a commit, especially towards the end of release > when everyone's feature is being merged. > > With 4.2, I dealt with this issue by writing a script that rebuilt my testbed > by replaying the API calls I made to set it up in the first place. > > Is this a solution you've considered for your dev environment? > > DL > >> -----Original Message----- >> From: Daan Hoogland [mailto:daan.hoogl...@gmail.com] >> Sent: 06 March 2014 12:50 >> To: dev >> Cc: Daan Hoogland >> Subject: Re: [DISCUSS] Checking in code that will break others' environments >> >> No, but how do you mean with respect to this thread? >> >> On Thu, Mar 6, 2014 at 1:48 PM, Donal Lafferty <donal.laffe...@citrix.com> >> wrote: >> > Hi Daan, >> > >> > Is there anything stopping you from scripting the configuration of >> > your CloudStack testbed? E.g. with Marvin or CloudMonkey >> > >> > DL >> > >> >> -----Original Message----- >> >> From: Daan Hoogland [mailto:daan.hoogl...@gmail.com] >> >> Sent: 06 March 2014 09:53 >> >> To: dev >> >> Cc: Daan Hoogland >> >> Subject: Re: [DISCUSS] Checking in code that will break others' >> >> environments >> >> >> >> I totally agree with the incremental approach. I am a fascist at time >> >> because i would even want people to add downgrade scripts to any db >> >> change they make. Having them not adjust their sql is a good first step, >> though. >> >> >> >> On Thu, Mar 6, 2014 at 10:43 AM, Miguel Ferreira >> >> <mferre...@schubergphilis.com> wrote: >> >> > Hi all, >> >> > >> >> > I've been looking at the way DB related code (SQL scripts and Java >> >> > classes) >> >> are updated and what is the impact of those updates on a live cloudstack >> DB. >> >> > By the way, my intention is to support a per-commit DB upgrade of a >> >> running system. >> >> > >> >> > Anyway, I completely agree that sending an email warning people >> >> > about >> >> changes that can potentially break running environments is a good >> >> thing, and can save a lot of time and headaches. >> >> > I also like the idea of Rajani of using tools to help in the DB >> >> > migration >> >> process. >> >> > However, I would like to put forward another idea: what about >> >> > making sure >> >> that the changes to the DB (schema and data) are always incremental? >> >> > >> >> > I mean, once a SQL statement (say A) is added to a script it could >> >> > be kept as >> >> is, and subsequent modifications cloud be made via new statements >> >> (say B), instead of adapting A. >> >> > This would allow people to upgrade their databases to the point >> >> > they ran >> >> statement A, and afterwards upgrade it again to the point they ran >> >> statement B. >> >> > The same principle could also be applied to the Java classes that >> >> > do data >> >> migration, but maybe there it might be a bit more involved. >> >> > >> >> > Cheers, >> >> > Miguel >> >> > >> >> > -----Original Message----- >> >> > From: Koushik Das [mailto:koushik....@citrix.com] >> >> > Sent: donderdag 6 maart 2014 8:25 >> >> > To: <dev@cloudstack.apache.org> >> >> > Subject: Re: [DISCUSS] Checking in code that will break others' >> >> > environments >> >> > >> >> > Before doing a git pull, I generally check the sql schema changes >> >> > and run >> >> the delta manually on my existing setup. In most of the cases that >> >> works for me without having to redeploy the db. >> >> > >> >> > -Koushik >> >> > >> >> > On 06-Mar-2014, at 11:43 AM, Mike Tutkowski >> >> <mike.tutkow...@solidfire.com> wrote: >> >> > >> >> >> Yeah, I definitely just meant a "heads up" during development if >> >> >> you are going to change something that will break other people's >> >> >> environments who update. If these people know in advance, they may >> >> >> choose to postpone an update until they are at a better point. >> >> >> >> >> >> >> >> >> On Wed, Mar 5, 2014 at 11:01 PM, Rajani Karuturi >> >> >> <rajani.karut...@citrix.com >> >> >>> wrote: >> >> >> >> >> >>> Across versions db migration is taken care. I think this is bound >> >> >>> to occur while working on a release, if multiple people work on >> >> >>> the same branch with different work-in-progress features. >> >> >>> >> >> >>> Could we move to flyway or liquibase which can take care of db >> >> >>> versioning and migration? >> >> >>> >> >> >>> >> >> >>> ~Rajani >> >> >>> >> >> >>> >> >> >>> >> >> >>> On 06-Mar-2014, at 2:08 am, Mike Tutkowski >> >> >>> <mike.tutkow...@solidfire.com> >> >> >>> wrote: >> >> >>> >> >> >>>> Yeah, in this case, I'm not referring to erroneous code that >> >> >>>> breaks a person's environment (since hopefully the person >> >> >>>> wouldn't have knowingly checked in such code), but rather, say, >> >> >>>> DB-type changes that improve the system, but break current >> setups. >> >> >>>> >> >> >>>> Just a heads-up e-mail with some easily identifiable tag. >> >> >>>> >> >> >>>> Can anyone think of a good tag for this? It's not always DB >> >> >>>> related, so >> >> >>> we >> >> >>>> might want the tag to be more general. >> >> >>>> >> >> >>>> >> >> >>>> On Wed, Mar 5, 2014 at 1:28 PM, Ian Duffy <i...@ianduffy.ie> >> wrote: >> >> >>>> >> >> >>>>> +1 to this. >> >> >>>>> >> >> >>>>> Having the build suddenly break due to a git pull has been very >> >> >>> annoying! >> >> >>>>> I usually end up searching through the commit log and doing a >> >> >>>>> resets until I find a commit where it works. Then waiting >> >> >>>>> awhile until I do a git pull again and hoping the code was fixed. >> >> >>>>> >> >> >>>>> On 5 March 2014 20:19, Mike Tutkowski >> >> >>>>> <mike.tutkow...@solidfire.com> >> >> >>>>> wrote: >> >> >>>>>> Hi, >> >> >>>>>> >> >> >>>>>> I encountered a bit of a problem this morning and thought I >> >> >>>>>> would bring >> >> >>>>> it >> >> >>>>>> up for discussion. >> >> >>>>>> >> >> >>>>>> If we already have a policy around this, please let me know. >> >> >>>>>> >> >> >>>>>> So, I fetched the latest and rebased my local 4.4 development >> >> >>>>>> branch on >> >> >>>>> top >> >> >>>>>> of master. This all went just fine. >> >> >>>>>> >> >> >>>>>> When I rebuilt and re-started the CS Management Server, I soon >> >> >>> realized I >> >> >>>>>> could no longer log in from the GUI. >> >> >>>>>> >> >> >>>>>> As it turns out, the DB schema had been updated and so my >> >> >>>>>> database was >> >> >>>>> out >> >> >>>>>> of date. The code was querying for fields that didn't exist in my >> DB. >> >> >>>>>> >> >> >>>>>> As far as I know, the easiest way to get around this is to >> >> >>>>>> destroy my current cloud, run the script to re-build my >> >> >>>>>> database, then re-create >> >> >>> my >> >> >>>>>> cloud, which is somewhat time consuming. >> >> >>>>>> >> >> >>>>>> Do we have a process in place currently in which we ask those >> >> >>>>>> who make >> >> >>>>> such >> >> >>>>>> changes to send out a notification e-mail to dev@ to give >> >> >>>>>> people a >> >> >>>>> heads up >> >> >>>>>> that updating will lead to such issues? On previous projects, >> >> >>>>>> we would >> >> >>>>> send >> >> >>>>>> out an e-mail and then people could be aware to only update if >> >> >>>>>> they >> >> >>> were >> >> >>>>>> prepared for such re-work. >> >> >>>>>> >> >> >>>>>> To be clear here, I'm not meaning to pick on anyone in >> >> >>> particular...this >> >> >>>>>> has happened several times over the course of my CloudStack >> >> >>>>>> development >> >> >>>>> and >> >> >>>>>> I expect that I, too, have checked in such code (without >> >> >>>>>> sending out a relevant e-mail) that lead people to have to >> >> >>>>>> perform such a complete re-build action un-expectedly. >> >> >>>>>> >> >> >>>>>> What do people think about this? Maybe we should just add an >> >> >>>>>> e-mail tag >> >> >>>>> or >> >> >>>>>> something and point people to the relevant commit? >> >> >>>>>> >> >> >>>>>> Thanks! >> >> >>>>>> >> >> >>>>>> -- >> >> >>>>>> *Mike Tutkowski* >> >> >>>>>> *Senior CloudStack Developer, SolidFire Inc.* >> >> >>>>>> e: mike.tutkow...@solidfire.com >> >> >>>>>> o: 303.746.7302 >> >> >>>>>> Advancing the way the world uses the >> >> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play> >> >> >>>>>> *(tm)* >> >> >>>>> >> >> >>>> >> >> >>>> >> >> >>>> >> >> >>>> -- >> >> >>>> *Mike Tutkowski* >> >> >>>> *Senior CloudStack Developer, SolidFire Inc.* >> >> >>>> e: mike.tutkow...@solidfire.com >> >> >>>> o: 303.746.7302 >> >> >>>> Advancing the way the world uses the >> >> >>>> cloud<http://solidfire.com/solution/overview/?video=play> >> >> >>>> *(tm)* >> >> >>> >> >> >>> >> >> >> >> >> >> >> >> >> -- >> >> >> *Mike Tutkowski* >> >> >> *Senior CloudStack Developer, SolidFire Inc.* >> >> >> e: mike.tutkow...@solidfire.com >> >> >> o: 303.746.7302 >> >> >> Advancing the way the world uses the >> >> >> cloud<http://solidfire.com/solution/overview/?video=play> >> >> >> *(tm)* >> >> > >> >> >> >> >> >> >> >> -- >> >> Daan >> >> >> >> -- >> Daan -- Daan
