Re: [yocto] [layerindex-web][PATCH v2 00/12] Add Distro, dependency and recommends detection, and import_project
On 10/10/16 10:24 PM, Paul Eggleton wrote: > On Tue, 11 Oct 2016 09:19:15 Paul Eggleton wrote: >> On Mon, 10 Oct 2016 12:47:11 Mark Hatle wrote: >>> On 10/10/16 12:37 PM, Paul Eggleton wrote: On Mon, 10 Oct 2016 04:48:41 Hatle, Mark wrote: >> On Oct 10, 2016, at 2:54 AM, Paul Eggleton >> >> >> wrote: >>> On Fri, 07 Oct 2016 13:20:50 Mark Hatle wrote: >>> FYI, I have made sure these are re-based on top of paule/django18 and >>> pushed to: >>> >>> git://git.yoctoproject.org/layerindex-web mhatle/django18 >>> >>> BTW for me upgrading from paule/django18 to this version. I only had >>> to >>> run 'manage.py syncdb'. >>> >>> This created the new database and added the missing field for the >>> recommend.> >> >> I'm not sure how that worked there but it doesn't work here. If I take >> a >> database that works on paule/django18, run syncdb then open the admin > >> interface and click on "Layer branches" I get: > We found a severe lack of instructions for creating migrations. There > were > instructions for adding and deleting full DBs but not changing them. > We > will need pointers or help to resolve this. > >> no such column: layerindex_layerbranch.collection > > I never got that, but I'm also running SQLite. May be the difference. I'm using sqlite here also. >> I'm afraid we really do need to create migrations whenever we change >> the >> models. > > Based on your prior comments we failed to find ANY reasonable examples, > docs or instructions on doing this. So we are stuck without help. I'm happy to help directly if needed, but did you come across this already? https://docs.djangoproject.com/en/1.8/topics/migrations/ >>> >>> Unfortunately that is the exactly document I was looking at and could not >>> figure it out. >>> >>> I can the various commands, makemigrations and related.. and the system >>> simply told me "no migrations necessary".. and didn't generate any. >>> >>> So we were never able to figure out what a migration should look like or >>> how to generate it. >>> >>> If you (or someone else) can help us figure out how to create a migration, >>> I suspect after the first one --- future ones will be much easier to >>> generate. >> >> OK, so here's a step-by-step procedure - I suspect you'll need to stash and >> unstash your config changes along the way as well: >> >> 1) Ensure you are using a database created at the place you want to start, >> i.e. in this case paule/django18. >> >> 2) Determine which commits make changes to the models. This is one easy way: >> >> $ git log -p paule/django18..mhatle/django18 layerindex/models.py >> >> 3) We don't have an initial migration, so we need to create that first in a >> single commit before all of the others, so to do that we'll direct the >> rebase to edit the top commit on paule/django18 but instead of modifying >> that we insert a commit after it. We also want to edit each commit found in >> step 1 that made changes so we can add the migrations to them. >> >> $ git checkout mhatle/django18 >> $ git rebase -i paule/django18^ >> (now set the command to "e" on the oldest commit, in this case 6f8201fe, and >> do the same for all the commits you found in step 1, then save & quit the >> editor) >> >> 4) Create the initial migration: >> >> $ python3 manage.py makemigrations layerindex >> Migrations for 'layerindex': >> 0001_initial.py: >> - Create model BBAppend >> - Create model BBClass >> - Create model Branch >> - Create model LayerBranch >> - Create model LayerDependency >> - Create model LayerItem >> - Create model LayerMaintainer >> - Create model LayerNote >> - Create model Machine >> - Create model PythonEnvironment >> - Create model Recipe >> - Create model RecipeChange >> - Create model RecipeChangeset >> - Create model RecipeFileDependency >> - Create model ClassicRecipe >> - Add field recipe to recipefiledependency >> - Add field changeset to recipechange >> - Add field recipe to recipechange >> - Add field layerbranch to recipe >> - Add field dependency to layerdependency >> - Add field layerbranch to layerdependency >> - Add field layer to layerbranch >> - Add field update_environment to branch >> - Add field layerbranch to bbclass >> - Add field layerbranch to bbappend >> - Add field cover_layerbranch to classicrecipe >> >> 5) Apply the migrations to the database - this time we make it "fake" >> because the tables already exist: >> >> $ python manage.py migrate layerindex --fake-initial >> >> 6) Create the initial migrations commit. (We won't need to do this ever >> again) >> >> $ git add layerindex/migrations/ >> $ git commit >> >> 7) Continue the rebase: >> >> $ git rebase --continue >> >> 8) Create the migration for the changes in this commit
Re: [yocto] [layerindex-web][PATCH v2 00/12] Add Distro, dependency and recommends detection, and import_project
On Tue, 11 Oct 2016 09:19:15 Paul Eggleton wrote: > On Mon, 10 Oct 2016 12:47:11 Mark Hatle wrote: > > On 10/10/16 12:37 PM, Paul Eggleton wrote: > > > On Mon, 10 Oct 2016 04:48:41 Hatle, Mark wrote: > > >>> On Oct 10, 2016, at 2:54 AM, Paul Eggleton > > >>> > > >>> > > >>> wrote: > > On Fri, 07 Oct 2016 13:20:50 Mark Hatle wrote: > > FYI, I have made sure these are re-based on top of paule/django18 and > > pushed to: > > > > git://git.yoctoproject.org/layerindex-web mhatle/django18 > > > > BTW for me upgrading from paule/django18 to this version. I only had > > to > > run 'manage.py syncdb'. > > > > This created the new database and added the missing field for the > > recommend.> > > >>> > > >>> I'm not sure how that worked there but it doesn't work here. If I take > > >>> a > > >>> database that works on paule/django18, run syncdb then open the admin > > >> > > >>> interface and click on "Layer branches" I get: > > >> We found a severe lack of instructions for creating migrations. There > > >> were > > >> instructions for adding and deleting full DBs but not changing them. > > >> We > > >> will need pointers or help to resolve this. > > >> > > >>> no such column: layerindex_layerbranch.collection > > >> > > >> I never got that, but I'm also running SQLite. May be the difference. > > > > > > I'm using sqlite here also. > > > > > >>> I'm afraid we really do need to create migrations whenever we change > > >>> the > > >>> models. > > >> > > >> Based on your prior comments we failed to find ANY reasonable examples, > > >> docs or instructions on doing this. So we are stuck without help. > > > > > > I'm happy to help directly if needed, but did you come across this > > > already? > > > > > > https://docs.djangoproject.com/en/1.8/topics/migrations/ > > > > Unfortunately that is the exactly document I was looking at and could not > > figure it out. > > > > I can the various commands, makemigrations and related.. and the system > > simply told me "no migrations necessary".. and didn't generate any. > > > > So we were never able to figure out what a migration should look like or > > how to generate it. > > > > If you (or someone else) can help us figure out how to create a migration, > > I suspect after the first one --- future ones will be much easier to > > generate. > > OK, so here's a step-by-step procedure - I suspect you'll need to stash and > unstash your config changes along the way as well: > > 1) Ensure you are using a database created at the place you want to start, > i.e. in this case paule/django18. > > 2) Determine which commits make changes to the models. This is one easy way: > > $ git log -p paule/django18..mhatle/django18 layerindex/models.py > > 3) We don't have an initial migration, so we need to create that first in a > single commit before all of the others, so to do that we'll direct the > rebase to edit the top commit on paule/django18 but instead of modifying > that we insert a commit after it. We also want to edit each commit found in > step 1 that made changes so we can add the migrations to them. > > $ git checkout mhatle/django18 > $ git rebase -i paule/django18^ > (now set the command to "e" on the oldest commit, in this case 6f8201fe, and > do the same for all the commits you found in step 1, then save & quit the > editor) > > 4) Create the initial migration: > > $ python3 manage.py makemigrations layerindex > Migrations for 'layerindex': > 0001_initial.py: > - Create model BBAppend > - Create model BBClass > - Create model Branch > - Create model LayerBranch > - Create model LayerDependency > - Create model LayerItem > - Create model LayerMaintainer > - Create model LayerNote > - Create model Machine > - Create model PythonEnvironment > - Create model Recipe > - Create model RecipeChange > - Create model RecipeChangeset > - Create model RecipeFileDependency > - Create model ClassicRecipe > - Add field recipe to recipefiledependency > - Add field changeset to recipechange > - Add field recipe to recipechange > - Add field layerbranch to recipe > - Add field dependency to layerdependency > - Add field layerbranch to layerdependency > - Add field layer to layerbranch > - Add field update_environment to branch > - Add field layerbranch to bbclass > - Add field layerbranch to bbappend > - Add field cover_layerbranch to classicrecipe > > 5) Apply the migrations to the database - this time we make it "fake" > because the tables already exist: > > $ python manage.py migrate layerindex --fake-initial > > 6) Create the initial migrations commit. (We won't need to do this ever > again) > > $ git add layerindex/migrations/ > $ git commit > > 7) Continue the rebase: > > $ git rebase --continue > > 8) Create the migration for the changes in this commit: > > $ python manage.py makemigration
Re: [yocto] [layerindex-web][PATCH v2 00/12] Add Distro, dependency and recommends detection, and import_project
On Mon, 10 Oct 2016 12:47:11 Mark Hatle wrote: > On 10/10/16 12:37 PM, Paul Eggleton wrote: > > On Mon, 10 Oct 2016 04:48:41 Hatle, Mark wrote: > >>> On Oct 10, 2016, at 2:54 AM, Paul Eggleton > >>> > >>> > >>> wrote: > On Fri, 07 Oct 2016 13:20:50 Mark Hatle wrote: > FYI, I have made sure these are re-based on top of paule/django18 and > pushed to: > > git://git.yoctoproject.org/layerindex-web mhatle/django18 > > BTW for me upgrading from paule/django18 to this version. I only had > to > run 'manage.py syncdb'. > > This created the new database and added the missing field for the > recommend.> > >>> > >>> I'm not sure how that worked there but it doesn't work here. If I take a > >>> database that works on paule/django18, run syncdb then open the admin > >> > >>> interface and click on "Layer branches" I get: > >> We found a severe lack of instructions for creating migrations. There > >> were > >> instructions for adding and deleting full DBs but not changing them. We > >> will need pointers or help to resolve this. > >> > >>> no such column: layerindex_layerbranch.collection > >> > >> I never got that, but I'm also running SQLite. May be the difference. > > > > I'm using sqlite here also. > > > >>> I'm afraid we really do need to create migrations whenever we change the > >>> models. > >> > >> Based on your prior comments we failed to find ANY reasonable examples, > >> docs or instructions on doing this. So we are stuck without help. > > > > I'm happy to help directly if needed, but did you come across this > > already? > > > > https://docs.djangoproject.com/en/1.8/topics/migrations/ > > Unfortunately that is the exactly document I was looking at and could not > figure it out. > > I can the various commands, makemigrations and related.. and the system > simply told me "no migrations necessary".. and didn't generate any. > > So we were never able to figure out what a migration should look like or how > to generate it. > > If you (or someone else) can help us figure out how to create a migration, I > suspect after the first one --- future ones will be much easier to > generate. OK, so here's a step-by-step procedure - I suspect you'll need to stash and unstash your config changes along the way as well: 1) Ensure you are using a database created at the place you want to start, i.e. in this case paule/django18. 2) Determine which commits make changes to the models. This is one easy way: $ git log -p paule/django18..mhatle/django18 layerindex/models.py 3) We don't have an initial migration, so we need to create that first in a single commit before all of the others, so to do that we'll direct the rebase to edit the top commit on paule/django18 but instead of modifying that we insert a commit after it. We also want to edit each commit found in step 1 that made changes so we can add the migrations to them. $ git checkout mhatle/django18 $ git rebase -i paule/django18^ (now set the command to "e" on the oldest commit, in this case 6f8201fe, and do the same for all the commits you found in step 1, then save & quit the editor) 4) Create the initial migration: $ python3 manage.py makemigrations layerindex Migrations for 'layerindex': 0001_initial.py: - Create model BBAppend - Create model BBClass - Create model Branch - Create model LayerBranch - Create model LayerDependency - Create model LayerItem - Create model LayerMaintainer - Create model LayerNote - Create model Machine - Create model PythonEnvironment - Create model Recipe - Create model RecipeChange - Create model RecipeChangeset - Create model RecipeFileDependency - Create model ClassicRecipe - Add field recipe to recipefiledependency - Add field changeset to recipechange - Add field recipe to recipechange - Add field layerbranch to recipe - Add field dependency to layerdependency - Add field layerbranch to layerdependency - Add field layer to layerbranch - Add field update_environment to branch - Add field layerbranch to bbclass - Add field layerbranch to bbappend - Add field cover_layerbranch to classicrecipe 5) Apply the migrations to the database - this time we make it "fake" because the tables already exist: $ python manage.py migrate layerindex --fake-initial 6) Create the initial migrations commit. (We won't need to do this ever again) $ git add layerindex/migrations/ $ git commit 7) Continue the rebase: $ git rebase --continue 8) Create the migration for the changes in this commit: $ python manage.py makemigrations layerindex Migrations for 'layerindex': 0002_distro.py: - Create model Distro 9) Apply the migrations (*not* fake this time): $ python manage.py migrate layerindex 10) Amend the migrations to the current commit: $ git add layerindex/migrations/ $ git commit --amend 11) Continue the rebase: $ git rebase --conti
Re: [yocto] [layerindex-web][PATCH v2 00/12] Add Distro, dependency and recommends detection, and import_project
On 10/10/16 12:37 PM, Paul Eggleton wrote: > On Mon, 10 Oct 2016 04:48:41 Hatle, Mark wrote: >>> On Oct 10, 2016, at 2:54 AM, Paul Eggleton >>> wrote: On Fri, 07 Oct 2016 13:20:50 Mark Hatle wrote: FYI, I have made sure these are re-based on top of paule/django18 and pushed to: git://git.yoctoproject.org/layerindex-web mhatle/django18 BTW for me upgrading from paule/django18 to this version. I only had to run 'manage.py syncdb'. This created the new database and added the missing field for the recommend.> >>> I'm not sure how that worked there but it doesn't work here. If I take a >>> database that works on paule/django18, run syncdb then open the admin >>> interface and click on "Layer branches" I get: >> >> We found a severe lack of instructions for creating migrations. There were >> instructions for adding and deleting full DBs but not changing them. We >> will need pointers or help to resolve this. >> >>> no such column: layerindex_layerbranch.collection >> >> I never got that, but I'm also running SQLite. May be the difference. > > I'm using sqlite here also. > >>> I'm afraid we really do need to create migrations whenever we change the >>> models. >> >> Based on your prior comments we failed to find ANY reasonable examples, docs >> or instructions on doing this. So we are stuck without help. > > I'm happy to help directly if needed, but did you come across this already? > > https://docs.djangoproject.com/en/1.8/topics/migrations/ Unfortunately that is the exactly document I was looking at and could not figure it out. I can the various commands, makemigrations and related.. and the system simply told me "no migrations necessary".. and didn't generate any. So we were never able to figure out what a migration should look like or how to generate it. If you (or someone else) can help us figure out how to create a migration, I suspect after the first one --- future ones will be much easier to generate. --Mark > Cheers, > Paul > -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] [layerindex-web][PATCH v2 00/12] Add Distro, dependency and recommends detection, and import_project
On Mon, 10 Oct 2016 04:48:41 Hatle, Mark wrote: > > On Oct 10, 2016, at 2:54 AM, Paul Eggleton > > wrote: > >> On Fri, 07 Oct 2016 13:20:50 Mark Hatle wrote: > >> FYI, I have made sure these are re-based on top of paule/django18 and > >> pushed to: > >> > >> git://git.yoctoproject.org/layerindex-web mhatle/django18 > >> > >> BTW for me upgrading from paule/django18 to this version. I only had to > >> run 'manage.py syncdb'. > >> > >> This created the new database and added the missing field for the > >> recommend.> > > I'm not sure how that worked there but it doesn't work here. If I take a > > database that works on paule/django18, run syncdb then open the admin > > interface and click on "Layer branches" I get: > > We found a severe lack of instructions for creating migrations. There were > instructions for adding and deleting full DBs but not changing them. We > will need pointers or help to resolve this. > > > no such column: layerindex_layerbranch.collection > > I never got that, but I'm also running SQLite. May be the difference. I'm using sqlite here also. > > I'm afraid we really do need to create migrations whenever we change the > > models. > > Based on your prior comments we failed to find ANY reasonable examples, docs > or instructions on doing this. So we are stuck without help. I'm happy to help directly if needed, but did you come across this already? https://docs.djangoproject.com/en/1.8/topics/migrations/ Cheers, Paul -- Paul Eggleton Intel Open Source Technology Centre -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] [layerindex-web][PATCH v2 00/12] Add Distro, dependency and recommends detection, and import_project
> On Oct 10, 2016, at 2:54 AM, Paul Eggleton > wrote: > >> On Fri, 07 Oct 2016 13:20:50 Mark Hatle wrote: >> FYI, I have made sure these are re-based on top of paule/django18 and pushed >> to: >> >> git://git.yoctoproject.org/layerindex-web mhatle/django18 >> >> BTW for me upgrading from paule/django18 to this version. I only had to run >> 'manage.py syncdb'. >> >> This created the new database and added the missing field for the recommend. > > I'm not sure how that worked there but it doesn't work here. If I take a > database that works on paule/django18, run syncdb then open the admin > interface and click on "Layer branches" I get: > We found a severe lack of instructions for creating migrations. There were instructions for adding and deleting full DBs but not changing them. We will need pointers or help to resolve this. > no such column: layerindex_layerbranch.collection I never got that, but I'm also running SQLite. May be the difference. > I'm afraid we really do need to create migrations whenever we change the > models. Based on your prior comments we failed to find ANY reasonable examples, docs or instructions on doing this. So we are stuck without help. > Cheers, > Paul > > -- > > Paul Eggleton > Intel Open Source Technology Centre -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] [layerindex-web][PATCH v2 00/12] Add Distro, dependency and recommends detection, and import_project
On Fri, 07 Oct 2016 13:20:50 Mark Hatle wrote: > FYI, I have made sure these are re-based on top of paule/django18 and pushed > to: > > git://git.yoctoproject.org/layerindex-web mhatle/django18 > > BTW for me upgrading from paule/django18 to this version. I only had to run > 'manage.py syncdb'. > > This created the new database and added the missing field for the recommend. I'm not sure how that worked there but it doesn't work here. If I take a database that works on paule/django18, run syncdb then open the admin interface and click on "Layer branches" I get: no such column: layerindex_layerbranch.collection I'm afraid we really do need to create migrations whenever we change the models. Cheers, Paul -- Paul Eggleton Intel Open Source Technology Centre -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] [layerindex-web][PATCH v2 00/12] Add Distro, dependency and recommends detection, and import_project
FYI, I have made sure these are re-based on top of paule/django18 and pushed to: git://git.yoctoproject.org/layerindex-web mhatle/django18 BTW for me upgrading from paule/django18 to this version. I only had to run 'manage.py syncdb'. This created the new database and added the missing field for the recommend. I then ran update.py --forcereload and everything showed up as expected. --Mark On 10/7/16 10:57 AM, Liam R. Howlett wrote: > This set of patches adds a number of features to the layerindex code: > - Adds the ability to set the actual-branch to import_layer. > - Adds distro to the database and web interface. > - Adds collection & version information to layerbranch. > - Uses layer name and collection name to determine dependencies and > recommends. > - Adds import_project to simplify pulling all layers and openembedded-core > into a layer index. > > Liam R. Howlett (11): > import_layer: Add --actual-branch option > layerindex/tools/import_layer.py: Sanitize layer name. > layerindex/tools/import_layer.py: Avoid failing if there is any layer > to add. > layerindex/utils: Update runcmd to decode binary strings to strings. > layerindex: Add distro to web interface and model. > layerindex/tools/import_project: Add import_project > layerindex/recipeparse.py: refactor setup_tinfoil, > checkout_layer_branch, parse_layer_conf to utils.py > layerindex: Detect dependencies from layer.conf files > layerindex: Add collection and version to layerbranch > layerindexer: Add layer recommends support > recipeparse: remove unnecessary else statement. > > Mark Hatle (1): > layerindex/update_layer.py: Preserve the recipedependency files > > layerindex/admin.py| 11 +++ > layerindex/bulkchange.py | 4 +- > layerindex/layerconfparse.py | 50 +++ > layerindex/models.py | 23 + > layerindex/recipeparse.py | 53 +++ > layerindex/restviews.py| 10 ++- > layerindex/tools/import_classic.py | 2 + > layerindex/tools/import_layer.py | 47 +- > layerindex/tools/import_project.py | 180 > + > layerindex/update.py | 42 - > layerindex/update_layer.py | 87 -- > layerindex/urls.py | 3 + > layerindex/urls_branch.py | 6 +- > layerindex/utils.py| 159 +++- > layerindex/views.py| 29 +- > templates/layerindex/detail.html | 55 +--- > templates/layerindex/distros.html | 76 > templates/layerindex/layers.html | 1 + > templates/layerindex/machines.html | 1 + > templates/layerindex/recipes.html | 1 + > 20 files changed, 767 insertions(+), 73 deletions(-) > create mode 100644 layerindex/layerconfparse.py > create mode 100755 layerindex/tools/import_project.py > create mode 100644 templates/layerindex/distros.html > -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] [layerindex-web][PATCH v2 00/12] Add Distro, dependency and recommends detection, and import_project
This set of patches adds a number of features to the layerindex code: - Adds the ability to set the actual-branch to import_layer. - Adds distro to the database and web interface. - Adds collection & version information to layerbranch. - Uses layer name and collection name to determine dependencies and recommends. - Adds import_project to simplify pulling all layers and openembedded-core into a layer index. Liam R. Howlett (11): import_layer: Add --actual-branch option layerindex/tools/import_layer.py: Sanitize layer name. layerindex/tools/import_layer.py: Avoid failing if there is any layer to add. layerindex/utils: Update runcmd to decode binary strings to strings. layerindex: Add distro to web interface and model. layerindex/tools/import_project: Add import_project layerindex/recipeparse.py: refactor setup_tinfoil, checkout_layer_branch, parse_layer_conf to utils.py layerindex: Detect dependencies from layer.conf files layerindex: Add collection and version to layerbranch layerindexer: Add layer recommends support recipeparse: remove unnecessary else statement. Mark Hatle (1): layerindex/update_layer.py: Preserve the recipedependency files layerindex/admin.py| 11 +++ layerindex/bulkchange.py | 4 +- layerindex/layerconfparse.py | 50 +++ layerindex/models.py | 23 + layerindex/recipeparse.py | 53 +++ layerindex/restviews.py| 10 ++- layerindex/tools/import_classic.py | 2 + layerindex/tools/import_layer.py | 47 +- layerindex/tools/import_project.py | 180 + layerindex/update.py | 42 - layerindex/update_layer.py | 87 -- layerindex/urls.py | 3 + layerindex/urls_branch.py | 6 +- layerindex/utils.py| 159 +++- layerindex/views.py| 29 +- templates/layerindex/detail.html | 55 +--- templates/layerindex/distros.html | 76 templates/layerindex/layers.html | 1 + templates/layerindex/machines.html | 1 + templates/layerindex/recipes.html | 1 + 20 files changed, 767 insertions(+), 73 deletions(-) create mode 100644 layerindex/layerconfparse.py create mode 100755 layerindex/tools/import_project.py create mode 100644 templates/layerindex/distros.html -- 1.9.1 -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
