Hi Simon, For 8.0 and earlier, I would deploy ElasticSearch before deploy_end and LMA collector after post_deploy_start
For Mitaka and Newton releases, the task graph now skips dependencies that are not found for the role being processed. Now this "requires" dependency will work that previously errored. Best Regards, Matthew Mosesohn On Tue, May 17, 2016 at 6:27 PM, Simon Pasquier <spasqu...@mirantis.com> wrote: > I'm resurrecting this thread because I didn't manage to find a satisfying > solution to deal with this issue. > > First let me provide more context on the use case. The Elasticsearch/Kibana > and LMA collector plugins need to synchronize their deployment. Without too > many details, here is the workflow when both plugins are deployed: > 1. [Deployment] Install the Elasticsearch/Kibana primary node. > 2. [Deployment] Install the other Elasticsearch/Kibana nodes. > 3. [Post-Deployment] Configure the Elasticsearch cluster. > 4. [Post-Deployment] Install and configure the LMA collector. > > Task #4 should happen after #3 so we've specified the dependency in > deployement_tasks.yaml [0] but when the Elasticsearch/Kibana plugin isn't > deployed in the same environment (which is a valid case), it fails [1] with: > > Tasks 'elasticsearch-kibana-configuration, influxdb-configuration' can't be > in requires|required_for|groups|tasks for [lma-backends] because they don't > exist in the graph > > To workaround this restriction, we're using 'upload_nodes_info' as an anchor > task [2][3] since it is always present in the graph but this isn't really > elegant. Any suggestion to improve this? > > BR, > Simon > > [0] > https://github.com/openstack/fuel-plugin-lma-collector/blob/fd9337b43b6bdae6012f421e22847a1b0307ead0/deployment_tasks.yaml#L123-L139 > [1] https://bugs.launchpad.net/lma-toolchain/+bug/1573087 > [2] > https://github.com/openstack/fuel-plugin-lma-collector/blob/56ef5c42f4cd719958c4c2ac3fded1b08fe2b90f/deployment_tasks.yaml#L25-L37 > [3] > https://github.com/openstack/fuel-plugin-elasticsearch-kibana/blob/4c5736dadf457b693c30e20d1a2679165ae1155a/deployment_tasks.yaml#L156-L173 > > On Fri, Jan 29, 2016 at 4:27 PM, Igor Kalnitsky <ikalnit...@mirantis.com> > wrote: >> >> Hey folks, >> >> Simon P. wrote: >> > 1. Run task X for plugin A (if installed). >> > 2. Run task Y for plugin B (if installed). >> > 3. Run task Z for plugin A (if installed). >> >> Simon, could you please explain do you need this at the first place? I >> can imagine this case only if your two plugins are kinda dependent on >> each other. In this case, it's better to do what was said by Andrew W. >> - set 'Task Y' to require 'Task X' and that requirement will be >> satisfied anyway (even if Task X doesn't exist in the graph). >> >> >> Alex S. wrote: >> > Before we get rid of tasks.yaml can we provide a mechanism for plugin >> > devs could leverage to have tasks executes at specific points in the >> > deploy process. >> >> Yeah, I think that may be useful sometime. However, I'd prefer to >> avoid anchor usage as much as possible. There's no guarantees that >> other plugin didn't make any destructive actions early, that breaks >> you later. Anchors is good way to resolve possible conflicts, but they >> aren't bulletproof. >> >> - igor >> >> On Thu, Jan 28, 2016 at 1:31 PM, Bogdan Dobrelya <bdobre...@mirantis.com> >> wrote: >> > On 27.01.2016 14:44, Simon Pasquier wrote: >> >> Hi, >> >> >> >> I see that tasks.yaml is going to be deprecated in the future MOS >> >> versions [1]. I've got one question regarding the ordering of tasks >> >> between different plugins. >> >> With tasks.yaml, it was possible to coordinate the execution of tasks >> >> between plugins without prior knowledge of which plugins were installed >> >> [2]. >> >> For example, lets say we have 2 plugins: A and B. The plugins may or >> >> may >> >> not be installed in the same environment and the tasks execution should >> >> be: >> >> 1. Run task X for plugin A (if installed). >> >> 2. Run task Y for plugin B (if installed). >> >> 3. Run task Z for plugin A (if installed). >> >> >> >> Right now, we can set task priorities like: >> >> >> >> # tasks.yaml for plugin A >> >> - role: ['*'] >> >> stage: post_deployment/1000 >> >> type: puppet >> >> parameters: >> >> puppet_manifest: puppet/manifests/task_X.pp >> >> puppet_modules: puppet/modules >> >> >> >> - role: ['*'] >> >> stage: post_deployment/3000 >> >> type: puppet >> >> parameters: >> >> puppet_manifest: puppet/manifests/task_Z.pp >> >> puppet_modules: puppet/modules >> >> >> >> # tasks.yaml for plugin B >> >> - role: ['*'] >> >> stage: post_deployment/2000 >> >> type: puppet >> >> parameters: >> >> puppet_manifest: puppet/manifests/task_Y.pp >> >> puppet_modules: puppet/modules >> >> >> >> How would it be handled without tasks.yaml? >> > >> > I created a kinda related bug [0] and submitted a patch [1] to MOS docs >> > [2] to kill some entropy on the topic of tasks schema roles versus >> > groups and using wildcards for basic and custom roles from plugins as >> > well. There is also a fancy picture to clarify things a bit. Would be >> > nice to put more details there about custom stages as well! >> > >> > If plugins are not aware of each other, they cannot be strictly ordered >> > like "to be the very last in the deployment" as one and only shall be >> > so. That is why "coordinating the execution of tasks >> > between plugins without prior knowledge of which plugins were installed" >> > looks very confusing for me. Though, maybe wildcards with the "skipped" >> > task type may help to organize things better? >> > >> > Perhaps the Fuel plugins team could answer the question better. >> > >> > [0] https://bugs.launchpad.net/fuel/+bug/1538982 >> > [1] https://review.fuel-infra.org/16509 >> > [2] >> > >> > https://docs.mirantis.com/openstack/fuel/fuel-7.0/reference-architecture.html#task-based-deployment >> > >> >> >> >> Regards, >> >> Simon >> >> >> >> [1] https://review.openstack.org/#/c/271417/ >> >> [2] >> >> https://wiki.openstack.org/wiki/Fuel/Plugins#Plugins_deployment_order >> >> >> >> >> >> >> >> __________________________________________________________________________ >> >> OpenStack Development Mailing List (not for usage questions) >> >> Unsubscribe: >> >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> > >> > >> > -- >> > Best regards, >> > Bogdan Dobrelya, >> > Irc #bogdando >> > >> > >> > __________________________________________________________________________ >> > OpenStack Development Mailing List (not for usage questions) >> > Unsubscribe: >> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
