I'm resurrecting this thread because I didn't manage to find a satisfying solution to deal with this issue.
First let me provide more context on the use case. The Elasticsearch/Kibana and LMA collector plugins need to synchronize their deployment. Without too many details, here is the workflow when both plugins are deployed: 1. [Deployment] Install the Elasticsearch/Kibana primary node. 2. [Deployment] Install the other Elasticsearch/Kibana nodes. 3. [Post-Deployment] Configure the Elasticsearch cluster. 4. [Post-Deployment] Install and configure the LMA collector. Task #4 should happen after #3 so we've specified the dependency in deployement_tasks.yaml [0] but when the Elasticsearch/Kibana plugin isn't deployed in the same environment (which is a valid case), it fails [1] with: Tasks 'elasticsearch-kibana-configuration, influxdb-configuration' can't be in requires|required_for|groups|tasks for [lma-backends] because they don't exist in the graph To workaround this restriction, we're using 'upload_nodes_info' as an anchor task [2][3] since it is always present in the graph but this isn't really elegant. Any suggestion to improve this? BR, Simon [0] https://github.com/openstack/fuel-plugin-lma-collector/blob/fd9337b43b6bdae6012f421e22847a1b0307ead0/deployment_tasks.yaml#L123-L139 [1] https://bugs.launchpad.net/lma-toolchain/+bug/1573087 [2] https://github.com/openstack/fuel-plugin-lma-collector/blob/56ef5c42f4cd719958c4c2ac3fded1b08fe2b90f/deployment_tasks.yaml#L25-L37 [3] https://github.com/openstack/fuel-plugin-elasticsearch-kibana/blob/4c5736dadf457b693c30e20d1a2679165ae1155a/deployment_tasks.yaml#L156-L173 On Fri, Jan 29, 2016 at 4:27 PM, Igor Kalnitsky <ikalnit...@mirantis.com> wrote: > Hey folks, > > Simon P. wrote: > > 1. Run task X for plugin A (if installed). > > 2. Run task Y for plugin B (if installed). > > 3. Run task Z for plugin A (if installed). > > Simon, could you please explain do you need this at the first place? I > can imagine this case only if your two plugins are kinda dependent on > each other. In this case, it's better to do what was said by Andrew W. > - set 'Task Y' to require 'Task X' and that requirement will be > satisfied anyway (even if Task X doesn't exist in the graph). > > > Alex S. wrote: > > Before we get rid of tasks.yaml can we provide a mechanism for plugin > > devs could leverage to have tasks executes at specific points in the > > deploy process. > > Yeah, I think that may be useful sometime. However, I'd prefer to > avoid anchor usage as much as possible. There's no guarantees that > other plugin didn't make any destructive actions early, that breaks > you later. Anchors is good way to resolve possible conflicts, but they > aren't bulletproof. > > - igor > > On Thu, Jan 28, 2016 at 1:31 PM, Bogdan Dobrelya <bdobre...@mirantis.com> > wrote: > > On 27.01.2016 14:44, Simon Pasquier wrote: > >> Hi, > >> > >> I see that tasks.yaml is going to be deprecated in the future MOS > >> versions [1]. I've got one question regarding the ordering of tasks > >> between different plugins. > >> With tasks.yaml, it was possible to coordinate the execution of tasks > >> between plugins without prior knowledge of which plugins were installed > [2]. > >> For example, lets say we have 2 plugins: A and B. The plugins may or may > >> not be installed in the same environment and the tasks execution should > be: > >> 1. Run task X for plugin A (if installed). > >> 2. Run task Y for plugin B (if installed). > >> 3. Run task Z for plugin A (if installed). > >> > >> Right now, we can set task priorities like: > >> > >> # tasks.yaml for plugin A > >> - role: ['*'] > >> stage: post_deployment/1000 > >> type: puppet > >> parameters: > >> puppet_manifest: puppet/manifests/task_X.pp > >> puppet_modules: puppet/modules > >> > >> - role: ['*'] > >> stage: post_deployment/3000 > >> type: puppet > >> parameters: > >> puppet_manifest: puppet/manifests/task_Z.pp > >> puppet_modules: puppet/modules > >> > >> # tasks.yaml for plugin B > >> - role: ['*'] > >> stage: post_deployment/2000 > >> type: puppet > >> parameters: > >> puppet_manifest: puppet/manifests/task_Y.pp > >> puppet_modules: puppet/modules > >> > >> How would it be handled without tasks.yaml? > > > > I created a kinda related bug [0] and submitted a patch [1] to MOS docs > > [2] to kill some entropy on the topic of tasks schema roles versus > > groups and using wildcards for basic and custom roles from plugins as > > well. There is also a fancy picture to clarify things a bit. Would be > > nice to put more details there about custom stages as well! > > > > If plugins are not aware of each other, they cannot be strictly ordered > > like "to be the very last in the deployment" as one and only shall be > > so. That is why "coordinating the execution of tasks > > between plugins without prior knowledge of which plugins were installed" > > looks very confusing for me. Though, maybe wildcards with the "skipped" > > task type may help to organize things better? > > > > Perhaps the Fuel plugins team could answer the question better. > > > > [0] https://bugs.launchpad.net/fuel/+bug/1538982 > > [1] https://review.fuel-infra.org/16509 > > [2] > > > https://docs.mirantis.com/openstack/fuel/fuel-7.0/reference-architecture.html#task-based-deployment > > > >> > >> Regards, > >> Simon > >> > >> [1] https://review.openstack.org/#/c/271417/ > >> [2] > https://wiki.openstack.org/wiki/Fuel/Plugins#Plugins_deployment_order > >> > >> > >> > __________________________________________________________________________ > >> OpenStack Development Mailing List (not for usage questions) > >> Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> > > > > > > -- > > Best regards, > > Bogdan Dobrelya, > > Irc #bogdando > > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev