[openstack-dev] [vitrage] Pushed specification of Vitrage templates, for supporting Root Cause Analysis (RCA) and additional features

[Rosensweig, Elisha (Nokia - IL)]

Hi All,

Please see the following specification for the Vitrage Templates:

https://review.openstack.org/#/c/272033/1/doc/source/vitrage-template-format.rst

Vitrage Templates are used to support the RCA capabilities of Vitrage, as well 
as additional features such as deduced alarms and states.

For more information on Vitrage, please see 
https://wiki.openstack.org/wiki/Vitrage.

Elisha R.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Vitrage] putting overlapping-templates-basic-support on the back-burner

[Rosensweig, Elisha (Nokia - IL)]

Hi All,

Since 
* the overlapping templates BP 
(https://blueprints.launchpad.net/vitrage/+spec/overlapping-templates-basic-support)
 is a large one, which deserves a lot of design and discussion
* there is a lot of focus on stability and clarity for the summit

I'll be putting working on this BP on hold for the coming week, and focus 
instead on Austin-tasks. Specifically, today I think I'll work on reviewing the 
Vitrage documentation. 

In parallel, I'll be trying to automate the nagios installation as part of the 
vitrage "Stack"

If anyone has any issue with the above, please let me know.

Thanks,

Elisha Rosensweig, Ph.D.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Vitrage] questions about "relationships"

[Rosensweig, Elisha (Nokia - IL)]

Hi Tony,

Glad to help :) . Regarding your questions from the last two emails:


1.   Yes, the Entity Graph (what you called the global graph), is generated 
regardless of what templates were defined. You can get this graph using the CLI 
command "vitrage topology show", and we are currently in the finishing stages 
of also having Horizon support for this that displays the graph.

2.   Yes, the template definitions have no direct impact on the Entity 
Graph. The impact on the Entity Graph is only via the template actions (see 
answer for next question)

3.   The "add causal relationship" will indeed add an edge to the Entity 
graph, between vertices already in the graph. Other actions, like raise_alarm, 
can cause vitrage to raise an alarm which will be added into the graph as well.

As a result, depending on the templates you add, the graph could indeed become 
very "dense" with edges. As Vitrage matures, it might also support many more 
types of relationships and entities, and the graph will grow accordingly.

We've given much thought to this issue, and have plans on addressing possible 
problems that this could cause. For example, the Entity Graph is currently 
in-memory, and we have plans for supporting also persistant-DB implementations, 
e.g., Neo4J, Titan.



As for your last question, yes - that is what the EvaluatorEventTransformer 
handles.



Keep those questions coming...



Elisha


From: EXT Wang, Tony T (Nokia - CN) [mailto:tony_t.w...@nokia.com]
Sent: Tuesday, April 19, 2016 5:37 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Vitrage] questions about "relationships"

One more question, I suppose "EvaluatorEventTransformer" is used to handle 
events sent by ScenarioEvaluator, and add/delete vertex/edge to the "global 
graph" triggered by Actions. Is it right?

From: EXT Wang, Tony T (Nokia - CN) [mailto:tony_t.w...@nokia.com]
Sent: Tuesday, April 19, 2016 10:01 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Vitrage] questions about "relationships"

Hi Elisha,

Thanks for your quick response. I think I got your points.
To double confirm, further questions are coming :):

1.   Regardless if there is template definition, Vitrage constructs a graph 
based on the data from plugins defined in datasources conf. Let's call it 
"global graph". Is it right?

2.   "entities" and "relationships" defined in template "definitions" won't 
add any new vertex and edge to the "global graph". They are only used to be 
referred  by "condition"/"action" in "scenarios" section. "condition" also uses 
them to construct another graph("ScenarioEvaluator._evaluate_and_condition"), 
let's call it "template graph". This template graph is used to  find 
occurrences of a template graph in the "global" 
graph("NXAlgorithm.sub_graph_matching").

3.   If so, which graph will be applied by the actions by ScenarioEvaluator?

For example, will "AddCausalRelationship" add "causes" edge to the "global 
graph" or "template graph"?

I suppose it should be "global graph". If so, it is possible the "global graph" 
will grow very huge someday?

Thanks,
Tony

From: Rosensweig, Elisha (Nokia - IL)
Sent: Tuesday, April 19, 2016 7:00 PM
To: Wang, Tony T (Nokia - CN); Weyl, Alexey (Nokia - IL); Afek, Ifat (Nokia - 
IL); Shamir, Ohad (Nokia - IL)
Subject: RE: [Vitrage] questions about "relationships"

Hi Tony,

Thanks for your interest in Vitrage! Any questions you might have, please send 
us. You may also send them to the openstack mailing list, and we will answer 
there as well.

Regarding your question:


* In the templates, the definitions in the "definitions" section are 
used only by the scenario(s) within the template - the condition, as well as 
the action, might reference these.

* These templates are used for business logic for the Scenario 
Evaluator, to detect patterns in the Entity Graph and perform subsequent 
actions.

* On the other hand, the populating of this Entity Graph is done via 
Datasources. In the Transformer of each datasource, we define how we want to 
integrate the data received into the Entity Graph, what labels each edge 
(=relationship) will have etc.

For example, the InstanceTransformer will define that whenever we add an 
instance to the Entity Graph, the instance is related to it's host with a 
"contains" relationship. The template, on the other hand, states "IF there is a 
host in error AND this host contains an instance, THEN do XXX".

So indeed, we need the conditions in the template to use the same "language" as 
the transfo

[openstack-dev] [keystone] Problem with WSGI on keystone

[Rosensweig, Elisha (Nokia - IL)]

Hi All,

Recently, I've been having trouble running stack.sh from scratch. With the 
default configuration I've been using for a while, I get the following error in 
/opt/stack/logs/key.log:

2016-04-19 16:05:26.630 22047 DEBUG keystone.common.controller 
[req-3d384c1f-8b9a-4ff0-aa30-a4aad73963fb 3bdfb473e0e544a4a1889c9192d2ba3c 
fd622e15c12a49a299a63022c5d34088 - default default] RBAC: Authorization granted 
inner /opt/stack/keystone/keystone/common/controller.py:180
mod_wsgi (pid=22045): Target WSGI script '/usr/local/bin/keystone-wsgi-admin' 
cannot be loaded as Python module.
mod_wsgi (pid=22045): Exception occurred processing WSGI script 
'/usr/local/bin/keystone-wsgi-admin'.
Traceback (most recent call last):
  File "/usr/local/bin/keystone-wsgi-admin", line 36, in 
application = initialize_admin_application()
  File "/opt/stack/keystone/keystone/server/wsgi.py", line 62, in 
initialize_admin_application
return initialize_application('admin')
  File "/opt/stack/keystone/keystone/server/wsgi.py", line 39, in 
initialize_application
common.configure()
  File "/opt/stack/keystone/keystone/server/common.py", line 31, in configure
config.configure()
  File "/opt/stack/keystone/keystone/common/config.py", line 1131, in configure
help='Do not monkey-patch threading system modules.'))
  File "/usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py", line 2106, 
in __inner
result = f(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py", line 2282, 
in register_cli_opt
raise ArgsAlreadyParsedError("cannot register CLI option")
ArgsAlreadyParsedError: arguments already parsed: cannot register CLI option

As a workaround, I've set, in local.conf,  
   KEYSTONE_USE_MOD_WSGI=False 

And this seems to solve the issue (meaning, stack passes with no errors - have 
not done extensive testing of functionality). 

Does anyone else experience this? Any other suggestions to fix this?

Thanks

Elisha Rosensweig


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Vitrage] questions about "relationships"

[Rosensweig, Elisha (Nokia - IL)]

Hi Tony,

I agree. Looking forward for those challenges - should be an interesting 
development process.

Elisha

From: EXT Wang, Tony T (Nokia - CN) [mailto:tony_t.w...@nokia.com]
Sent: Wednesday, April 20, 2016 9:31 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Vitrage] questions about "relationships"

Elisha,

Thanks for your detail explanation. I think I can understand how Vitrage works 
now. :)
Since Vitrage needs to read data from different data sources and build one 
Entity Graph, the elasticity of vitrage-grpah process will be a big challenge 
per my understanding. :)

Thanks again,
Tony

From: EXT Rosensweig, Elisha (Nokia - IL) [mailto:elisha.rosensw...@nokia.com]
Sent: Wednesday, April 20, 2016 12:26 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Vitrage] questions about "relationships"

Hi Tony,

Glad to help :) . Regarding your questions from the last two emails:


1.   Yes, the Entity Graph (what you called the global graph), is generated 
regardless of what templates were defined. You can get this graph using the CLI 
command "vitrage topology show", and we are currently in the finishing stages 
of also having Horizon support for this that displays the graph.

2.   Yes, the template definitions have no direct impact on the Entity 
Graph. The impact on the Entity Graph is only via the template actions (see 
answer for next question)

3.   The "add causal relationship" will indeed add an edge to the Entity 
graph, between vertices already in the graph. Other actions, like raise_alarm, 
can cause vitrage to raise an alarm which will be added into the graph as well.

As a result, depending on the templates you add, the graph could indeed become 
very "dense" with edges. As Vitrage matures, it might also support many more 
types of relationships and entities, and the graph will grow accordingly.

We've given much thought to this issue, and have plans on addressing possible 
problems that this could cause. For example, the Entity Graph is currently 
in-memory, and we have plans for supporting also persistant-DB implementations, 
e.g., Neo4J, Titan.



As for your last question, yes - that is what the EvaluatorEventTransformer 
handles.



Keep those questions coming...



Elisha


From: EXT Wang, Tony T (Nokia - CN) [mailto:tony_t.w...@nokia.com]
Sent: Tuesday, April 19, 2016 5:37 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Vitrage] questions about "relationships"

One more question, I suppose "EvaluatorEventTransformer" is used to handle 
events sent by ScenarioEvaluator, and add/delete vertex/edge to the "global 
graph" triggered by Actions. Is it right?

From: EXT Wang, Tony T (Nokia - CN) [mailto:tony_t.w...@nokia.com]
Sent: Tuesday, April 19, 2016 10:01 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Vitrage] questions about "relationships"

Hi Elisha,

Thanks for your quick response. I think I got your points.
To double confirm, further questions are coming :):

1.   Regardless if there is template definition, Vitrage constructs a graph 
based on the data from plugins defined in datasources conf. Let's call it 
"global graph". Is it right?

2.   "entities" and "relationships" defined in template "definitions" won't 
add any new vertex and edge to the "global graph". They are only used to be 
referred  by "condition"/"action" in "scenarios" section. "condition" also uses 
them to construct another graph("ScenarioEvaluator._evaluate_and_condition"), 
let's call it "template graph". This template graph is used to  find 
occurrences of a template graph in the "global" 
graph("NXAlgorithm.sub_graph_matching").

3.   If so, which graph will be applied by the actions by ScenarioEvaluator?

For example, will "AddCausalRelationship" add "causes" edge to the "global 
graph" or "template graph"?

I suppose it should be "global graph". If so, it is possible the "global graph" 
will grow very huge someday?

Thanks,
Tony

From: Rosensweig, Elisha (Nokia - IL)
Sent: Tuesday, April 19, 2016 7:00 PM
To: Wang, Tony T (Nokia - CN); Weyl, Alexey (Nokia - IL); Afek, Ifat (Nokia - 
IL); Shamir, Ohad (Nokia - IL)
Subject: RE: [Vitrage] questions about "relationships"

Hi Tony,

Thanks for your interest in Vitrage! Any questions you might have, please send 
us. You may also send them to the openstack mailing list, and we will answer 
there as well.

Regarding your question:


* In the templates, the definitions in the "definitions" section are 
used only by the 

[openstack-dev] [vitrage] Design discussion for overlapping templates support & new section in Vitrage wiki

[Rosensweig, Elisha (Nokia - IL)]

Hi all,

I've added a new section in the Vitrage Wiki page where links to active design 
discussions will take place. You can find it here: 
https://wiki.openstack.org/wiki/Vitrage#Open_Design_Discussions

Once a design has been finalized, it will be moved to the "Design Documents" 
section.

The first member of this section is an etherpad for discussing Vitrage support 
for overlapping templates. Here is the link: 
https://etherpad.openstack.org/p/vitrage-overlapping-templates-support-design 

Thanks,

Elisha


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [telemetry] [ceilometer] [kilo] trouble reloading ceilometer pipeline

[Rosensweig, Elisha (Nokia - IL)]

Hi All,

I'm running with Kilo and I want to make changes to my ceilometer pipeline file 
(changing the rate of sampling, in this case). What I did was:
(1) make the changes, 
(2) restart the ceilometer agents on the compute [service 
openstack-ceilometer-compute restart]
(3) restart the nova agent [service openstack-nova-compute restart] 
(4) restart the collector [service openstack-ceilometer-collector restart]

The changes are disregarded. What am I doing wrong?

Thanks

Elisha Rosensweig

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [openstack-infra] [vitrage] branch marking policy

[Rosensweig, Elisha (Nokia - IL)]

Hi,

We've prepared a (local) branch with Vitrage that is *Liberty-compatible*, and 
would like to mark (tag?) the branch.

What is the standard way to do this?

Thanks,

Elisha Rosensweig, Ph.D.
R&D Director
CloudBand, Nokia 
T: +972 9793 3159


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-infra] [vitrage] branch marking policy

[Rosensweig, Elisha (Nokia - IL)]

Thanks! From a brief look, this seems exactly what we need.

Elisha

From: Joshua Hesketh [mailto:joshua.hesk...@gmail.com]
Sent: Thursday, June 30, 2016 3:51 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [openstack-infra] [vitrage] branch marking policy

Hey Elisha,

Have you looked at http://docs.openstack.org/infra/manual/drivers.html ?

Cheers,
Josh

On Thu, Jun 30, 2016 at 9:16 PM, Rosensweig, Elisha (Nokia - IL) 
mailto:elisha.rosensw...@nokia.com>> wrote:
Hi,

We've prepared a (local) branch with Vitrage that is *Liberty-compatible*, and 
would like to mark (tag?) the branch.

What is the standard way to do this?

Thanks,

Elisha Rosensweig, Ph.D.
R&D Director
CloudBand, Nokia
T: +972 9793 3159


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-infra] [vitrage] branch marking policy

[Rosensweig, Elisha (Nokia - IL)]

Hi,

I tried to push "stable/liberty" as described in the link you sent (the command 
from my environment: "git branch stable/liberty HEAD && git push gerrit 
stable/liberty"), but I get:

"! [remote rejected] stable/liberty -> stable/liberty (prohibited by Gerrit)"

Is there a problem to retroactively mark liberty branches?

Elisha


From: Rosensweig, Elisha (Nokia - IL)
Sent: Thursday, June 30, 2016 3:57 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: RE: [openstack-dev] [openstack-infra] [vitrage] branch marking policy

Thanks! From a brief look, this seems exactly what we need.

Elisha

From: Joshua Hesketh [mailto:joshua.hesk...@gmail.com]
Sent: Thursday, June 30, 2016 3:51 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [openstack-infra] [vitrage] branch marking policy

Hey Elisha,

Have you looked at http://docs.openstack.org/infra/manual/drivers.html ?

Cheers,
Josh

On Thu, Jun 30, 2016 at 9:16 PM, Rosensweig, Elisha (Nokia - IL) 
mailto:elisha.rosensw...@nokia.com>> wrote:
Hi,

We've prepared a (local) branch with Vitrage that is *Liberty-compatible*, and 
would like to mark (tag?) the branch.

What is the standard way to do this?

Thanks,

Elisha Rosensweig, Ph.D.
R&D Director
CloudBand, Nokia
T: +972 9793 3159


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] scenario evaluator not enabled by default

[Rosensweig, Elisha (Nokia - IL)]

This is on purpose. 

When Vitrage is started, it first runs a "consistency" round where it gets all 
the resources from its datasources and inserts them into the entity graph. Once 
this initial phase is over, the evaluator is run over all the entity graph to 
check for meaningful patterns based on it's templates.

The reason for this process is to avoid too much churn during the initial phase 
when Vitrage comes up. With so many changes done to the entity graph, it's best 
to wait for the initial collection phase to finish and then to do the analysis.

Elisha

> From: Yujun Zhang [mailto:zhangyujun+...@gmail.com] 
> Sent: Thursday, August 11, 2016 5:49 PM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [vitrage] scenario evaluator not enabled by 
> default
>
> Sorry for having put a url from forked repo. It should be 
> https://github.com/openstack/vitrage/commit/bdba10cb71b2fa3744e4178494fa860303ae0bbe#diff>
>  -6f1a277a2f6e9a567b38d646f19728bcL36

> But the content is the same
> --
> Yujun

> On Thu, Aug 11, 2016 at 10:43 PM Yujun Zhang  wrote:
> It seems the scenario evaluator is not enabled when vitrage is started in 
> devstack installer.
>
> I dig a bit in the history, it seems the default value for the evaluator is 
> changed from True to False > in a history commit [1].
>
> Is it breaking the starting of evaluator or I have missed some steps to 
> enable it explictily?
>
> - [1] 
> https://github.com/openzero-zte/vitrage/commit/bdba10cb71b2fa3744e4178494fa860303ae0bbe#diff-
>  6f1a277a2f6e9a567b38d646f19728bcL36

> --
> Yujun
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] reference a type of alarm in template

[Rosensweig, Elisha (Nokia - IL)]

Hi,

The "type" means where it was generated - aodh, vitrage, nagios...

I think you are looking for"name", a field that describes the actual problem. 
We should add that to our documentation to clarify.

Sent from Nine

From: Yujun Zhang 
Sent: Aug 15, 2016 16:10
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [vitrage] reference a type of alarm in template

I have a question on how to reference a type of alarm in template so that we 
can build scenarios.

In the template sample [1], an alarm entity has three keys: `category`, `type` 
and `template_id`. It seems `type` is the only information to distinguish 
different alarms. However, when an alarm is raised by aodh, it seems all alarms 
are assigned entity type `aodh` [2], so are they shown in dashboard.

Suppose we have two different types of alarms from `aodh`, e.g. 
`volume.corrupt` and `volume.deleted`. How should I reference them separately 
in a template?

×
[Screen Shot 2016-08-15 at 8.44.56 PM.png]
×

×

[1] 
https://github.com/openstack/vitrage/blob/master/etc/vitrage/templates.sample/deduced_host_disk_space_to_instance_at_risk.yaml#L8
[2] 
https://github.com/openstack/vitrage/blob/master/vitrage/datasources/aodh/transformer.py#L75

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] relationship_type in static_datasources

[Rosensweig, Elisha (Nokia - IL)]

What is the problem you are running into with mock_sync?
Elisha

From: Yujun Zhang [mailto:zhangyujun+...@gmail.com]
Sent: Tuesday, August 30, 2016 5:09 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [vitrage] relationship_type in static_datasources

Patch work in progress [1] but local test fails [2].

It seems to be caused by the mock_sync.

I'm still looking into it. Any help would be appreciated.

[1] https://review.openstack.org/#/c/362525
[2] http://pastebin.com/iepqxUAP


On Mon, Aug 29, 2016 at 4:59 PM Yujun Zhang 
mailto:zhangyujun%2b...@gmail.com>> wrote:
Thanks, Alexey. Point 1 and 3 are pretty clear.

As for point 2, if I understand it correctly, you are suggesting to modify the 
static_physical.yaml as following

entities:
 - type: switch
   name: switch-1
   id: switch-1 # should be same as name
   state: available
   relationships:
 - type: nova.host
   name: host-1
   id: host-1 # should be same as name

   is_source: true # entity is `source` in this relationship

   relation_type: attached

 - type: switch

   name: switch-2

   id: switch-2 # should be same as name

   is_source: false # entity is `target` in this relationship
   relation_type: backup
But I wonder why the static physical configuration file use a different format 
from vitrage template definitions[1]

[1] 
https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-template-format.rst

On Sun, Aug 28, 2016 at 4:14 PM Weyl, Alexey (Nokia - IL) 
mailto:alexey.w...@nokia.com>> wrote:
Hi Yujun,

In order for the static_physical to work for different datasources without the 
restrictions, you need to do the following changes:
Go to the static_physical transformer:

1.   Remove the methods: _register_relations_direction, 
_find_relation_direction_source.

2.   Add to the static_physical.yaml for each definition also a field for 
direction which will indicate the source and the destination between the 
datasources.

3.   In method: _create_neighbor, remove the usage of method 
_find_relation_direction_source, and use the new definition from the yaml file 
here to decide the edge direction.

Is it ok?


From: Yujun Zhang 
[mailto:zhangyujun+...@gmail.com]
Sent: Friday, August 26, 2016 4:22 AM

To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [vitrage] relationship_type in static_datasources

Lost in the code...It seems the datasource just construct the entities and send 
them over event bus to entity graph processor. I need to dig further to find 
out the exact point the "backup" relationship is filtered.

I think we should some how keep the validation of relationship type. It is so 
easy to make typo when creating the template manually (I did this quite 
often...).

My idea is to delegate the validation to datasource instead of enumerating all 
constants it in evaluator. I think this will introduce better extensibility. 
Any comments?

On Thu, Aug 25, 2016 at 1:32 PM Weyl, Alexey (Nokia - IL) 
mailto:alexey.w...@nokia.com>> wrote:
Hi Yujun,

You can find the names of the lables in the constants.py file.

In addition, the restriction on the physical_static datasource is done in it’s 
driver.py.

Alexey

From: Yujun Zhang 
[mailto:zhangyujun+...@gmail.com]
Sent: Thursday, August 25, 2016 4:50 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [vitrage] relationship_type in static_datasources

Hi, Ifat,

I searched for edge_labels in the project. It seems it is validated only in 
`vitrage/evaluator/template_validation/template_syntax_validator.py`. Where is 
such restriction applied in static_datasources?

--
Yujun

On Wed, Aug 24, 2016 at 3:19 PM Afek, Ifat (Nokia - IL) 
mailto:ifat.a...@nokia.com>> wrote:
Hi Yujun,

Indeed, we have some restrictions on the relationship types that can be used in 
the static datasources. I think we should remove these restrictions, and allow 
any kind of relationship type.

Best regards,
Ifat.

From: Yujun Zhang
Date: Monday, 22 August 2016 at 08:37
I'm following the sample configuration in docs [1] to verify how static 
datasources works.

It seems `backup` relationship is not displayed in the entity graph view and 
neither is it included in topology show.

There is an enumeration for edge labels [2]. Should relationship in static 
datasource be limited to it?

[1] 
https://github.com/openstack/vitrage/blob/master/doc/source/static-physical-config.rst
[2] 
https://github.com/openstack/vitrage/blob/master/vitrage/common/constants.py#L49
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openst

Re: [openstack-dev] [vitrage] relationship_type in static_datasources

[Rosensweig, Elisha (Nokia - IL)]

Yes. Please add it to the file

/workspace/dev/vitrage/vitrage/tests/resources/mock_configurations/driver/driver_switch_snapshot_dynamic.json

under the "relationships" section, just like in your commit.

If you need help understanding how to work with the mock_sync, let me know.

Elisha

From: Yujun Zhang [mailto:zhangyujun+...@gmail.com]
Sent: Tuesday, August 30, 2016 9:59 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [vitrage] relationship_type in static_datasources

I added a new key 'is_source' to static physical configuration [1] and the test 
currently fails.

Not sure we need to update mock_sync or not.

[1] 
https://review.openstack.org/#/c/362525/1/vitrage/tests/resources/static_datasources/switch_to_host_datasource.yaml

On Tue, Aug 30, 2016 at 2:53 PM Rosensweig, Elisha (Nokia - IL) 
mailto:elisha.rosensw...@nokia.com>> wrote:
What is the problem you are running into with mock_sync?
Elisha

From: Yujun Zhang 
[mailto:zhangyujun+...@gmail.com<mailto:zhangyujun%2b...@gmail.com>]
Sent: Tuesday, August 30, 2016 5:09 AM

To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [vitrage] relationship_type in static_datasources

Patch work in progress [1] but local test fails [2].

It seems to be caused by the mock_sync.

I'm still looking into it. Any help would be appreciated.

[1] https://review.openstack.org/#/c/362525
[2] http://pastebin.com/iepqxUAP


On Mon, Aug 29, 2016 at 4:59 PM Yujun Zhang 
mailto:zhangyujun%2b...@gmail.com>> wrote:
Thanks, Alexey. Point 1 and 3 are pretty clear.

As for point 2, if I understand it correctly, you are suggesting to modify the 
static_physical.yaml as following

entities:
 - type: switch
   name: switch-1
   id: switch-1 # should be same as name
   state: available
   relationships:
 - type: nova.host
   name: host-1
   id: host-1 # should be same as name

   is_source: true # entity is `source` in this relationship

   relation_type: attached

 - type: switch

   name: switch-2

   id: switch-2 # should be same as name

   is_source: false # entity is `target` in this relationship
   relation_type: backup
But I wonder why the static physical configuration file use a different format 
from vitrage template definitions[1]

[1] 
https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-template-format.rst

On Sun, Aug 28, 2016 at 4:14 PM Weyl, Alexey (Nokia - IL) 
mailto:alexey.w...@nokia.com>> wrote:
Hi Yujun,

In order for the static_physical to work for different datasources without the 
restrictions, you need to do the following changes:
Go to the static_physical transformer:

1.   Remove the methods: _register_relations_direction, 
_find_relation_direction_source.

2.   Add to the static_physical.yaml for each definition also a field for 
direction which will indicate the source and the destination between the 
datasources.

3.   In method: _create_neighbor, remove the usage of method 
_find_relation_direction_source, and use the new definition from the yaml file 
here to decide the edge direction.

Is it ok?


From: Yujun Zhang 
[mailto:zhangyujun+...@gmail.com<mailto:zhangyujun%2b...@gmail.com>]
Sent: Friday, August 26, 2016 4:22 AM

To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [vitrage] relationship_type in static_datasources

Lost in the code...It seems the datasource just construct the entities and send 
them over event bus to entity graph processor. I need to dig further to find 
out the exact point the "backup" relationship is filtered.

I think we should some how keep the validation of relationship type. It is so 
easy to make typo when creating the template manually (I did this quite 
often...).

My idea is to delegate the validation to datasource instead of enumerating all 
constants it in evaluator. I think this will introduce better extensibility. 
Any comments?

On Thu, Aug 25, 2016 at 1:32 PM Weyl, Alexey (Nokia - IL) 
mailto:alexey.w...@nokia.com>> wrote:
Hi Yujun,

You can find the names of the lables in the constants.py file.

In addition, the restriction on the physical_static datasource is done in it’s 
driver.py.

Alexey

From: Yujun Zhang 
[mailto:zhangyujun+...@gmail.com<mailto:zhangyujun%2b...@gmail.com>]
Sent: Thursday, August 25, 2016 4:50 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [vitrage] relationship_type in static_datasources

Hi, Ifat,

I searched for edge_labels in the project. It seems it is validated only in 
`vitrage/evaluator/template_validation/template_syntax_validator.py`. Where is 
such restriction applied in static_datasources?

--
Yujun

On Wed, Aug 24, 2016 at 3:19 PM Afek, Ifat (Nokia - IL) 
mailto:ifat.a...@nokia.com>> wrote:
Hi Yujun,

Indeed, we have some restrictions on th

Re: [openstack-dev] [vitrage] how to use mock driver

[Rosensweig, Elisha (Nokia - IL)]

Hi,


· In Vitrage Datasources, we can have a different input format for 
snapshots and updates. Thus, we need a different JSON file for each.

· Also, as part of the Mock feature, we need to support (for each 
resource) things that will be static, such as it’s name, and things that change 
over time, such as timestamps. We support this partially via different JSON 
files. In general, the dynamic file (marked with “D”) overwrites the static one 
(marked with “S”).

· In the code you can further inject specific fields you want to have 
for a specific test, in addition to the JSON files. See examples in 
test_scenario_evaluator.py.

Elisha

From: Yujun Zhang [mailto:zhangyujun+...@gmail.com]
Sent: Monday, December 12, 2016 8:23 AM
To: OpenStack Development Mailing List (not for usage questions) 

Subject: [openstack-dev] [vitrage] how to use mock driver

Is there any documentation on how to use mock driver for unit testing?

It seems it generates fake events from json spec but what is the different 
between

- `xxx_snapshot_X.json` and `xxx_dynamic_X.json`
- `xxx_S` and `xxx_D`

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] how to use mock driver

[Rosensweig, Elisha (Nokia - IL)]

Yes. We actually had to remove the regex generation support a few months ago, 
since the python package we were using – exrex – was not one that OpenStack 
supported.

From: Yujun Zhang [mailto:zhangyujun+...@gmail.com]
Sent: Tuesday, December 13, 2016 8:32 AM
To: OpenStack Development Mailing List (not for usage questions) 

Subject: Re: [openstack-dev] [vitrage] how to use mock driver

Elisha, thanks for the explanation. The difference is clear to me now.

If I understand it correctly, the regular expression in spec JSON is for 
information only. It is never compiled into a `re` object.

The actual values are generated in `static_info_parsers` from the `mapping`. 
The regular expression is neither used as a value template nor for value 
validation.

Is that right?

On Mon, Dec 12, 2016 at 8:47 PM Rosensweig, Elisha (Nokia - IL) 
mailto:elisha.rosensw...@nokia.com>> wrote:
Hi,


• In Vitrage Datasources, we can have a different input format for 
snapshots and updates. Thus, we need a different JSON file for each.

• Also, as part of the Mock feature, we need to support (for each 
resource) things that will be static, such as it’s name, and things that change 
over time, such as timestamps. We support this partially via different JSON 
files. In general, the dynamic file (marked with “D”) overwrites the static one 
(marked with “S”).

• In the code you can further inject specific fields you want to have 
for a specific test, in addition to the JSON files. See examples in 
test_scenario_evaluator.py.

Elisha

From: Yujun Zhang 
[mailto:zhangyujun+...@gmail.com<mailto:zhangyujun%2b...@gmail.com>]
Sent: Monday, December 12, 2016 8:23 AM
To: OpenStack Development Mailing List (not for usage questions) 
mailto:openstack-dev@lists.openstack.org>>
Subject: [openstack-dev] [vitrage] how to use mock driver

Is there any documentation on how to use mock driver for unit testing?

It seems it generates fake events from json spec but what is the different 
between

- `xxx_snapshot_X.json` and `xxx_dynamic_X.json`
- `xxx_S` and `xxx_D`

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev