from:"Yujun Zhang"

Re: [openstack-dev] [vitrage] rules in vitrage_aggregated_state()

2018-01-11 Thread Yujun Zhang (ZTE)

I have almost understood it thanks to your explanation.

The confusion is mainly caused by the naming. I guess the main reason is
that the scope evolves but the naming is not updated with it.

For example

   1. `vitrage_aggregated_state` actually applies for both resource state
   and alarm severity as defined in `value_properties`. So
   `vitrage_aggregated_values` could be a better name.
   2. For data source in static configuration, we may use `static.yaml` as
   a fallback. The name `default.yaml` will mislead user that it should be
   applied to data source configured in "types" but without a values
   configuration.
   3. The UNDEFINED value is named UNDEFINED_DATASOURCE = "undefined
   datasource", it is not a consistent type of severity and state enumeration.
   4. The behavior for data source defined in static without values
   configuration and data source defined in "types" without values
   configuration are inconsistent. The former will fallback to `default.yaml`
   but the latter will lead to undefined value.

I know it is there for historical reasons and current developers may
already get used to it, but it gives new contributors too many surprises.

What do you think? Shall we amend them?

On Tue, Jan 9, 2018 at 11:29 PM Afek, Ifat (Nokia - IL/Kfar Sava) <
ifat.a...@nokia.com> wrote:

> Hi,
>
>
>
> I agree that the code is confusing…
>
>
>
> This is part of a change that was made in order to support default states
> for static entities. For example, in the static configuration yaml file you
> can add entities of types ‘switch’ and ‘br-ex’. In the past, in order to
> support states for these new types, you needed to add switch.yaml and
> br-ex.yaml under /etc/vitrage/datasources_values, which you would most
> likely copy from another datasource. Now, we have under
> /etc/vitrage/datasources_values a default.yaml file that is used for all
> static entities.
>
>
>
> Back to the code, I believe this is the logic:
>
>
>
> · If the datasource is part of ‘types’ (as defined in
> vitrage.conf) and has states configuration – use it. This is the normal
> behavior.
>
> · If the datasource is not part of ‘types’, we understand that it
> was defined in a static configuration file. Use the default states
> configuration. I assume that it is somehow handled in the first part of the
> if statement (I’m not so familiar with that code)
>
> · If neither is true – it means that the datasource is “real” and
> not static, and was defined in vitrage.conf types. And it also means that
> its states configuration is missing, so the state is UNDEFINED.
>
>
>
> And to your questions:
>
>
>
>1. the data source is not defined -> the default states should be used
>2. data source defined but state config not exist -> UNDEFINED state
>3. data source defined, state config exist but the state is not found.
>-> I believe that somewhere in the first part of the if statement you will
>    get UNDEFINED
>
>
>
>
>
> Hope that’s more clear now. It might be a good idea to add some comments
> to that function…
>
>
>
> Best Regards,
>
> Ifat.
>
>
>
>
>
> *From: *"Yujun Zhang (ZTE)" <zhangyujun+...@gmail.com>
> *Reply-To: *"OpenStack Development Mailing List (not for usage
> questions)" <openstack-dev@lists.openstack.org>
> *Date: *Tuesday, 9 January 2018 at 8:34
> *To: *"OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> *Subject: *Re: [openstack-dev] [vitrage] rules in
> vitrage_aggregated_state()
>
>
>
> Forgot to paste the link to the related code:
>
>
>
>
> https://git.openstack.org/cgit/openstack/vitrage/tree/vitrage/entity_graph/mappings/datasource_info_mapper.py#n61
>
>
>
>
>
>
>
> On Tue, Jan 9, 2018 at 2:34 PM Yujun Zhang (ZTE) <zhangyujun+...@gmail.com>
> wrote:
>
> Hi root causers
>
>
>
> I have been inspecting the code about aggregated state recently and have a
> question regarding the rules.
>
>
>
> The "not" operator in the if clause confuses me. If it is not a configured
> data source, how do we apply the aggregation rules? It seems this is
> handled in else clause.
>
>
>
> if datasource_name in self.datasources_state_confs or \
>
> datasource_name *not* in self.conf.datasources.types: 
>...
>
> else:
>
> self.category_normalizer[vitrage_category].set_aggregated_value(
>
> new_vertex, self.UNDEFINED_DATASOURCE)
>
> self.category_normalizer[vitrage_category].set_operat

Re: [openstack-dev] [vitrage] rules in vitrage_aggregated_state()

2018-01-08 Thread Yujun Zhang (ZTE)

Forgot to paste the link to the related code:

https://git.openstack.org/cgit/openstack/vitrage/tree/vitrage/entity_graph/mappings/datasource_info_mapper.py#n61



On Tue, Jan 9, 2018 at 2:34 PM Yujun Zhang (ZTE) <zhangyujun+...@gmail.com>
wrote:

> Hi root causers
>
> I have been inspecting the code about aggregated state recently and have a
> question regarding the rules.
>
> The "not" operator in the if clause confuses me. If it is not a configured
> data source, how do we apply the aggregation rules? It seems this is
> handled in else clause.
>
> if datasource_name in self.datasources_state_confs or \
> datasource_name *not* in self.conf.datasources.types: 
>...
>
> else:
> self.category_normalizer[vitrage_category].set_aggregated_value(
> new_vertex, self.UNDEFINED_DATASOURCE)
> self.category_normalizer[vitrage_category].set_operational_value(
> new_vertex, self.UNDEFINED_DATASOURCE)
>
>
> There are some test case describing the expected behavior. But I couldn't 
> understand the design philosophy behind it. What is expected when
>
>1. the data source is not defined
>2. data source defined but state config not exist
>3. data source defined, state config exist but the state is not found.
>
> Could somebody shed some light on it?
>
>
>
> --
> Yujun Zhang
>


-- 
Yujun Zhang
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [vitrage] rules in vitrage_aggregated_state()

2018-01-08 Thread Yujun Zhang (ZTE)

Hi root causers

I have been inspecting the code about aggregated state recently and have a
question regarding the rules.

The "not" operator in the if clause confuses me. If it is not a configured
data source, how do we apply the aggregation rules? It seems this is
handled in else clause.

if datasource_name in self.datasources_state_confs or \
datasource_name *not* in self.conf.datasources.types:
  ...

else:
self.category_normalizer[vitrage_category].set_aggregated_value(
new_vertex, self.UNDEFINED_DATASOURCE)
self.category_normalizer[vitrage_category].set_operational_value(
new_vertex, self.UNDEFINED_DATASOURCE)


There are some test case describing the expected behavior. But I
couldn't understand the design philosophy behind it. What is expected
when

   1. the data source is not defined
   2. data source defined but state config not exist
   3. data source defined, state config exist but the state is not found.

Could somebody shed some light on it?



-- 
Yujun Zhang
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] support graphviz in document publishing

2017-11-14 Thread Yujun Zhang (ZTE)

On Tue, Nov 14, 2017 at 5:05 AM Antoine Musso <has...@free.fr> wrote:

> Hello,
>
> I had some success with http://blockdiag.com/ which, IIRC, is pure
> python. It supports various kind of graphs (block, sequences, network ...).
>
> The few I did were for Zuul:
> https://docs.openstack.org/infra/zuul/user/gating.html
>
> In sphinx that looks like:
>
> .. blockdiag::
>
>   blockdiag foo {
> node_width = 40;
> span_width = 40;
> A <- B <- C <- D <- E;
>   }
>
>
> sphinxcontrib-blockdiag
> https://pypi.python.org/pypi/sphinxcontrib-blockdiag
>
> Might be worth looking at if you want something simpler than graphviz
> and to not depend on it.
>

Really nice diagram and syntax. I will surely consider it for some workflow
in the specification.

Thank you for the recommendation, Antoine

-- 
Yujun Zhang
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] support graphviz in document publishing

2017-11-14 Thread Yujun Zhang (ZTE)

On Mon, Nov 13, 2017 at 5:41 PM Andreas Jaeger <a...@suse.com> wrote:

> You can easily include them for any repository - add the python
> requirements to test-requirments and add the binary requirements to
> bindep.txt - as explained in
> https://docs.openstack.org/infra/manual/drivers.html#package-requirements


Thanks Andreas.

Besides the dependency you mentioned, I just realized that we need also to
add the extension to sphinx conf[1] to make it effective.

https://review.openstack.org/#/c/518196/8/doc/source/conf.py
-- 
Yujun Zhang
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [infra] support graphviz in document publishing

2017-11-13 Thread Yujun Zhang (ZTE)

Hi, all

Sometimes a picture worths thousands word. I wonder if it is possible to
support graphviz in https://docs.openstack.org/

What I expect is to include a dot file and render it as a picture in HTML
output. The syntax is simple

.. graphviz:: aggregate-equivalent-alarms.files/example.dot

And should be supported by sphinx graphviz plugin[2]

See full example[1]. Currently I render the picture offline and include it
manually in the document. It would be much more convenient for developer if
the extension is added to document build job.

[1]:
https://review.openstack.org/#/c/518196/5/proposals/aggregate-equivalent-alarms.rst
[2]: http://www.sphinx-doc.org/en/stable/ext/graphviz.html

-- 
Yujun Zhang
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [oslo][oslo.middleware] when will next release arrive

2017-09-03 Thread Yujun Zhang (ZTE)

Hi oslo cores,

It has been more than one month since last release of oslo.middleware
(3.30.0 on July 18th). I wonder when will next release arrive.

I'm asking it because some bug fix[1] is depending the patch sets to be
released[2].

[1]: https://review.openstack.org/#/c/487388/
[2]: https://review.openstack.org/#/c/487454/

-- 
Yujun Zhang
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [vitrage] status of blueprint crud-templates

2017-07-02 Thread Yujun Zhang (ZTE)

Hi, root causers :-)

We recently starts to looking into the CRUD of templates to allow loading
them when vitrage is up. It seems there is already one blueprint
registered[1] for exactly the same purpose.

Is there any detailed design about it? Do you have already some ideas in
mind?

[1] https://blueprints.launchpad.net/vitrage/+spec/crud-templates

-- 
Yujun Zhang
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] Nominating Yujun Zhang to Vitrage core

2017-06-26 Thread Yujun Zhang (ZTE)

Thanks all.

It is my pleasure to work with you in Vitrage project :-)

--
Yujun

On Mon, Jun 26, 2017 at 4:34 PM Afek, Ifat (Nokia - IL/Kfar Sava) <
ifat.a...@nokia.com> wrote:

> Hi,
>
>
>
> I added Yujun Zhang to the vitrage-core team. Yujun, Welcome J
>
>
>
> Best Regards,
>
> Ifat.
>
>
>
> *From: *דני אופק <dany.of...@gmail.com>
> *Date: *Monday, 26 June 2017 at 11:00
>
>
>
> +1
>
>
>
> On Mon, Jun 26, 2017 at 7:45 AM, Weyl, Alexey (Nokia - IL/Kfar Sava) <
> alexey.w...@nokia.com> wrote:
>
> +1
>
>
>
> > -Original Message-
> > From: Afek, Ifat (Nokia - IL/Kfar Sava) [mailto:ifat.a...@nokia.com]
> > Sent: Sunday, June 25, 2017 3:18 PM
> > To: OpenStack Development Mailing List (not for usage questions)
> > <openstack-dev@lists.openstack.org>
> >
> > Hi,
> >
> > I’d like to nominate Yujun Zhang to the Vitrage core team.
> >
> > In the last year Yujun has made a significant contribution in
> Vitrage[1], both
> > by adding new features and by reviewing other people’s work. He has an
> > extensive knowledge of the Vitrage architecture and code, and I believe
> he
> > would make a great addition to our team.
> >
> > Best Regards,
> > Ifat.
> >
> > [1]
> > https://review.openstack.org/#/q/owner:zhang.yujunz%2540zte.com.cn+pr
> > ojects:openstack/vitrage+is:merged
> >
>
> ______
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-- 
Yujun Zhang
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] First Vitrage Pike release by the end of this week

2017-06-21 Thread Yujun Zhang (ZTE)

Is it done yet?

How to fetch the released version for downstream developing?

On Tue, Jun 6, 2017 at 2:17 PM Afek, Ifat (Nokia - IL/Kfar Sava) <
ifat.a...@nokia.com> wrote:

> Hi,
>
> Pike-2 milestone is at the end of this week, and although we are not
> working by the milestones model (we are working by a
> cycle-with-intermediary model) we need to have the first Vitrage Pike
> release by the end of this week.
>
> I would like to release vitrage, python-vitrageclient and
> vitrage-dashboard tomorrow. Any objections? Please let me know if you think
> something has to be changed/added before the release.
>
> Also, we need to add release notes for the newly added features. This list
> includes (let me know if I missed something):
>
> vitrage
> • Vitrage ID
> • Support ‘not’ operator in the evaluator templates
> • Performance improvements
> • Support entity equivalences
> • SNMP notifier
>
> python-vitrageclient
> • Multi tenancy support
> • Resources API
>
> vitrage-dashboard
> • Multi tenancy support – Vitrage in admin menu
> • Added ‘search’ option in the entity graph
>
> Please add a release notes file for each of your features (I’ll send an
> explanation in a separate mail), or send me a few lines of the feature’s
> description and I’ll add it.
>
> Thanks,
> Ifat.
>
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-- 
Yujun Zhang
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] Gate issues

2017-06-01 Thread Yujun Zhang (ZTE)

We have encountered similar issue also in OPNFV.

It seems to be a problem of setuptools 36.0.0 and it is now removed from
pypi. Hope it resolves the vitrage gate tests as well.

See discussion in https://github.com/pypa/setuptools/issues/1042


On Thu, Jun 1, 2017 at 11:08 PM Afek, Ifat (Nokia - IL/Kfar Sava) <
ifat.a...@nokia.com> wrote:

> Hi,
>
>
>
> Note that we are currently having problems with the Vitrage gate tests,
> related to python-setuptools. Other projects experience similar problems.
> We hope to fix it by the beginning of next week.
>
>
>
> Best Regards,
>
> Ifat.
>
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-- 
Yujun Zhang
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] error handling

2017-06-01 Thread Yujun Zhang (ZTE)

On Thu, Jun 1, 2017 at 10:49 PM Afek, Ifat (Nokia - IL/Kfar Sava) <
ifat.a...@nokia.com> wrote:

So for now we agree that we need to add a UI for configuration information
> and datasources status.
>
>
Sounds good. In order to implement in UI, we shall also need api to expose
them right?

-- 
Yujun Zhang
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] error handling

2017-05-30 Thread Yujun Zhang (ZTE)

On Tue, May 30, 2017 at 3:59 PM Afek, Ifat (Nokia - IL/Kfar Sava) <
ifat.a...@nokia.com> wrote:

> Hi Yujun,
>
>
>
> You started an interesting discussion. I think that the distinction
> between an operational error and a programmer error is correct and we
> should always keep that in mind.
>
>
>
> I agree that having an overall design for error handling in Vitrage is a
> good idea; but I disagree that until then we better let it crash.
>
>
>
> I think that Vitrage is made out of many pieces that don’t necessarily
> depend on one another. For example, if one datasource fails, everything
> else can work as usual – so why crash? Similarly, if one template fails to
> load, all other templates can still be activated.
>

This usually or always happens during initialization phase, doesn't it? It
is a period with human inspecting and should be detected in the deployment
or user acceptance test. So if something fails, it is better to isolate
them before continue running, e.g. correct the invalid template, invalid
data source configuration or remove the template and disable the data
source. This is because such error is permanent and they won't recover
automatically.

Here we need to distinguish the error that data source is temporarily
unavailable due to network connection issue or data source not up yet. In
this case, I agree we'd better start the rest component and perform a retry
periodically until it recovers.

> Another aspect is that the main purpose of Vitrage is to provide insights.
> In case of a failure in one datasource/template, some of the insights might
> be missing. But this will not lead to inaccurate behavior or to wrong
> actions being executed in the system. IMO, we should give the user as much
> information as possible given that we have only part of the input.
>

I agree, if enough insights could be provided by the running system. We can
improve the handling of permanent error. What is even better is supporting
of a hot load for the components and templates.

What I don't like much is sometimes errors are handled but without enough
details. In this case, a crash with trace stack is more useful than a user
"friendly" message like "failed to start xxx component" or "invalid
configuration file" (I'm not talking about vitrage, it is quite common in
many projects)

My preference is "good error handling" > "no error handling" > "bad error
handling". Though it is difficult to distinguish what is a good error
handling and what is bad...

Regarding the use cases that you mentioned:
>
>
>
>1. invalid configuration file
>
> [Ifat] This should depend on the specific configuration. If keystone is
> misconfigured, nothing will work of course. But if for example Zabbix is
> misconfigured, Vitrage should work and show the topology and the non-Zabbix
> alarms.
>

Agree. It should be handled in a different way regarding what kind of error
and how critical it is.

>
>1. failed to communicate with data source
>
> [Ifat] I think that the error should be logged, and all other datasources
> should work as usual.
>

Yes, and it would be good to have a retry mechanism

>
>1. malformed data from data source
>
> [Ifat] I think that the error should be logged, and all other datasources
> should work as usual. This problem means we must modify the code in the
> datasource itself, but until then Vitrage should work, right?
>
Yes, I think it is possible when the data source version changes and we
should discard the data and indicate the error. The other part should not
be affected.

>1. failed to execute an action
>
> [Ifat] Again, that’s a problem that requires code changes; but why fail
> other actions?
>

What I meant here is temporary failure, e.g. when you try to mark host down
but not able to reach it due to network connection issue or other reasons

>1. ...
>
> BTW, it might be a good idea to add API/UI for showing the configuration
> and the status of the datasources. We all know that errors in the log files
> are often ignored…
>

Sure, the errors I mentioned above is what the system operators could
encounter even with a correct configuration and not related to software
bugs. Display them in UI would be very helpful. The log files are more for
the engineers to analyse the root cause.

> Best Regards,
>
> Ifat.
>
>
>
>
>
> *From: *"Yujun Zhang (ZTE)" <zhangyujun+...@gmail.com>
> *Reply-To: *"OpenStack Development Mailing List (not for usage
> questions)" <openstack-dev@lists.openstack.org>
> *Date: *Monday, 29 May 2017 at 16:13
> *To: *"OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
>

[openstack-dev] [vitrage] error handling

2017-05-29 Thread Yujun Zhang (ZTE)

Brought up by a recent code review, I think it worth a thorough discussion
about the error handling rule.

I once read an article[1] from Joyent and it impressed me on the
distinguish between *Operational* errors vs. *programmer* errors. The
article is written for nodejs, but the principle also applies for other
programming language.

The basic rule recommended by Joyent is
Handling operational errors
(Not) handling programmer errors
There is also one rule in openstack style guide line[2] close to this idea.

[H201] Do not write except:, use except Exception: at the very least. When
catching an exception you should be as specific so you don’t mistakenly
catch unexpected exceptions.

I do think before we have a well designed error handling, it is better to
let it crash. It is dangerous to hide the errors and keep the system
running in undetermined states.

So the question is *what kind of operational errors are we facing in
vitrage?* I can think of something like

   1. invalid configuration file
   2. failed to communicate with data source
   3. malformed data from data source
   4. failed to execute an action
   5. ...

Maybe this could be the first step for the error handling design.

[1]: https://www.joyent.com/node-js/production/design/errors
[2]: https://docs.openstack.org/developer/hacking/

-- 
Yujun Zhang
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] trouble installing Nagios on devstack on ubuntu 16.04 ...

2017-05-03 Thread Yujun Zhang (ZTE)

One easy way could be writing a scenario to raise deduced alarm based on a
simple rule, e.g. when a host is discovered, raise an alarm saying host is
up.

Waines, Greg <greg.wai...@windriver.com>于2017年5月4日 周四04:35写道：

> I don’t think I saw any responses to this.
>
>
>
> Alternative question ... so I’ve got vitrage up and running fine ...
>
>
>
> What’s the easiest way to generate an alarm against a host ?   ( OTHER
> than NAGIOS, due to problem in original email ) ???
>
>
>
> let me know any ideas,
>
> Greg.
>
>
>
>
>
> *From: *Greg Waines <greg.wai...@windriver.com>
> *Date: *Tuesday, May 2, 2017 at 9:03 AM
> *To: *"openstack-dev@lists.openstack.org" <
> openstack-dev@lists.openstack.org>
> *Subject: *[openstack-dev] [vitrage] trouble installing Nagios on
> devstack on ubuntu 16.04 ...
>
>
>
> Hey ... I’m working thru the ‘Vitrage - Getting Started Guide’
>
>
>
> https://docs.openstack.org/developer/vitrage/vitrage-first_steps.html
>
>
>
> Was able to get vitrage up and running and enabled in horizon ... on
> ubuntu 16.04 .
>
>  ( I tried on ubuntu 14.04 and ‘./stack.sh’ warned that it had not
> been tested on trusty (14.04), I FORCE=yes it ... but it failed. )
>
>
>
>
>
> Now trying to install Nagios in devstack
>
>
> https://docs.openstack.org/developer/vitrage/nagios-devstack-installation.html
>
>
>
> BUT it doesn’t seem like there is an OMD package available for ubuntu
> 16.04 ... and the trusty (14.04) package won’t install due to dependency
> issues.
>
>
>
>
>
>
>
> Any suggestions ?
>
>
>
> Greg.
>
>
>
>
>
>
> ______
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-- 
Yujun Zhang
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [vitrage] use case repository

2017-04-17 Thread Yujun Zhang (ZTE)

Hi root causers,

Use case repository is considered as one of the very important subject to
work on in Pike release[1].

I wonder what kind of use cases we have deployed/validated in vitrage, no
matter in production or experimental, and where can I find related
information?

[1]: https://wiki.openstack.org/wiki/Vitrage/RoadMap#Very_important

-- 
Yujun Zhang
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [stackalytics] user correction seems not effective

2017-03-19 Thread Yujun Zhang (ZTE)

Hi, Trinath

What failure are you referring to exactly? It seems all jobs passed
normally.

I know there are some error message in the console logs, but that seems to
be expected failures for negative test cases.

On Mon, Mar 20, 2017 at 12:16 PM Trinath Somanchi <trinath.soman...@nxp.com>
wrote:

Jenkins is throwing some Failures in new submissions. That might be an
issue.

Get Outlook for iOS <https://aka.ms/o0ukef>

------
*From:* Yujun Zhang (ZTE) <zhangyujun+...@gmail.com>
*Sent:* Monday, March 20, 2017 8:47:30 AM
*To:* shak...@gmail.com
*Cc:* OpenStack Development Mailing List (not for usage questions); 张玉军
*Subject:* [openstack-dev] [stackalytics] user correction seems not
effective

Hi, Ilya

I submitted a patch for user correction[1] several months ago. It is
supposed to reset the Email list of user `zhangyujun`. But it seems not
effective from the response of stackalytics api.

curl http://stackalytics.com/api/1.0/users/zhangyujun

{"user": {"launchpad_id": "zhangyujun", "user_id": "zhangyujun", "seq":
65434, "company_link": "EasyStack", "text": "Zhang
Yujun", "companies": [{"company_name": "ZTE Corporation", "end_date":
1474588800}, {"company_name": "EasyStack", "end_date": 0}], "id":
"zhangyujun", "static": true, "gerrit_id": "zhangyujun", "user_name":
"Zhang Yujun", "emails": [*"zhangyujun.d...@gmail.com
<zhangyujun.d...@gmail.com>", "zhangyu...@gmail.com <zhangyu...@gmail.com>"*,
"284517...@qq.com", "zhangyujun+...@gmail.com", "yujun.zh...@easystack.cn",
"zhang.yuj...@zte.com.cn"]}}

The email address in red should be removed if the patch works as expected.

Is there any way I can make further investigation on this issue? The log
message from the stackalytics server might be helpful.

[1]: https://review.openstack.org/#/c/426502/

-- 
Yujun Zhang
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-- 
Yujun Zhang
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [stackalytics] user correction seems not effective

2017-03-19 Thread Yujun Zhang (ZTE)

Hi, Ilya

I submitted a patch for user correction[1] several months ago. It is
supposed to reset the Email list of user `zhangyujun`. But it seems not
effective from the response of stackalytics api.

curl http://stackalytics.com/api/1.0/users/zhangyujun

{"user": {"launchpad_id": "zhangyujun", "user_id": "zhangyujun", "seq":
65434, "company_link": "EasyStack", "text": "Zhang
Yujun", "companies": [{"company_name": "ZTE Corporation", "end_date":
1474588800}, {"company_name": "EasyStack", "end_date": 0}], "id":
"zhangyujun", "static": true, "gerrit_id": "zhangyujun", "user_name":
"Zhang Yujun", "emails": [*"zhangyujun.d...@gmail.com
<zhangyujun.d...@gmail.com>", "zhangyu...@gmail.com <zhangyu...@gmail.com>"*,
"284517...@qq.com", "zhangyujun+...@gmail.com", "yujun.zh...@easystack.cn",
"zhang.yuj...@zte.com.cn"]}}

The email address in red should be removed if the patch works as expected.

Is there any way I can make further investigation on this issue? The log
message from the stackalytics server might be helpful.

[1]: https://review.openstack.org/#/c/426502/

-- 
Yujun Zhang
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [vitrage] what is "API to register on Vitrage notificaitons"

2017-03-16 Thread Yujun Zhang (ZTE)

Hi root causers

I want to confirm some detail about one subject in vitrage Pike roadmap


   - API to register on Vitrage notificaitons


Is it linked to blueprint
https://blueprints.launchpad.net/vitrage/+spec/configurable-notifications ?

-- 
Yujun Zhang
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] New Vitrage project mascot

2017-01-29 Thread Yujun Zhang

Not remembering the original design.

This one looks good to me :-)

On Sun, Jan 29, 2017 at 8:18 PM Afek, Ifat (Nokia - IL) 
wrote:

> Hi,
>
>
>
> This is the new draft of our Giraffe mascot. The illustrators tried to
> make its skin look more like stained glass.
>
> What do you think?
>
>
>
> Ifat.
>
>
>
>
>
>
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [stackalytics] how to remove obsoleted email address from one id

2017-01-28 Thread Yujun Zhang

Stackalytics cores:

I reported a bug on ownership of commits in stackalytics months ago[1]. It
seems to be caused by obsoleted email address in user default data.

In `user_processor`, it merges email addresses from `default_data.json` to
runtime storage[2]. It seems removing email address from user profile is
not possible.

*This will cause a statistic data error when user changes his launchpad id.*

My commits submitted with email address "zhang.yuj...@zte.com.cn" are now
updated to EasyStack instead of ZTE Corporation[3].

Below is the detailed analysis:

I modified my launchpad id from `zhangyujun` to `yujunz` and forced an
association of my email address "zhang.yuj...@zte.com.cn" to `yujunz` by
patching `default_data.json`[4].

Now I realized the association between `zhang.yuj...@zte.com.cn
` still exists in runtime storage. I found my
commits associated to `zhangyujun` and `yujunz` randomly even after trying
to rewrite the old user id[5]

So when another user registered with my old launchpad id[6], it started to
cause statistic chaos not only in user but also in company.

My questions is: *is there any way to remove the obsolete email address*?

[1]: https://bugs.launchpad.net/stackalytics/+bug/1634020
[2]:
http://git.openstack.org/cgit/openstack/stackalytics/tree/stackalytics/processor/user_processor.py#n99
[3]:
http://stackalytics.com/?project_type=opnfv-group=commits_id=zhangyujun
[4]: https://review.openstack.org/#/c/365375/1/etc/default_data.json
[5]: https://review.openstack.org/#/c/384000/1/etc/default_data.json
[6]: https://review.openstack.org/#/c/384801/1/etc/default_data.json
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [infra][mitmstack] initial member of mitmstack groups

2017-01-19 Thread Yujun Zhang

Hi, Infra team,

Could you please help add me  as initial member in
mitmstack-core 
and mitmstack-release
? Thank you.

--
Yujun
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Vitrage] About alarms reported by datasource and the alarms generated by vitrage evaluator

2017-01-16 Thread Yujun Zhang

Sounds good.

Have you created an etherpad page for collecting topics, Ifat?

On Mon, Jan 16, 2017 at 10:43 PM Afek, Ifat (Nokia - IL) <
ifat.a...@nokia.com> wrote:

>
>
> *From: *Yujun Zhang <zhangyujun+...@gmail.com>
> *Date: *Sunday, 15 January 2017 at 17:53
>
>
>
> About fault and alarm, what I was thinking about the causal/deducing chain
> in root cause analysis.
>
>
>
> Fault state means the resource is not fully functional and it is evaluated
> by related indicators. There are alarms on events like power loss or
> measurands like CPU high, memory low, temperature high. There are also
> alarms based on deduced state, such as "host fault", "instance fault".
>
>
>
> So an example chain would be
>
> · "FAULT: power line cut off" =(monitor)=> "ALARM: host power
> loss" =(inspect)=> "FAULT: host is unavailable" =(action)=> "ALARM: host
> fault"
>
> · "FAULT: power line cut off" =(monitor)=> "ALARM: host power
> loss" =(inspect)=> "FAULT: host is unavailable" =(inspect)=> "FAULT:
> instance is unavailable" =(action)=> "ALARM: instance fault"
>
> If we omit the resource, then we get the causal chain as it is in Vitrage
>
> · "ALARM: host power loss" =(causes)=> "ALARM: host fault"
>
> · "ALARM: host power loss" =(causes)=> "ALARM: instance fault"
>
> But what the user care about might be there "FAULT: power line cut off"
> causes all these alarms. What I haven't made clear yet is the equivalence
> between fault and alarm.
>
>
>
> I may have made it more complex with my *immature* thoughts. It could be
> even more complex if we consider multiple upstream causes and downstream
> outcome. It may be an interesting topic to be discussed in design session.
>
>
>
>
>
> [Ifat] I agree. Let’s discuss this in the next design session we’ll have
>
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Vitrage] About alarms reported by datasource and the alarms generated by vitrage evaluator

2017-01-15 Thread Yujun Zhang

About fault and alarm, what I was thinking about the causal/deducing chain
in root cause analysis.

Fault state means the resource is not fully functional and it is evaluated
by related indicators. There are alarms on events like power loss or
measurands like CPU high, memory low, temperature high. There are also
alarms based on deduced state, such as "host fault", "instance fault".

So an example chain would be

   - "FAULT: power line cut off" =(monitor)=> "ALARM: host power loss"
   =(inspect)=> "FAULT: host is unavailable" =(action)=> "ALARM: host fault"
   - "FAULT: power line cut off" =(monitor)=> "ALARM: host power loss"
   =(inspect)=> "FAULT: host is unavailable" =(inspect)=> "FAULT: instance is
   unavailable" =(action)=> "ALARM: instance fault"

If we omit the resource, then we get the causal chain as it is in Vitrage

   - "ALARM: host power loss" =(causes)=> "ALARM: host fault"
   - "ALARM: host power loss" =(causes)=> "ALARM: instance fault"

But what the user care about might be there "FAULT: power line cut off"
causes all these alarms. What I haven't made clear yet is the equivalence
between fault and alarm.

I may have made it more complex with my *immature* thoughts. It could be
even more complex if we consider multiple upstream causes and downstream
outcome. It may be an interesting topic to be discussed in design session.

On Sun, Jan 15, 2017 at 9:21 PM Afek, Ifat (Nokia - IL) 
wrote:

> Hi Yinliyin,
>
>
>
> There are two use cases:
>
> One is yours, where you have a single monitor that generates “real”
> alarms, and Vitrage that generates deduced alarms.
>
> Another is where someone has a few monitors, and there might be a
> collision/equivalence between their alarms.
>
>
>
> The solution that you suggested might solve the first use case, but I
> wouldn’t want to ignore the second one, which is also valid.
>
>
>
> Regarding some of your specific suggestions:
>
> 1.   In templates, we only define the alarm entity for the datasource
> that the alarm is reported by, such as Nagios.
>
> [Ifat] This will only work for a single monitor.
>
>2.  When evaluator deduce an alarm, it would raise the alarm with
> the type set to be the datasource that would report the alarm, not be
> vitrage.
>
> [Ifat] I don’t think this is right. In Vitrage Alarm view in the UI,
> displaying the deduced alarm as “Nagios” is misleading, since Nagios did
> not report this alarm.
>
>
>
> I can think of a solution that is specific to the deduced alarms case,
> where we will replace a Vitrage alarm with a “real” alarm whenever there is
> a collision. This solution is easier, but we should carefully examine all
> use cases to make sure there is no ambiguity. However, for the more general
> use case I would prefer the option that we discussed in a previous mail, of
> having two (or more) alarms connected with a ‘equivalent’ relationship.
>
>
>
> What do you think?
>
> Ifat.
>
>
>
>
>
> *From: *"yinli...@zte.com.cn" 
> *Date: *Saturday, 14 January 2017 at 09:57
>
> · It won’t solve the general problem of two different monitors
> that raise the same alarm
>
> ·   [yinliyin] Generally, we would only deploy one monitor for a
> same alarm.
>
> · It won’t solve possible conflicts of timestamp and severity
> between different monitors
>
> ·  [yinliyin] Please see the following contents.
>
> · It will make the decision of when to delete the alarm more
> complex (delete it when the deduced alarm is deleted? When Nagios alarm is
> deleted? both? And how to change the timestamp and severity in these cases?)
>
> ·  [yinliyin] Please see the following contents.
>
>The following is the basic idea of solving the problem in this
> situation:
>
>1.  In templates, we only define the alarm entity for the
> datasource that the alarm is reported by, such as Nagios.
>
>2.  When evaluator deduce an alarm, it would raise the alarm with
> the type set to be the datasource that would report the alarm, not be
> vitrage.
>
>3.  When entity_graph get the events from the "evaluator_queue"(all
> the alarms in the "evaluator_queue" are deduced alarms), it queries the
> graph to find out whether there was a same alarm reported  by datasource.
> If  it was true,  it would discard the alarm.
>
>   4.  When entity_graph get the events from "queue",  it queries the
> graph to find out whether there was a same alarm deduced by evaluator. If
> it was true, it would replace the alarm in the graph with the newly arrived
> alarm reported by the datasource.
>
>  5.  When the evaluator deduced that an alarm would be deleted, it
> deletes the alarm whatever the generation type of the alarm be(Generated by
> datasource or deduced by evaluator).
>
>  6. When datasource reports recover event of an alarm, entity_graph
> would query graph to find out whether the alarm was exist. If the alarm was
> not exist, entity_graph would

Re: [openstack-dev] [vitrage] code organization for trace generator and

2017-01-15 Thread Yujun Zhang

I heard the exrex story from Elisha, though I didn't quite understand the
rule of allowed library in OpenStack.

I have also kept it in static datasource, but it was not so convenient to
jump over these long files to find the static datasource related function

On Sun, Jan 15, 2017 at 5:24 PM Afek, Ifat (Nokia - IL) <ifat.a...@nokia.com>
wrote:

> Hi Yujun,
>
>
>
> The motivation in building the mocks this way was that we could easily
> generate a series of events which are different from one another. There
> used to be a distinction between ‘static fields’ and ‘dynamic fileds’, and
> a library named exrex was used to generate random values by regex
> definitions (e.g. "instance-[0-9]{7}"). We were then contacted by the infra
> team that exrex was not a known library in OpenStack (did not appear in
> global-requirements), and we decided to remove it. You can still see in the
> code leftovers from the old implementation.
>
>
>
> I’m not sure if all datasources need the ability to generate a series of
> events. But since this is the way it was built, I decided to keep the
> structure in the Doctor datasourcde that I’ve recently written.
>
>
>
> I’ll be happy to also hear Elisha’s opinion about it, as he created this
> mechanism.
>
>
>
> Best Regards,
>
> Ifat.
>
>
>
>
>
> *From: *Yujun Zhang <zhangyujun+...@gmail.com>
> *Reply-To: *"OpenStack Development Mailing List (not for usage
> questions)" <openstack-dev@lists.openstack.org>
> *Date: *Friday, 13 January 2017 at 09:43
> *To: *"OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> *Subject: *[openstack-dev] [vitrage] code organization for trace
> generator and
>
>
>
> Hi, Root Causers
>
>
>
> In the implementation of static driver unit test, the most difficult part
> I've encountered is the mock driver and trace generator.
>
>
>
> Is there any particular reason to put all mock driver and transformer
> specs in the same file `trace_generator.py` and `mock_driver.py`? Would it
> be easier to maintain and extend if we organize the specs and drivers in
> datasource, e.g. `mock.static`, `mock.nova.host` and etc?
>
>
>
> --
>
> Yujun
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [machine learning] Question: Why there is no serious project for machine learning ?

2017-01-13 Thread Yujun Zhang

Not sure what you mean by serious.

Maybe you could have a look at Meteos[1]. It is a young project but surely
focuses on machine learning.

[1]: https://wiki.openstack.org/wiki/Meteos

On Fri, Jan 13, 2017 at 3:57 PM 严超  wrote:

> Hi, all,
> I'm wondering if there is a serious project for machine learning in
> Openstack ? For us to easily build a model in industrial operational level.
>
> Thanks,
> Andy Yan
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [vitrage] code organization for trace generator and

2017-01-12 Thread Yujun Zhang

Hi, Root Causers

In the implementation of static driver unit test, the most difficult part
I've encountered is the mock driver and trace generator.

Is there any particular reason to put all mock driver and transformer specs
in the same file `trace_generator.py` and `mock_driver.py`? Would it be
easier to maintain and extend if we organize the specs and drivers in
datasource, e.g. `mock.static`, `mock.nova.host` and etc?

--
Yujun
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Vitrage] About alarms reported by datasource and the alarms generated by vitrage evaluator

2017-01-12 Thread Yujun Zhang

Hi, Ifat

You comments is quite right. See my additional explanation inline.

On Thu, Jan 12, 2017 at 5:12 PM Afek, Ifat (Nokia - IL) 
wrote:

>
>
> One possible solution would be introducing a high level (abstract)
> template from users view. Then convert it to Vitrage scenario templates (or
> directly to graph). The *more sources* (nagios, vitrage deduction) for an
> abstract alarm we get from the system, the *more confidence* we get for a
> real fault. And the confidence of an alarm could be included in the
> scenario condition.
>
>
>
> [Ifat] I understand your idea, not sure yet if it helps with the use case.
>
> How would you imagine the ‘confidence’ property? As Boolean or a counter?
> One option is ‘deduced’ vs. ‘monitored’.
>
Another option is to count the number of monitors that reported it.
>

'deduced' vs 'monitored' would be good enough for most cases. Unless we
have identify some real use case, I also think there is no need for bring
in quantitative indicator like counter or probability.

> Personally, I don’t think this is needed. I think that if Nagios reports
> an error, then it is confident enough without getting it from another
> monitor.
>

You are right. We would consider a reported alarm as a reliable indicator
of fault. What I was thinking about is: when we the alarm is not seen, can
we be sure there is no fault?

Another situation is slow upstream alarm with fast downstream alarm. I
don't have an actual example for the moment, so please allow me to imagine
an extreme condition.

Suppose host fault will cause instance fault. But due to some restriction,
the host fault is scanned every 1 hour, but instance fault can be scanned
every 1 second. Now, we get alarms from 10 instance in the same host. Can
we deduce that the host is likely in fault status? And we may raise a
"deduced" alarm on the host and trigger an immediate scan which may result
in a "monitored" alarm. In this way, we reduce the time of detecting the
root cause, i.e host fault.

An alternative solution is to distinguish fault from alarm. Alarm is
actually a reflection of fault status.  Beside the directly linked alarm,
fault status can also be deduced from downstream alarms. I haven't think
over this model yet, it just flashed over my mind. Any comments are welcome.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Vitrage] About alarms reported by datasource and the alarms generated by vitrage evaluator

2017-01-11 Thread Yujun Zhang

I have just realized abstract alarm is not a good term. What I was talking
about is *fault* and *alarm*.

Fault is what actually happens, and alarm is how it is detected (or
deduced).

On Wed, Jan 11, 2017 at 5:13 PM Yujun Zhang <zhangyujun+...@gmail.com>
wrote:

> Yes, if we consider the Vitrage scenario evaluator as a pseudo monitor.
>
> I think YinLiYin's idea is a reasonable requirement from end user. They
> care more about the *real faults* in the system, not how they are
> detected. Though it will bring much challenge to design and engineering, it
> creates value for customers. I'm quite positive on this evolution.
>
> One possible solution would be introducing a high level (abstract)
> template from users view. Then convert it to Vitrage scenario templates (or
> directly to graph). The *more sources* (nagios, vitrage deduction) for an
> abstract alarm we get from the system, the *more confidence* we get for a
> real fault. And the confidence of an alarm could be included in the
> scenario condition.
>
> On Wed, Jan 11, 2017 at 4:08 PM Afek, Ifat (Nokia - IL) <
> ifat.a...@nokia.com> wrote:
>
> You are right. But as I see it, the case of Vitrage suspect vs. the real
> Nagios alarm is just one example of the more general case of two monitors
> reporting the same alarm.
>
> Don’t you think so?
>
>
>
> *From: *Yujun Zhang <zhangyujun+...@gmail.com>
>
>
> *Reply-To: *"OpenStack Development Mailing List (not for usage
> questions)" <openstack-dev@lists.openstack.org>
>
> *Date: *Wednesday, 11 January 2017 at 09:46
> *To: *"OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>, "yinli...@zte.com.cn" <
> yinli...@zte.com.cn>
> *Cc: *"han.jin...@zte.com.cn" <han.jin...@zte.com.cn>, "
> wang.we...@zte.com.cn" <wang.we...@zte.com.cn>, "zhang.yuj...@zte.com.cn"
> <zhang.yuj...@zte.com.cn>, "jia.peiy...@zte.com.cn" <
> jia.peiy...@zte.com.cn>, "gong.yah...@zte.com.cn" <gong.yah...@zte.com.cn>
>
>
> *Subject: *Re: [openstack-dev] [Vitrage] About alarms reported by
> datasource and the alarms generated by vitrage evaluator
>
>
>
> Hi, Ifat
>
>
>
> If I understand it correctly, your concerns are mainly on same alarm from
> different monitor, but not "suspect" status as discussed in another thread.
>
>
>
> On Tue, Jan 10, 2017 at 10:21 PM Afek, Ifat (Nokia - IL) <
> ifat.a...@nokia.com> wrote:
>
> Hi Yinliyin,
>
>
>
> At first I thought that changing the deduced to be a property on the alarm
> might help in solving your use case. But now I think most of the problems
> will remain the same:
>
>
>
> ·  It won’t solve the general problem of two different monitors that
> raise the same alarm
>
> ·  It won’t solve possible conflicts of timestamp and severity between
> different monitors
>
> ·  It will make the decision of when to delete the alarm more complex
> (delete it when the deduced alarm is deleted? When Nagios alarm is deleted?
> both? And how to change the timestamp and severity in these cases?)
>
>
>
> So I don’t think that making this change is beneficial.
>
> What do you think?
>
>
>
> Best Regards,
>
> Ifat.
>
>
>
>
>
> *From: *"yinli...@zte.com.cn" <yinli...@zte.com.cn>
> *Date: *Monday, 9 January 2017 at 05:29
> *To: *"Afek, Ifat (Nokia - IL)" <ifat.a...@nokia.com>
> *Cc: *"openstack-dev@lists.openstack.org" <
> openstack-dev@lists.openstack.org>, "han.jin...@zte.com.cn" <
> han.jin...@zte.com.cn>, "wang.we...@zte.com.cn" <wang.we...@zte.com.cn>, "
> zhang.yuj...@zte.com.cn" <zhang.yuj...@zte.com.cn>, "
> jia.peiy...@zte.com.cn" <jia.peiy...@zte.com.cn>, "gong.yah...@zte.com.cn"
> <gong.yah...@zte.com.cn>
> *Subject: *Re: [openstack-dev] [Vitrage] About alarms reported by
> datasource and the alarms generated by vitrage evaluator
>
>
>
> Hi Ifat,
>
>  I think there is a situation that all the alarms are reported by
> the monitored system. We use vitrage to:
>
> 1.  Found the relationships of the alarms, and find the root
> cause.
>
> 2.  Deduce the alarm before it really occured. This comprise
> two aspects:
>
>  1) A cause B:  When A occured,  we deduce that B would
> occur
>
>  2) B is caused by A:  When B occured, we deduce that A
> must occured
>
> In "2",   we do expect vitrage to raise the alarm before the
&g

Re: [openstack-dev] [Vitrage] About alarms reported by datasource and the alarms generated by vitrage evaluator

2017-01-11 Thread Yujun Zhang

Yes, if we consider the Vitrage scenario evaluator as a pseudo monitor.

I think YinLiYin's idea is a reasonable requirement from end user. They
care more about the *real faults* in the system, not how they are detected.
Though it will bring much challenge to design and engineering, it creates
value for customers. I'm quite positive on this evolution.

One possible solution would be introducing a high level (abstract) template
from users view. Then convert it to Vitrage scenario templates (or directly
to graph). The *more sources* (nagios, vitrage deduction) for an abstract
alarm we get from the system, the *more confidence* we get for a real
fault. And the confidence of an alarm could be included in the scenario
condition.

On Wed, Jan 11, 2017 at 4:08 PM Afek, Ifat (Nokia - IL) <ifat.a...@nokia.com>
wrote:

> You are right. But as I see it, the case of Vitrage suspect vs. the real
> Nagios alarm is just one example of the more general case of two monitors
> reporting the same alarm.
>
> Don’t you think so?
>
>
>
> *From: *Yujun Zhang <zhangyujun+...@gmail.com>
>
>
> *Reply-To: *"OpenStack Development Mailing List (not for usage
> questions)" <openstack-dev@lists.openstack.org>
>
> *Date: *Wednesday, 11 January 2017 at 09:46
> *To: *"OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>, "yinli...@zte.com.cn" <
> yinli...@zte.com.cn>
> *Cc: *"han.jin...@zte.com.cn" <han.jin...@zte.com.cn>, "
> wang.we...@zte.com.cn" <wang.we...@zte.com.cn>, "zhang.yuj...@zte.com.cn"
> <zhang.yuj...@zte.com.cn>, "jia.peiy...@zte.com.cn" <
> jia.peiy...@zte.com.cn>, "gong.yah...@zte.com.cn" <gong.yah...@zte.com.cn>
>
>
> *Subject: *Re: [openstack-dev] [Vitrage] About alarms reported by
> datasource and the alarms generated by vitrage evaluator
>
>
>
> Hi, Ifat
>
>
>
> If I understand it correctly, your concerns are mainly on same alarm from
> different monitor, but not "suspect" status as discussed in another thread.
>
>
>
> On Tue, Jan 10, 2017 at 10:21 PM Afek, Ifat (Nokia - IL) <
> ifat.a...@nokia.com> wrote:
>
> Hi Yinliyin,
>
>
>
> At first I thought that changing the deduced to be a property on the alarm
> might help in solving your use case. But now I think most of the problems
> will remain the same:
>
>
>
> ·  It won’t solve the general problem of two different monitors that
> raise the same alarm
>
> ·  It won’t solve possible conflicts of timestamp and severity between
> different monitors
>
> ·  It will make the decision of when to delete the alarm more complex
> (delete it when the deduced alarm is deleted? When Nagios alarm is deleted?
> both? And how to change the timestamp and severity in these cases?)
>
>
>
> So I don’t think that making this change is beneficial.
>
> What do you think?
>
>
>
> Best Regards,
>
> Ifat.
>
>
>
>
>
> *From: *"yinli...@zte.com.cn" <yinli...@zte.com.cn>
> *Date: *Monday, 9 January 2017 at 05:29
> *To: *"Afek, Ifat (Nokia - IL)" <ifat.a...@nokia.com>
> *Cc: *"openstack-dev@lists.openstack.org" <
> openstack-dev@lists.openstack.org>, "han.jin...@zte.com.cn" <
> han.jin...@zte.com.cn>, "wang.we...@zte.com.cn" <wang.we...@zte.com.cn>, "
> zhang.yuj...@zte.com.cn" <zhang.yuj...@zte.com.cn>, "
> jia.peiy...@zte.com.cn" <jia.peiy...@zte.com.cn>, "gong.yah...@zte.com.cn"
> <gong.yah...@zte.com.cn>
> *Subject: *Re: [openstack-dev] [Vitrage] About alarms reported by
> datasource and the alarms generated by vitrage evaluator
>
>
>
> Hi Ifat,
>
>  I think there is a situation that all the alarms are reported by
> the monitored system. We use vitrage to:
>
> 1.  Found the relationships of the alarms, and find the root
> cause.
>
> 2.  Deduce the alarm before it really occured. This comprise
> two aspects:
>
>  1) A cause B:  When A occured,  we deduce that B would
> occur
>
>  2) B is caused by A:  When B occured, we deduce that A
> must occured
>
> In "2",   we do expect vitrage to raise the alarm before the
> alarm is reported because the alarm would be lost or be delayed for some
> reason.  So we would write "raise alarm" actions in the scenarios of the
> template.  I think that the alarm is reported or is deduced should be a
> state property of the alarm. The vertex reported and the vertex deduced of
> the same alarm should b

Re: [openstack-dev] [Vitrage] About alarms reported by datasource and the alarms generated by vitrage evaluator

2017-01-10 Thread Yujun Zhang

Hi, Ifat

If I understand it correctly, your concerns are mainly on same alarm from
different monitor, but not "suspect" status as discussed in another thread.

On Tue, Jan 10, 2017 at 10:21 PM Afek, Ifat (Nokia - IL) <
ifat.a...@nokia.com> wrote:

Hi Yinliyin,

At first I thought that changing the deduced to be a property on the alarm
might help in solving your use case. But now I think most of the problems
will remain the same:

- It won’t solve the general problem of two different monitors that
raise the same alarm
- It won’t solve possible conflicts of timestamp and severity between
different monitors
- It will make the decision of when to delete the alarm more complex
(delete it when the deduced alarm is deleted? When Nagios alarm is deleted?
both? And how to change the timestamp and severity in these cases?)

So I don’t think that making this change is beneficial.

What do you think?

Best Regards,

Ifat.

*From: *"yinli...@zte.com.cn"
*Date: *Monday, 9 January 2017 at 05:29
*To: *"Afek, Ifat (Nokia - IL)"
*Cc: *"openstack-dev@lists.openstack.org" ,
"han.jin...@zte.com.cn" , "wang.we...@zte.com.cn" <
wang.we...@zte.com.cn>, "zhang.yuj...@zte.com.cn" ,
"jia.peiy...@zte.com.cn" , "gong.yah...@zte.com.cn"

*Subject: *Re: [openstack-dev] [Vitrage] About alarms reported by
datasource and the alarms generated by vitrage evaluator

Hi Ifat,

I think there is a situation that all the alarms are reported by
the monitored system. We use vitrage to:

1. Found the relationships of the alarms, and find the root
cause.

2. Deduce the alarm before it really occured. This comprise
two aspects:

1) A cause B: When A occured, we deduce that B would
occur

2) B is caused by A: When B occured, we deduce that A
must occured

In "2", we do expect vitrage to raise the alarm before the
alarm is reported because the alarm would be lost or be delayed for some
reason. So we would write "raise alarm" actions in the scenarios of the
template. I think that the alarm is reported or is deduced should be a
state property of the alarm. The vertex reported and the vertex deduced of
the same alarm should be merged to one vertex.

Best Regards,

Yinliyin.

原始邮件

*发件人：* ＜ifat.a...@nokia.com＞;

*收件人：* ＜openstack-dev@lists.openstack.org＞;

*抄送人：*韩静6838;王维雅00042110;章宇军10200531;贾培源10101785;龚亚辉6092001895
<(609)%20200-1895>;

*日* *期* *：*2017年01月07日 02:18

*主* *题* *：**Re: [openstack-dev] [Vitrage] About alarms reported by
datasource and the alarms generated by vitrage evaluator*

Hi YinLiYin,

This is an interesting question. Let me divide my answer to two parts.

First, the case that you described with Nagios and Vitrage. This problem
depends on the specific Nagios tests that you configure in your system, as
well as on the Vitrage templates that you use. For example, you can use
Nagios/Zabbix to monitor the physical layer, and Vitrage to raise deduced
alarms on the virtual and application layers. This way you will never have
duplicated alarms. If you want to use Nagios to monitor the other layers
as well, you can simply modify Vitrage templates so they don’t raise the
deduced alarms that Nagios may generate, and use the templates to show RCA
between different Nagios alarms.

Now let’s talk about the more general case. Vitrage can receive alarms from
different monitors, including Nagios, Zabbix, collectd and Aodh. If you are
using more than one monitor, it is possible that the same alarm (maybe
with a different name) will be raised twice. We need to create a mechanism
to identify such cases and create a single alarm with the properties of
both monitors. This has not been designed in details yet, so if you have
any suggestion we will be happy to hear them.

Best Regards,

Ifat.

*From: *"yinli...@zte.com.cn" ＜yinli...@zte.com.cn＞
*Reply-To: *"OpenStack Development Mailing List (not for usage questions)" ＜
openstack-dev@lists.openstack.org＞
*Date: *Friday, 6 January 2017 at 03:27
*To: *"openstack-dev@lists.openstack.org" ＜openstack-dev@lists.openstack.org
＞
*Cc: *"gong.yah...@zte.com.cn" ＜gong.yah...@zte.com.cn＞, "
han.jin...@zte.com.cn" ＜han.jin...@zte.com.cn＞, "wang.we...@zte.com.cn" ＜
wang.we...@zte.com.cn＞, "jia.peiy...@zte.com.cn" ＜jia.peiy...@zte.com.cn＞, "
zhang.yuj...@zte.com.cn" ＜zhang.yuj...@zte.com.cn＞
*Subject: *[openstack-dev] [Vitrage] About alarms reported by datasource
and the alarms generated by vitrage evaluator

Hi all,

Vitrage generate alarms acording to the templates. All the alarms raised
by vitrage has the type "vitrage". Suppose Nagios has an alarm A. Alarm A
is raised by vitrage evaluator according to the action part of a scenario,
type of alarm A is "vitrage". If Nagios reported alarm A

Re: [openstack-dev] [Vitrage] About alarms reported by datasource and the alarms generated by vitrage evaluator

2017-01-09 Thread Yujun Zhang

I prefer 2.b from instinct.

Not sure it could be linked to the vitrage_id[1] evolution. If an uuid is
created for the alarm, the implementation could be quite straightforward.

[1]: https://blueprints.launchpad.net/vitrage/+spec/standard-vitrage-id

On Tue, Jan 10, 2017 at 1:55 AM Afek, Ifat (Nokia - IL) <ifat.a...@nokia.com>
wrote:

> Hi Yujun,
>
>
>
> I understand the use case now, thanks for the detailed explanation.
>
>
>
> Supporting this use case will require some development in Vitrage. Let me
> try to list down the requirements and options that we have.
>
>
>
> 1.   Requirement: Raise ‘suspect’ deduced alarms in Vitrage.
>
> Implementation: Quite straight forward. There is no way to set ‘suspect’
> property in Vitrage right now, but it should be easy to add this option.
>
>
>
> 2.   Requirement: Change a ‘suspect’ alarm of type ‘vitrage’ to a
> ‘real’ alarm of type ‘nagios’.
>
> Implementation: There are a few alternatives how to achieve this goal
>
>
>
> a.   Delete the ‘suspect’ alarm and create the ‘real’ alarm. This
> will require supporting ‘not’ condition in the templates. An example
> scenario:
>
> condition: vm_alarm and not nagios_alarm:
>
>(action: create vitrage alarm)
>
> condition: nagios_alarm and vitrage_alarm:
>
>(action: delete vitrage_alarm)
>
>
>
> b.   Have both ‘suspect’ alarm and ‘real’ alarm, and create a
> ‘equivalent’ relationship between them. Configuring the template should be
> easy, however it won’t look nice in the UI. In past discussions we
> mentioned an option to group some vertices together in the UI. If we have
> this option, we might want to group these two alarms together.
>
>
>
> c.   Merge the two alarms. This solution seems the most reasonable
> one at first, but it is not trivial. For example: suppose one alarm is
> defined as ‘critical’ and was raised at 10:01, and the other alarm was
> defined as ‘warning’ and was raised at 10:02. How will you combine the two?
> And what if the ‘critical’ alarm then goes down, will you know how to
> change the severity back to ‘warning’? in case of vitrage vs. nagios we
> would like to prefer nagios; but let’s think of the more general case of
> two different monitors.
>
>
>
> 3.   In one of your emails you mentioned an option of having two
> ‘suspects’. Suppose vm_alarm is raised, will you raise two suspect vitrage
> alarms, e.g. host_alarm and switch_alarm? And if you then receive
> host_alarm from nagios, would you like to delete the suspect switch_alarm,
> or keep it? If you would like to delete it, it will require supporting
> ‘not’ in the template condition.
>
>
>
> Personally I would go for option 2b, but I will be happy to hear your
> thoughts about it.
>
>
>
> Hope I helped, but I suspect I just made things more complicated ;-)
>
> Ifat.
>
>
>
>
>
> *From: *Yujun Zhang <zhangyujun+...@gmail.com>
>
>
> *Reply-To: *"OpenStack Development Mailing List (not for usage
> questions)" <openstack-dev@lists.openstack.org>
>
> *Date: *Sunday, 8 January 2017 at 17:38
>
>
> *To: *"OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> *Cc: *"han.jin...@zte.com.cn" <han.jin...@zte.com.cn>, "
> wang.we...@zte.com.cn" <wang.we...@zte.com.cn>, "gong.yah...@zte.com.cn" <
> gong.yah...@zte.com.cn>, "jia.peiy...@zte.com.cn" <jia.peiy...@zte.com.cn>,
> "zhang.yuj...@zte.com.cn" <zhang.yuj...@zte.com.cn>
> *Subject: *Re: [openstack-dev] [Vitrage] About alarms reported by
> datasource and the alarms generated by vitrage evaluator
>
> Maybe I have missed something in the scenario template, but it seems you
> have understood my idea quite correctly :-)
>
>
>
> See further explanation inline
>
> On Sun, Jan 8, 2017 at 3:06 PM Afek, Ifat (Nokia - IL) <
> ifat.a...@nokia.com> wrote:
>
> Hi Yujun,
>
>
>
> Thanks for the explanation, but I still don’t fully understand.
>
>
>
> Let me start with the current state:
>
> 1.   introduce a flexible `metadata` dict in to ALARM entity
>
> [Ifat] Already exists. An alarm is represented as a vertex in the entity
> graph, with a dictionary of properties.
>
>
>
>  [yujunz] Can the alarm vertex be updated by scenario action? e.g. raise
> an alarm and set the property `suspect` to true.
>
>
>
> 2.   Allow generating update event[1] on metadata change
>
> 3.   Allow using ALARM metadata in scenario condition
>
> [Ifat] Already exists. You can define properties in the ‘entities

Re: [openstack-dev] [ALU] Re: [ALU] Re: [ALU] Re: [ALU] Re: [ALU] Re: [ALU]Re: [ALU][vitrage]how touseplaceholder vertex

2017-01-09 Thread Yujun Zhang

Great! So let's come back to my original question on redundant link.

Before: PC_a is connected to PC_b with Cable
After: PC_a is connected to PC_b with Cable *AND* Wi-Fi

How to we model this?

   1. Create a new edge labeled "Cable and Wi-Fi" ?
   2. Assign a structured label like ["Cable", "Wi-Fi"] to the edge?
   3. Create two edges, one labeled "Cable", the other labeled "Wi-Fi"?

--
Yujun

On Mon, Jan 9, 2017 at 6:44 PM Weyl, Alexey (Nokia - IL) <
alexey.w...@nokia.com> wrote:

Hi Yujun,

After some more checks, I found out the current code will handle your case
as well, because it will go into the 2.c.1.

Alexey

> -Original Message-
> From: Weyl, Alexey (Nokia - IL) [mailto:alexey.w...@nokia.com]
> Sent: Monday, January 09, 2017 10:54 AM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: [ALU] Re: [openstack-dev] [ALU] Re: [ALU] Re: [ALU] Re: [ALU]
> Re: [ALU]Re: [ALU][vitrage]how touseplaceholder vertex
>
> What will happen is the following:
> We have a connection between PC_a and PC_b with label Cable.
> Now a new event arrives and the transformer builds the following trio:
> (Vertex_A, Neighbors((Vertex_B, Edge), UPDATE) This transformed data
> arrives to the processor that goes to the UPDATE action:
> 1. Updates the Vertex_A (PC_a) properties in the graph.
> 2. Calls the _update_neighbors method:
>a. finds all the valid_edges and the obsolete_edges
>b. deletes all the obsolete edges from the graph
>c. connects the valid edges to the graph:
>   iterates over all the neighbors, and for each neighbor checks:
>   1. if the neighbor_vertex doesn't exist yet or if the is_deleted
> property of the neighbor_vertex is false then:
>  a. Update this vertex in the graph.
>  b. check if the edge doesn't exist yet and if so then add the
> edge.
>
> In your case the edge the edge 'Cable' will be deleted from the graph
> because it is obsolete, but from what I see at the moment and can be
> seen in the code, it won't enter the to the 2.c.1 because the neighbor
> vertex already exists and the neighbor vertex isn't defined as
> is_delete=false.
>
> That is a problem, and I have a solution for the problem, which I need
> to make sure that it ruin anything else because everything goes through
> the processor.
>
> Alexey
>
> From: Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> Sent: Monday, January 09, 2017 10:26 AM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: [ALU] Re: [openstack-dev] [ALU] Re: [ALU] Re: [ALU] Re: [ALU]
> Re: [ALU][vitrage]how touseplaceholder vertex
>
> Thanks Alexey. Could you explain a bit more detail based on the example
> in comments below?
>
> On Mon, Jan 9, 2017 at 4:08 PM Weyl, Alexey (Nokia - IL)
> <alexey.w...@nokia.com> wrote:
> That’s not what I mean.
>
> When some data is changed in a some vertices, their event is pushed to
> the event queue, and thus the correct transformer is called.
> We update the data of the current vertex, and for each neighbor that we
> created in the transformer, we check the following:
> if the neighbor vertex doesn't exist yet or if the id_deleted property
> of the neighbor vertex is false, then update the vertex in the graph.
> Then it checks if the edge is not in the graph yet, then it adds it.
>
> Before: PC_a is linked to PC_b with Cable
> After: PC_a is linked to PC_b with Wi-Fi
>
> So this will results in a removal of original edge (labeled Cable) and
> adding of new edge (labeled Wi-Fi) . Is that correct?
>
> Alexey
>
> From: Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> Sent: Sunday, January 08, 2017 5:45 PM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: [ALU] Re: [openstack-dev] [ALU] Re: [ALU] Re: [ALU] Re: [ALU]
> [vitrage]how touseplaceholder vertex
>
> Hi, Alexey
> On Sun, Jan 8, 2017 at 2:50 PM Weyl, Alexey (Nokia - IL)
> <alexey.w...@nokia.com> wrote:
> Hi Yujun,
>
> A relationship is defined by the following trio: source id, target id
> and a label on the edge.
> In the processor, I add only edges that doesn't exists.
>
> Currently the vertex is mapping to entity, and the edge is mapping to
> relationship.
>
> So do you mean that we should update the label if the relationship
> between two entities changes? e.g. we change the link between two PC's
> from cable to Wi-Fi.
> ___
> ___
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-
> requ...@lists.openstack.org?subject:unsubscribe
> http://lists

Re: [openstack-dev] [ALU] Re: [ALU] Re: [ALU] Re: [ALU] Re: [ALU] [vitrage]how touseplaceholder vertex

2017-01-09 Thread Yujun Zhang

Thanks Alexey. Could you explain a bit more detail based on the example in
comments below?

On Mon, Jan 9, 2017 at 4:08 PM Weyl, Alexey (Nokia - IL) <
alexey.w...@nokia.com> wrote:

That’s not what I mean.

When some data is changed in a some vertices, their event is pushed to the
event queue, and thus the correct transformer is called.
We update the data of the current vertex, and for each neighbor that we
created in the transformer, we check the following:
if the neighbor vertex doesn't exist yet or if the id_deleted property of
the neighbor vertex is false, then update the vertex in the graph. Then it
checks if the edge is not in the graph yet, then it adds it.


Before: PC_a is linked to PC_b with Cable
After: PC_a is linked to PC_b with Wi-Fi

So this will results in a removal of original edge (labeled Cable) and
adding of new edge (labeled Wi-Fi) . Is that correct?

Alexey

From: Yujun Zhang [mailto:zhangyujun+...@gmail.com]
Sent: Sunday, January 08, 2017 5:45 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [ALU] Re: [openstack-dev] [ALU] Re: [ALU] Re: [ALU] Re: [ALU]
[vitrage]how touseplaceholder vertex

Hi, Alexey
On Sun, Jan 8, 2017 at 2:50 PM Weyl, Alexey (Nokia - IL) <
alexey.w...@nokia.com> wrote:
Hi Yujun,

A relationship is defined by the following trio: source id, target id and a
label on the edge.
In the processor, I add only edges that doesn't exists.

Currently the vertex is mapping to entity, and the edge is mapping to
relationship.

So do you mean that we should update the label if the relationship between
two entities changes? e.g. we change the link between two PC's from cable
to Wi-Fi.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ALU] Re: [ALU] Re: [ALU] Re: [ALU] [vitrage] how touseplaceholder vertex

2017-01-08 Thread Yujun Zhang

Hi, Alexey

On Sun, Jan 8, 2017 at 2:50 PM Weyl, Alexey (Nokia - IL) <
alexey.w...@nokia.com> wrote:

> Hi Yujun,
>
> A relationship is defined by the following trio: source id, target id and
> a label on the edge.
> In the processor, I add only edges that doesn't exists.
>

Currently the vertex is mapping to entity, and the edge is mapping to
relationship.

So do you mean that we should update the label if the relationship between
two entities changes? e.g. we change the link between two PC's from cable
to Wi-Fi.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Vitrage] About alarms reported by datasource and the alarms generated by vitrage evaluator

2017-01-08 Thread Yujun Zhang

Maybe I have missed something in the scenario template, but it seems you
have understood my idea quite correctly :-)

See further explanation inline

On Sun, Jan 8, 2017 at 3:06 PM Afek, Ifat (Nokia - IL) <ifat.a...@nokia.com>
wrote:

> Hi Yujun,
>
>
>
> Thanks for the explanation, but I still don’t fully understand.
>
>
>
> Let me start with the current state:
>
> 1.   introduce a flexible `metadata` dict in to ALARM entity
>
> [Ifat] Already exists. An alarm is represented as a vertex in the entity
> graph, with a dictionary of properties.
>

 [yujunz] Can the alarm vertex be updated by scenario action? e.g. raise an
alarm and set the property `suspect` to true.

2.   Allow generating update event[1] on metadata change
>
> 3.   Allow using ALARM metadata in scenario condition
>
> [Ifat] Already exists. You can define properties in the ‘entities’ section
> in Vitrage templates
>

[yujunz] How do I specify the condition if one specified alarm is
'suspicious', e.g. condition: host_alarm.suspect ?

4.   Allow setting ALARM metadata in scenario action
>
>
>
> If I understand correctly, you are suggesting that one scenario will add
> metadata to an existing alarm, which will trigger an event, and as a result
> another scenario might be executed?
>

[yujunz] Exactly

Can you describe a use case where this behavior will help calculating the
> root cause?
>

[yujunz] Here's the simplified case derived from YinLiYin's example.
Suppose we add a causal relationship from `host_alarm` to `instance_alarm`,
i.e. host alarm will cause instance alarm. If an instance alarm is detected
(but no host alarm). It is "suspicious" that it may be caused by host
alarm. The reason could be event delay or lost. Instead of waiting for
snapshot service to update the host status, we want to run a diagnostic
action to check it initiatively.

In this case, we want to set the upstream (host) of a confirmed alarm
(instance) to "suspect" and trigger an diagnostic action on this change.

Hope that I have made the use case clear.

Thanks,
>
> Ifat.
>
>
>
>
>
> *From: *Yujun Zhang <zhangyujun+...@gmail.com>
>
>
> *Reply-To: *"OpenStack Development Mailing List (not for usage
> questions)" <openstack-dev@lists.openstack.org>
>
> *Date: *Saturday, 7 January 2017 at 09:27
>
>
> *To: *"OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
>
> *Cc: *"han.jin...@zte.com.cn" <han.jin...@zte.com.cn>, "
> wang.we...@zte.com.cn" <wang.we...@zte.com.cn>, "gong.yah...@zte.com.cn" <
> gong.yah...@zte.com.cn>, "jia.peiy...@zte.com.cn" <jia.peiy...@zte.com.cn>,
> "zhang.yuj...@zte.com.cn" <zhang.yuj...@zte.com.cn>
> *Subject: *Re: [openstack-dev] [Vitrage] About alarms reported by
> datasource and the alarms generated by vitrage evaluator
>
>
>
> The two questions raised by YinLiYin is actually one, i.e. *how to enrich
> the alarm properties *that can be used as an condition in root cause
> deducing.
>
>
>
> Both 'suspect' or 'datasource' are additional information that may be
> referred as a condition in general fault model, a.k.a. scenario in vitrage.
>
>
>
> It seems it could be done by
>
>1. introduce a flexible `metadata` dict in to ALARM entity
>
> 2.  Allow generating update event[1] on metadata change
>
> 3.  Allow using ALARM metadata in scenario condition
>
> 4.  Allow setting ALARM metadata in scenario action
>
> This will leave the flexibility to continuous development by defining a
> complex scenario template and keep the vitrage evaluator simple and generic.
>
>
>
> My two cents.
>
>
>
> [1]:
> http://docs.openstack.org/developer/vitrage/scenario-evaluator.html#concepts-and-guidelines
>
>
>
>
> On Sat, Jan 7, 2017 at 2:23 AM Afek, Ifat (Nokia - IL) <
> ifat.a...@nokia.com> wrote:
>
> Hi YinLiYin,
>
>
>
> This is an interesting question. Let me divide my answer to two parts.
>
>
>
> First, the case that you described with Nagios and Vitrage. This problem
> depends on the specific Nagios tests that you configure in your system, as
> well as on the Vitrage templates that you use. For example, you can use
> Nagios/Zabbix to monitor the physical layer, and Vitrage to raise deduced
> alarms on the virtual and application layers. This way you will never have
> duplicated alarms. If you want to use Nagios to monitor the other layers as
> well, you can simply modify Vitrage templates so they don’t raise the
> deduced alarms that Nagios may generate, and use the templates to show RCA
> between

Re: [openstack-dev] [Vitrage] About alarms reported by datasource and the alarms generated by vitrage evaluator

2017-01-06 Thread Yujun Zhang

The two questions raised by YinLiYin is actually one, i.e. *how to enrich
the alarm properties *that can be used as an condition in root cause
deducing.

Both 'suspect' or 'datasource' are additional information that may be
referred as a condition in general fault model, a.k.a. scenario in vitrage.

It seems it could be done by

   1. introduce a flexible `metadata` dict in to ALARM entity
   2. Allow generating update event[1] on metadata change
   3. Allow using ALARM metadata in scenario condition
   4. Allow setting ALARM metadata in scenario action

This will leave the flexibility to continuous development by defining a
complex scenario template and keep the vitrage evaluator simple and generic.

My two cents.

[1]:
http://docs.openstack.org/developer/vitrage/scenario-evaluator.html#concepts-and-guidelines


On Sat, Jan 7, 2017 at 2:23 AM Afek, Ifat (Nokia - IL) 
wrote:

> Hi YinLiYin,
>
>
>
> This is an interesting question. Let me divide my answer to two parts.
>
>
>
> First, the case that you described with Nagios and Vitrage. This problem
> depends on the specific Nagios tests that you configure in your system, as
> well as on the Vitrage templates that you use. For example, you can use
> Nagios/Zabbix to monitor the physical layer, and Vitrage to raise deduced
> alarms on the virtual and application layers. This way you will never have
> duplicated alarms. If you want to use Nagios to monitor the other layers as
> well, you can simply modify Vitrage templates so they don’t raise the
> deduced alarms that Nagios may generate, and use the templates to show RCA
> between different Nagios alarms.
>
>
>
> Now let’s talk about the more general case. Vitrage can receive alarms
> from different monitors, including Nagios, Zabbix, collectd and Aodh. If
> you are using more than one monitor, it is possible that the same alarm
> (maybe with a different name) will be raised twice. We need to create a
> mechanism to identify such cases and create a single alarm with the
> properties of both monitors. This has not been designed in details yet, so
> if you have any suggestion we will be happy to hear them.
>
>
>
> Best Regards,
>
> Ifat.
>
>
>
>
>
> *From: *"yinli...@zte.com.cn" 
> *Reply-To: *"OpenStack Development Mailing List (not for usage
> questions)" 
> *Date: *Friday, 6 January 2017 at 03:27
> *To: *"openstack-dev@lists.openstack.org" <
> openstack-dev@lists.openstack.org>
> *Cc: *"gong.yah...@zte.com.cn" , "
> han.jin...@zte.com.cn" , "wang.we...@zte.com.cn" <
> wang.we...@zte.com.cn>, "jia.peiy...@zte.com.cn" ,
> "zhang.yuj...@zte.com.cn" 
> *Subject: *[openstack-dev] [Vitrage] About alarms reported by datasource
> and the alarms generated by vitrage evaluator
>
>
>
> Hi all,
>
>Vitrage generate alarms acording to the templates. All the alarms
> raised by vitrage has the type "vitrage". Suppose Nagios has an alarm A.
> Alarm A is raised by vitrage evaluator according to the action part of a
> scenario, type of alarm A is "vitrage". If Nagios reported alarm A latter,
> a new alarm A with type "Nagios" would be generator in the entity graph.
>   There would be two vertices for the same alarm in the graph. And we have
> to define two alarm entities, two relationships, two scenarios in the
> template file to make the alarm propagation procedure work.
>
>It is inconvenient to describe fault model of system with lot of
> alarms. How to solve this problem?
>
>
>
> 殷力殷 YinLiYin
>
>
>
>
>
>
> 上海市浦东新区碧波路889号中兴研发大楼D502
> D502, ZTE Corporation R Center, 889# Bibo Road,
> Zhangjiang Hi-tech Park, Shanghai, P.R.China, 201203
> T: +86 21 68896229 <+86%2021%206889%206229>
> M: +86 13641895907 <+86%20136%204189%205907>
> E: yinli...@zte.com.cn
> www.zte.com.cn
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ALU] Re: [ALU] Re: [ALU] [vitrage] how to useplaceholder vertex

2017-01-05 Thread Yujun Zhang

A follow up question on relationships.

On Thu, Jan 5, 2017 at 9:59 PM Weyl, Alexey (Nokia - IL) <
alexey.w...@nokia.com> wrote:

> Hi Yujun,
>
> Lets see.
>
> 1. There is no need for the transformer to handle this duplication. What
> will happen at the moment is that we will receive twice every neighbor, and
> it is fine by us, because it is a quite small datasource, and 99.999% of
> the time it won't be changed.
>

It's fine for neighbor because vertex can be identified by id and there
won't be duplication.

But what about relationship, how do we model *redundant* links between two
entities? There seems to be no id for relationships.

2. It should be 2 events. We want to make it as simple as possible, and in
> the same time as flexible as possible. So you should create 2 events and
> each one will have the neighbor connection.
>
> Hope it answers everything.
>
> BR,
> Alexey
>
>
>
> From: Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> Sent: Thursday, January 05, 2017 2:32 PM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: [ALU] Re: [openstack-dev] [ALU] Re: [ALU] [vitrage] how to
> useplaceholder vertex
>
> Alexey,
>
> I have to dig this old thread to clarify some issues I met during static
> datasource implementation. Hope that you can still recall the context :-)
>
> I'll try to simplify this question with an example. The following
> configuration are snippet from static datasource
>
> 1. suppose we have three switches linked in a ring. What would be the
> expected entity events emit by the driver?
>
> In my proposed driver, there will be three entities. And each relationship
> will appear both in source entity and target entity, e.g. s1->s2 will be
> included in both s1 and s2. Should the transformer handle this duplication
> or the graph utils will?
> entities:
>   - config_id: s1
> type: switch
> name: switch-1
> id: 12345
> state: available
>   - config_id: s2
> type: switch
> name: switch-2
> id: 23456
> state: available
>   - config_id: s3
> type: switch
> name: switch-3
> id: 34567
> state: available
> relationships:
>   - source: s1
> target: s2
> relation_type: linked
>   - source: s2
> target: s3
> relation_type: linked
>   - source: s3
> target: s1
> relation_type: linked
> 2. suppose we created a link between switch and nova.host. What will be
> the expected entity events? Should it be one entity event of s1 with h1
> embedded as neighbor? Or two entity events, s1 and h1?
> entities:
>   - config_id: s1
> type: switch
> name: switch-1
> id: 12345
> state: available
>   - config_id: h1
> type: nova.host
> id: 1
> relationships:
>   - source: s1
> target: h1
> relation_type: attached
>
> On Wed, Dec 14, 2016 at 11:54 PM Weyl, Alexey (Nokia - IL) <
> alexey.w...@nokia.com> wrote:
> 1. That is correct.
>
> 2. That is not quite correct.
> In static we only define the main properties of each entity, meaning type,
> id, category and thus it is ok that for each main entity we will create its
> neighbors and connect between them. There is no need for any distinguish
> due to that.
>
>
> From: Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> Sent: Wednesday, December 14, 2016 5:00 PM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: [ALU] Re: [openstack-dev] [ALU] [vitrage] how to use placeholder
> vertex
>
> Hi, Alexey,
>
> Thanks for the detail example. It explains the existing mechanism of
> vertex creation well.
>
> So it looks like each resource type will have a primary datasource, e.g.
> nova.host for nova.host, nova.intance for nova.instance, that holds full
> details. Is that correct?
>
> Not sure that you remember the long discussion in static driver review[1]
> or not. At last, we agreed on a unified entity definition for both
> `nova.host` and `switch`, no extra key to indicate it is "external" (should
> create a placeholder).
>
> If I understand it correctly, no placeholder will be created in this case.
> Because we can not distinguish them from the static configuration. And the
> properties of `nova.host` resource shall to be merged from `static` and
> nova.host` datasources. Is that so?
>
> [1]: https://review.openstack.org/#/c/405354/
>
> On Wed, Dec 14, 2016 at 5:40 PM Weyl, Alexey (Nokia - IL) <
> alexey.w...@nokia.com> wrote:
> Hi Yujun,
>
> This is a good question, and let me explain for you how it works.
> Lets say we are supposed to get 2 entities from nova, nova.host called
> host1 and nova.instance cal

Re: [openstack-dev] [ALU] Re: [ALU] [vitrage] how to use placeholder vertex

2017-01-05 Thread Yujun Zhang

Alexey,

I have to dig this old thread to clarify some issues I met during static
datasource implementation. Hope that you can still recall the context :-)

I'll try to simplify this question with an example. The following
configuration are snippet from static datasource

1. suppose we have three switches linked in a ring. What would be the
expected entity events emit by the driver?

In my proposed driver, there will be three entities. And each relationship
will appear both in source entity and target entity, e.g. s1->s2 will be
included in both s1 and s2. *Should the transformer handle this duplication
or the graph utils will?*

entities:
  - config_id: s1
type: switch
name: switch-1
id: 12345
state: available
  - config_id: s2
type: switch
name: switch-2
id: 23456
state: available
  - config_id: s3
type: switch
name: switch-3
id: 34567
state: available
relationships:
  - source: s1
target: s2
relation_type: linked
  - source: s2
target: s3
relation_type: linked
  - source: s3
target: s1
relation_type: linked

2. suppose we created a link between switch and nova.host. What will be the
expected entity events? *Should it be one entity event of s1 with h1
embedded as neighbor? Or two entity events, s1 and h1?*

entities:
  - config_id: s1
type: switch
name: switch-1
id: 12345
state: available
  - config_id: h1
type: nova.host
id: 1
relationships:
  - source: s1
target: h1
relation_type: attached


On Wed, Dec 14, 2016 at 11:54 PM Weyl, Alexey (Nokia - IL) <
alexey.w...@nokia.com> wrote:

> 1. That is correct.
>
> 2. That is not quite correct.
> In static we only define the main properties of each entity, meaning type,
> id, category and thus it is ok that for each main entity we will create its
> neighbors and connect between them. There is no need for any distinguish
> due to that.
>
>
> From: Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> Sent: Wednesday, December 14, 2016 5:00 PM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: [ALU] Re: [openstack-dev] [ALU] [vitrage] how to use placeholder
> vertex
>
> Hi, Alexey,
>
> Thanks for the detail example. It explains the existing mechanism of
> vertex creation well.
>
> So it looks like each resource type will have a primary datasource, e.g.
> nova.host for nova.host, nova.intance for nova.instance, that holds full
> details. Is that correct?
>
> Not sure that you remember the long discussion in static driver review[1]
> or not. At last, we agreed on a unified entity definition for both
> `nova.host` and `switch`, no extra key to indicate it is "external" (should
> create a placeholder).
>
> If I understand it correctly, no placeholder will be created in this case.
> Because we can not distinguish them from the static configuration. And the
> properties of `nova.host` resource shall to be merged from `static` and
> nova.host` datasources. Is that so?
>
> [1]: https://review.openstack.org/#/c/405354/
>
> On Wed, Dec 14, 2016 at 5:40 PM Weyl, Alexey (Nokia - IL) <
> alexey.w...@nokia.com> wrote:
> Hi Yujun,
>
> This is a good question, and let me explain for you how it works.
> Lets say we are supposed to get 2 entities from nova, nova.host called
> host1 and nova.instance called vm1 and vm1 is supposed to be connected to
> host1.
> The nova.host driver and nova.instance driver are working simultaneously
> and thus we don’t know the order in which those events will arrive.
> We have 2 use cases:
> 1.   Host1 event arrives before vm1.
> In this case the processor will call the transformer of nova.host and will
> create a vertex for host1 in the graph with the full details of host1.
> Then, vm1 event will arrive, the processor will create the vm1 vertex in
> the graph, it will update the host1 properties in the graph (because the
> host1 details that are created in nova.instance are only its basic details
> such as: category, type, id, is_deleted, is_placeholder they host1
> properties won’t be changed in the graph because those details are basic
> and can’t be changed), and then it will create an edge between vm1 and
> host1 (only the nova.instance knows to which nova.host it is connected and
> not vice versa).
> 2.   Vm1 event arrives before host1.
> In this case the processor will add vm1 to the graph, then it will add the
> host1 placeholder to the graph (so we will know to which host vm1 is
> connected) and then add the edge between them.
> Then when the processor will handle with the host1 event, it will just add
> some properties of the host1 vertex, and of course will change the
> is_placeholder property of host1 to false.
> We also has the consistency service that runs every 10 minutes (its
>

Re: [openstack-dev] 答复: RE: Re: [ALU] [Vitrage] vitrage tempest job config

2017-01-04 Thread Yujun Zhang

It seems the file is truncated on Github preview. It stops at
openstack/security-specs

If you check the raw file[1], you will find entry of openstack/vitrage

[1]:
https://raw.githubusercontent.com/openstack-infra/project-config/master/zuul/layout.yaml


[image: Screen Shot 2017-01-05 at 9.11.08 AM.png]


On Thu, Jan 5, 2017 at 8:39 AM  wrote:

> Hi,
>
>
> I know. Why only python-vitrageclient and puppet-vitrage register in the
> zuul/layout.yaml? They also register in the jeknins/jobs/projects.yaml.
>
> And other vitrage projects only register in the
> jeknins/jobs/projects.yaml.
>
> I'm confused if i want to add a new jenkins job, which file should i
> register in?
>
>
> BR,
>
> dwj
>
>
>
>
>
> 原始邮件
> *发件人：*Weyl,Alexey(Nokia-IL)
> *收件人：*董文娟00101742;
> *抄送人：*＜openstack-dev@lists.openstack.org＞
> *日 期 ：*2017年01月04日 19:13
> *主 题 ：**RE: Re: [openstack-dev] [ALU]   [Vitrage] vitrage tempest job
> config*
>
>
> We do write code in the zuul/layout.yaml:
>
>   - name: openstack/puppet-vitrage
>
> template:
>
>   - name: merge-check
>
>   - name: puppet-check-jobs
>
>   - name: puppet-module-unit-jobs
>
>   - name: puppet-beaker-jobs
>
>   - name: puppet-beaker-jobs-xenial
>
>   - name: puppet-branch-tarball-jobs
>
>   - name: openstack-server-release-jobs
>
>   - name: release-notes-jobs
>
>   - name: openstack/python-vitrageclient
> template:
>   - name: merge-check
>   - name: python-jobs
>   - name: python35-jobs
>   - name: check-requirements
>   - name: publish-to-pypi
>   - name: osc-plugin-jobs
>
> Alexey
>
> From: dong.wenj...@zte.com.cn [mailto:dong.wenj...@zte.com.cn]
> Sent: Wednesday, January 04, 2017 10:59 AM
> To: Weyl, Alexey (Nokia - IL)
> Cc: openstack-dev@lists.openstack.org
> Subject: 答复: Re: [openstack-dev] [ALU] [Vitrage] vitrage tempest job config
>
> Hi Alexey,
>
>
> As far as i know, every project needs to register in the zuul/layout.yaml[1], 
> so zuul can submit the job to jenkins via gearman.
>
> So the jenkins knows which jobs need to run and then run the job which 
> defined in the jenkins/jobs/XXX.yaml.
>
> Did i missing something? Or the job registeration moved from zuul/layout.yaml 
> to jenkins/jobs/projects.yaml?
> If yes, how can zuul know which jobs need to submit to jenkins?Thanks~
>
> [1]
> https://github.com/openstack-infra/project-config/blob/master/zuul/layout.yaml
>
>
> BR,
> dwj
>
>
>
>
> 原始邮件
> 发件人：Weyl,Alexey(Nokia-IL)
> 收件人：OpenStack Development Mailing List (not for usage questions);
> 日 期 ：2017年01月04日 16:30
> 主 题 ：Re: [openstack-dev] [ALU]   [Vitrage] vitrage tempest job config
>
> Hi Dong,
>
>
> All of that data is configured the openstack-infra/project-config in github.
>
> There you can find our changes by searching for vitrage.
>
> The main files that we have pushed our changes are:
> jenkins/jobs/vitrage.yaml
> gerrit/projects.yaml
> jenkins/jobs/projects.yaml
>
> BR,
> Alexey
>
> From: dong.wenj...@zte.com.cn [mailto:dong.wenj...@zte.com.cn]
> Sent: Wednesday, January 04, 2017 9:49 AM
> To: openstack-dev@lists.openstack.org
> Subject: [ALU] [openstack-dev] [Vitrage] vitrage tempest job config
>
>
> Hi all,
>
> I didn't find any vitrage project jenkins jobs registered in zuul/layout.yaml.
> Where does all the jobs configed?Thanks~
> BR,
> dwj
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ALU] [vitrage] how to use placeholder vertex

2016-12-14 Thread Yujun Zhang

Hi, Alexey,

Thanks for the detail example. It explains the existing mechanism of vertex
creation well.

So it looks like each resource type will have a primary datasource, e.g.
nova.host for nova.host, nova.intance for nova.instance, that holds full
details. Is that correct?

Not sure that you remember the long discussion in static driver review[1]
or not. At last, we agreed on a unified entity definition for both
`nova.host` and `switch`, no extra key to indicate it is "external" (should
create a placeholder).

If I understand it correctly, no placeholder will be created in this case.
Because we can not distinguish them from the static configuration. And the
properties of `nova.host` resource shall to be merged from `static` and
nova.host` datasources. Is that so?

[1]: https://review.openstack.org/#/c/405354/

On Wed, Dec 14, 2016 at 5:40 PM Weyl, Alexey (Nokia - IL) <
alexey.w...@nokia.com> wrote:

> Hi Yujun,
>
>
>
> This is a good question, and let me explain for you how it works.
>
> Lets say we are supposed to get 2 entities from nova, nova.host called
> host1 and nova.instance called vm1 and vm1 is supposed to be connected to
> host1.
>
> The nova.host driver and nova.instance driver are working simultaneously
> and thus we don’t know the order in which those events will arrive.
>
> We have 2 use cases:
>
> 1.   Host1 event arrives before vm1.
>
> In this case the processor will call the transformer of nova.host and will
> create a vertex for host1 in the graph with the full details of host1.
>
> Then, vm1 event will arrive, the processor will create the vm1 vertex in
> the graph, it will update the host1 properties in the graph (because the
> host1 details that are created in nova.instance are only its basic details
> such as: category, type, id, is_deleted, is_placeholder they host1
> properties won’t be changed in the graph because those details are basic
> and can’t be changed), and then it will create an edge between vm1 and
> host1 (only the nova.instance knows to which nova.host it is connected and
> not vice versa).
>
> 2.   Vm1 event arrives before host1.
>
> In this case the processor will add vm1 to the graph, then it will add the
> host1 placeholder to the graph (so we will know to which host vm1 is
> connected) and then add the edge between them.
>
> Then when the processor will handle with the host1 event, it will just add
> some properties of the host1 vertex, and of course will change the
> is_placeholder property of host1 to false.
>
> We also has the consistency service that runs every 10 minutes (its
> configurable with the snapshot_interval) and checks if there are vertices
> that are is_placeholder=True and are in the graph more then
> 2*snapshot_interval time then it means that such a vertex of host1 for
> example doesn’t really exists and it deletes it from the graph).
>
> In addition, when we understand that we need to delete some entity from
> the graph, upon delete notification for example, then we don’t delete it
> right away, we change the is_deleted property of that entity to True, and
> we will delete it on the next run of the consistency service. The reason we
> do that, is because we need it for the evaluator, because it runs a
> subgraph matching algorithm on the graph, and if the entity that is
> supposed to be there doesn’t appear then it won’t work correctly.
>
>
>
> The best way to create a placeholder is to call the placeholder method of
> the correct transformer using the transformers array that each transformer
> class has.
>
>
>
> I hope this helps.
>
>
>
> Alexey
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Wednesday, December 14, 2016 10:55 AM
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* [ALU] [openstack-dev] [vitrage] how to use placeholder vertex
>
>
>
> Hi, root causers (assuming this nickname has been agreed)
>
>
>
> It seems the placeholder is created for every neighbor of an entity[1].
> I'm not sure what will happen if the neighbor entity has been created
> before.
>
>
>
> Will the graph utility handle it? Or we need to check the existence in
> transformer and deal with it?
>
>
>
> Similar question for how to create a vertex when there is already a
> placeholder with same ID
>
>
>
> [1]:
> https://github.com/openstack/vitrage/blob/master/vitrage/datasources/static_physical/transformer.py#L114
>
>
> --
>
> Yujun
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [vitrage] how to use placeholder vertex

2016-12-14 Thread Yujun Zhang

Hi, root causers (assuming this nickname has been agreed)

It seems the placeholder is created for every neighbor of an entity[1]. I'm
not sure what will happen if the neighbor entity has been created before.

Will the graph utility handle it? Or we need to check the existence in
transformer and deal with it?

Similar question for how to create a vertex when there is already a
placeholder with same ID

[1]:
https://github.com/openstack/vitrage/blob/master/vitrage/datasources/static_physical/transformer.py#L114

--
Yujun
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] how to use mock driver

2016-12-13 Thread Yujun Zhang

Understood now.

It could be less confusing if we give a better variable name (without
`_re`) or put some comments after removing `exrex` :-)

On Tue, Dec 13, 2016 at 4:03 PM Rosensweig, Elisha (Nokia - IL) <
elisha.rosensw...@nokia.com> wrote:

> Yes. We actually had to remove the regex generation support a few months
> ago, since the python package we were using – exrex – was not one that
> OpenStack supported.
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Tuesday, December 13, 2016 8:32 AM
>
>
> *To:* OpenStack Development Mailing List (not for usage questions) <
> openstack-dev@lists.openstack.org>
>
> *Subject:* Re: [openstack-dev] [vitrage] how to use mock driver
>
>
>
> Elisha, thanks for the explanation. The difference is clear to me now.
>
>
>
> If I understand it correctly, the regular expression in spec JSON is for
> information only. It is never compiled into a `re` object.
>
>
>
> The actual values are generated in `static_info_parsers` from the
> `mapping`. The regular expression is neither used as a value template nor
> for value validation.
>
>
>
> Is that right?
>
>
>
> On Mon, Dec 12, 2016 at 8:47 PM Rosensweig, Elisha (Nokia - IL) <
> elisha.rosensw...@nokia.com> wrote:
>
> Hi,
>
>
>
> · In Vitrage Datasources, we can have a different input format
> for snapshots and updates. Thus, we need a different JSON file for each.
>
> · Also, as part of the Mock feature, we need to support (for each
> resource) things that will be static, such as it’s name, and things that
> change over time, such as timestamps. We support this partially via
> different JSON files. In general, the dynamic file (marked with “D”)
> *overwrites* the static one (marked with “S”).
>
> · In the code you can further inject specific fields you want to
> have for a specific test, in addition to the JSON files. See examples in
> test_scenario_evaluator.py.
>
>
>
> Elisha
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Monday, December 12, 2016 8:23 AM
> *To:* OpenStack Development Mailing List (not for usage questions) <
> openstack-dev@lists.openstack.org>
> *Subject:* [openstack-dev] [vitrage] how to use mock driver
>
>
>
> Is there any documentation on how to use mock driver for unit testing?
>
>
>
> It seems it generates fake events from json spec but what is the different
> between
>
>
>
> - `xxx_snapshot_X.json` and `xxx_dynamic_X.json`
>
> - `xxx_S` and `xxx_D`
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] how to use mock driver

2016-12-12 Thread Yujun Zhang

Elisha, thanks for the explanation. The difference is clear to me now.

If I understand it correctly, the regular expression in spec JSON is for
information only. It is never compiled into a `re` object.

The actual values are generated in `static_info_parsers` from the
`mapping`. The regular expression is neither used as a value template nor
for value validation.

Is that right?

On Mon, Dec 12, 2016 at 8:47 PM Rosensweig, Elisha (Nokia - IL) <
elisha.rosensw...@nokia.com> wrote:

> Hi,
>
>
>
> · In Vitrage Datasources, we can have a different input format
> for snapshots and updates. Thus, we need a different JSON file for each.
>
> · Also, as part of the Mock feature, we need to support (for each
> resource) things that will be static, such as it’s name, and things that
> change over time, such as timestamps. We support this partially via
> different JSON files. In general, the dynamic file (marked with “D”)
> *overwrites* the static one (marked with “S”).
>
> · In the code you can further inject specific fields you want to
> have for a specific test, in addition to the JSON files. See examples in
> test_scenario_evaluator.py.
>
>
>
> Elisha
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Monday, December 12, 2016 8:23 AM
> *To:* OpenStack Development Mailing List (not for usage questions) <
> openstack-dev@lists.openstack.org>
> *Subject:* [openstack-dev] [vitrage] how to use mock driver
>
>
>
> Is there any documentation on how to use mock driver for unit testing?
>
>
>
> It seems it generates fake events from json spec but what is the different
> between
>
>
>
> - `xxx_snapshot_X.json` and `xxx_dynamic_X.json`
>
> - `xxx_S` and `xxx_D`
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [vitrage] how to use mock driver

2016-12-11 Thread Yujun Zhang

Is there any documentation on how to use mock driver for unit testing?

It seems it generates fake events from json spec but what is the different
between

- `xxx_snapshot_X.json` and `xxx_dynamic_X.json`
- `xxx_S` and `xxx_D`
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ALU] Re: [ALU] Re: [ALU] [vitrage] datasource driverreturnsentities only?

2016-12-04 Thread Yujun Zhang

Your answer has made everything clear, Alexey.

And I will take into consideration of the neighbor transformer in static
datasource transformer implementation.

On Mon, Dec 5, 2016 at 2:22 PM Weyl, Alexey (Nokia - IL) <
alexey.w...@nokia.com> wrote:

> Hi,
>
>
>
> Yes, the entity_type is datasource_type, and the entity_id is the
> resources’ id which came from it’s datasource.
>
>
>
> The example is correct, and just for the clarification, the “12345678” is
> the id of that entity in the datasource.
>
>
>
> Another clarification that I want to make sure, is that the static
> datasource (and actually each datasource), retrieves the transformer of the
> neighbor from the transformers dictionary that each transformer holds, and
> using the neighbors’ transformer it creates the placeholder of the neighbor
> (this is because each transformer knows exactly how to build it’s own
> placeholder).
>
>
>
> Alexey
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Monday, December 05, 2016 2:38 AM
>
>
> *To:* OpenStack Development Mailing List (not for usage questions)
>
> *Subject:* [ALU] Re: [openstack-dev] [ALU] Re: [ALU] [vitrage] datasource
> driverreturnsentities only?
>
>
>
> Thanks for clarification.
>
>
>
> Is `entity_type` equivalent to datasource type? And `entity_id` should be
> interpreted by the related datasource, is that right.
>
>
>
> For example, "RESOURCE:nova.host:12345678" is from `nova.host` datasource
> and `12345678` is how `nova.host` identify a resource.
>
>
>
> --
>
> Yujun
>
>
>
> On Sun, Dec 4, 2016 at 4:35 PM Weyl, Alexey (Nokia - IL) <
> alexey.w...@nokia.com> wrote:
>
> Hi Yujun,
>
>
>
> I see the confusion.
>
>
>
> The way we find the resource entity (it’s different than finding the alarm
> entity, but it’s ok because the static datasource is defining relationships
> between resource entities) is in the following way:
>
> Each entity has a vitrage_id which is defined by
> “RESOURCE:: (we are planning in the very near
> future to change the vitrage_id to be a UUID, but at the moment your change
> doesn’t need to refer to that), for example: “RESOURCE:nova.host:12345678”.
>
>
>
> So in this way all you need to know about the other entity is it’s
>  and it’s .
>
>
>
> BR,
>
> Alexey
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Thursday, December 01, 2016 4:16 PM
>
>
> *To:* OpenStack Development Mailing List (not for usage questions)
>
> *Subject:* [ALU] Re: [openstack-dev] [ALU] [vitrage] datasource driver
> returnsentities only?
>
>
>
> Another question, how do we describe an entity from *another* datasource
> in static datasource config?
>
>
>
> In the test resources of static_physical datasource, it seems to be
> referred as following. Does it means that it will be `nova.host` to find
> the matched resource? If so, how will `nova.host` identify the resource, by
> name or by id?
>
> *relationships:*
>   - *type: *nova.host
> *name: *host-2
> *id: *2
> *relation_type: *attached
>
>
>
> On Thu, Dec 1, 2016 at 9:50 PM Weyl, Alexey (Nokia - IL) <
> alexey.w...@nokia.com> wrote:
>
> Hi Yujun,
>
>
>
> Get_all does returns a list of entities to be sent.
>
>
>
> Each event that is sent from the driver to the processor contains also all
> the details of the neighbors that it connects to.
>
> For example, the event and data that we receive from nova about an
> instance also contains the host (compute) that it sits on, and that is how
> we decide to connect it to the correct host.
>
>
>
> I think it is ok that the event of static (from driver to the processor)
> will contain for each entity it neighbors that it is supposed to be
> connected to.
>
>
>
> BR,
>
> Alexey
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Thursday, December 01, 2016 3:20 PM
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* [ALU] [openstack-dev] [vitrage] datasource driver returns
> entities only?
>
>
>
> During the implementation of static datasource driver[1], I got a question
> on the return format of `get_all` method.
>
>
>
> If I understand correctly, it should return a list of entities to be sent
> to the queue. Does it infer that the relationship should be embedded in
> entity, just like the legacy static_physical datasource?
>
>
>
> Suppose a link between two switches are created, how should we emit this
> change in `get_all` or `get_changes`?
>
>
>
&

Re: [openstack-dev] [ALU] Re: [ALU] [vitrage] datasource driver returnsentities only?

2016-12-04 Thread Yujun Zhang

Thanks for clarification.

Is `entity_type` equivalent to datasource type? And `entity_id` should be
interpreted by the related datasource, is that right.

For example, "RESOURCE:nova.host:12345678" is from `nova.host` datasource
and `12345678` is how `nova.host` identify a resource.

--
Yujun

On Sun, Dec 4, 2016 at 4:35 PM Weyl, Alexey (Nokia - IL) <
alexey.w...@nokia.com> wrote:

> Hi Yujun,
>
>
>
> I see the confusion.
>
>
>
> The way we find the resource entity (it’s different than finding the alarm
> entity, but it’s ok because the static datasource is defining relationships
> between resource entities) is in the following way:
>
> Each entity has a vitrage_id which is defined by
> “RESOURCE:: (we are planning in the very near
> future to change the vitrage_id to be a UUID, but at the moment your change
> doesn’t need to refer to that), for example: “RESOURCE:nova.host:12345678”.
>
>
>
> So in this way all you need to know about the other entity is it’s
>  and it’s .
>
>
>
> BR,
>
> Alexey
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Thursday, December 01, 2016 4:16 PM
>
>
> *To:* OpenStack Development Mailing List (not for usage questions)
>
> *Subject:* [ALU] Re: [openstack-dev] [ALU] [vitrage] datasource driver
> returnsentities only?
>
>
>
> Another question, how do we describe an entity from *another* datasource
> in static datasource config?
>
>
>
> In the test resources of static_physical datasource, it seems to be
> referred as following. Does it means that it will be `nova.host` to find
> the matched resource? If so, how will `nova.host` identify the resource, by
> name or by id?
>
>
> *relationships:  *- *type: *nova.host
> *name: *host-2
> *id: *2
> *relation_type: *attached
>
>
>
> On Thu, Dec 1, 2016 at 9:50 PM Weyl, Alexey (Nokia - IL) <
> alexey.w...@nokia.com> wrote:
>
> Hi Yujun,
>
>
>
> Get_all does returns a list of entities to be sent.
>
>
>
> Each event that is sent from the driver to the processor contains also all
> the details of the neighbors that it connects to.
>
> For example, the event and data that we receive from nova about an
> instance also contains the host (compute) that it sits on, and that is how
> we decide to connect it to the correct host.
>
>
>
> I think it is ok that the event of static (from driver to the processor)
> will contain for each entity it neighbors that it is supposed to be
> connected to.
>
>
>
> BR,
>
> Alexey
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Thursday, December 01, 2016 3:20 PM
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* [ALU] [openstack-dev] [vitrage] datasource driver returns
> entities only?
>
>
>
> During the implementation of static datasource driver[1], I got a question
> on the return format of `get_all` method.
>
>
>
> If I understand correctly, it should return a list of entities to be sent
> to the queue. Does it infer that the relationship should be embedded in
> entity, just like the legacy static_physical datasource?
>
>
>
> Suppose a link between two switches are created, how should we emit this
> change in `get_all` or `get_changes`?
>
>
>
> Currently I made a compromise by emitting the relationship as an update of
> the connected entity. This is not very elegant and it seems we are going
> back to the legacy static_physical datasource.
>
>
>
> [1] https://review.openstack.org/#/c/405354/
>
> --
>
> Yujun
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ALU] [vitrage] datasource driver returns entities only?

2016-12-01 Thread Yujun Zhang

Another question, how do we describe an entity from *another* datasource in
static datasource config?

In the test resources of static_physical datasource, it seems to be
referred as following. Does it means that it will be `nova.host` to find
the matched resource? If so, how will `nova.host` identify the resource, by
name or by id?

relationships:
  - type: nova.host
name: host-2
id: 2
relation_type: attached


On Thu, Dec 1, 2016 at 9:50 PM Weyl, Alexey (Nokia - IL) <
alexey.w...@nokia.com> wrote:

> Hi Yujun,
>
>
>
> Get_all does returns a list of entities to be sent.
>
>
>
> Each event that is sent from the driver to the processor contains also all
> the details of the neighbors that it connects to.
>
> For example, the event and data that we receive from nova about an
> instance also contains the host (compute) that it sits on, and that is how
> we decide to connect it to the correct host.
>
>
>
> I think it is ok that the event of static (from driver to the processor)
> will contain for each entity it neighbors that it is supposed to be
> connected to.
>
>
>
> BR,
>
> Alexey
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Thursday, December 01, 2016 3:20 PM
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* [ALU] [openstack-dev] [vitrage] datasource driver returns
> entities only?
>
>
>
> During the implementation of static datasource driver[1], I got a question
> on the return format of `get_all` method.
>
>
>
> If I understand correctly, it should return a list of entities to be sent
> to the queue. Does it infer that the relationship should be embedded in
> entity, just like the legacy static_physical datasource?
>
>
>
> Suppose a link between two switches are created, how should we emit this
> change in `get_all` or `get_changes`?
>
>
>
> Currently I made a compromise by emitting the relationship as an update of
> the connected entity. This is not very elegant and it seems we are going
> back to the legacy static_physical datasource.
>
>
>
> [1] https://review.openstack.org/#/c/405354/
>
> --
>
> Yujun
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [vitrage] datasource driver returns entities only?

2016-12-01 Thread Yujun Zhang

During the implementation of static datasource driver[1], I got a question
on the return format of `get_all` method.

If I understand correctly, it should return a list of entities to be sent
to the queue. Does it infer that the relationship should be embedded in
entity, just like the legacy static_physical datasource?

Suppose a link between two switches are created, how should we emit this
change in `get_all` or `get_changes`?

Currently I made a compromise by emitting the relationship as an update of
the connected entity. This is not very elegant and it seems we are going
back to the legacy static_physical datasource.

[1] https://review.openstack.org/#/c/405354/
--
Yujun
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [vitrage] entity_type in make_pickleable

2016-11-29 Thread Yujun Zhang

Several properties are added to entity in `DriverBase.make_pickleable`[1]
but the existence check is only applied in `_add_entity_type`.

I think it may not be necessary after we prepended vitrage namespace[2]. I
could not think of a scenario to let user shadow the built-in `entity_type`.

Is there any particular reason for keeping it? Or should we remove it?

@staticmethod
def _add_entity_type(entity, entity_type):
if DSProps.ENTITY_TYPE not in entity:
entity[DSProps.ENTITY_TYPE] = entity_type


[1]:
https://github.com/openstack/vitrage/blob/master/vitrage/datasources/driver_base.py#L54

[2]: https://review.openstack.org/#/c/399417/
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ALU] [vitrage] common vs utils

2016-11-29 Thread Yujun Zhang

Ready for review

https://review.openstack.org/#/c/404517/

On Mon, Nov 28, 2016 at 5:50 PM Weyl, Alexey (Nokia - IL) <
alexey.w...@nokia.com> wrote:

> Sounds good to me J
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Monday, November 28, 2016 10:43 AM
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* [ALU] [openstack-dev] [vitrage] common vs utils
>
>
>
> Currently, we have `common/utils.py`, `common/file_utils.py` and an empty
> module `utils` in `vitrage`.
>
>
>
> In my understanding, `common` means *common to vitrage package* and utils
> are more *general purpose utility* functions.
>
>
>
> Would it better that we move `utils.py`, `file_utils.py` and
> `datetime_utils.py` to `utils`?
>
>
>
> --
>
> Yujun
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Suspected SPAM - Re: Suspected SPAM - [vitrage] datasources terminology

2016-11-29 Thread Yujun Zhang

LGTM

On Tue, Nov 29, 2016 at 6:38 PM Weyl, Alexey (Nokia - IL) <
alexey.w...@nokia.com> wrote:

> Sounds good to me :)
>
> From: Afek, Ifat (Nokia - IL) [mailto:ifat.a...@nokia.com]
> Sent: Tuesday, November 29, 2016 11:23 AM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Suspected SPAM - Re: [openstack-dev] Suspected SPAM - [vitrage]
> datasources terminology
>
> Hi,
>
> A new suggestion:
>
> • event_type: the type of event/notification that comes from the
> datasource. For example: ‘compute.instance.delete.end’,
>  ‘volume.detach.start’
> • graph_action: used to tell the evaluator what to do with the entity -
> create, update or delete it.
> • datasource_action: the type of action that was performed by the driver:
> init_snapshot, snapshot or update.
>
> What do you think?
> Ifat.
>
>
> From: "Afek, Ifat (Nokia - IL)"
> Reply-To: "OpenStack Development Mailing List (not for usage questions)"
> Date: Tuesday, 29 November 2016 at 09:26
> To: "OpenStack Development Mailing List (not for usage questions)"
> Subject: Suspected SPAM - [openstack-dev] [vitrage] datasources terminology
>
> Hi,
>
> I think we have a confusing terminology in Vitrage datasources.
>
> The following parameters are used in the drivers and transformers:
> • event_type: the type of event/notification that comes from the
> datasource. For example: ‘compute.instance.delete.end’,
>  ‘volume.detach.start’
> • event_action: used to tell the evaluator what to do with the entity -
> create, update or delete it.
> • action_type: the type of action that was performed by the driver:
> init_snapshot, snapshot or update.
>
> Personally I find these names very confusing.
> My suggestion is to rename action_type to datasource_mode, or something
> similar.
>
> Your thoughts?
>
> Thanks,
> Ifat.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [vitrage] common vs utils

2016-11-28 Thread Yujun Zhang

Currently, we have `common/utils.py`, `common/file_utils.py` and an empty
module `utils` in `vitrage`.

In my understanding, `common` means *common to vitrage package* and utils
are more *general purpose utility* functions.

Would it better that we move `utils.py`, `file_utils.py` and
`datetime_utils.py` to `utils`?

--
Yujun
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] about aodh alarm notification

2016-11-27 Thread Yujun Zhang

Agree.

It seems I missed the first clause in Alexeys 2nd proposal. `get_all` in
constructor could be a good solution.

On Mon, Nov 28, 2016 at 12:18 PM <dong.wenj...@zte.com.cn> wrote:



Hi all,

Maybe call the get_all method in the AodhDriver constructor and cache all
the alarms is a simple way.

BR,
dwj



*Yujun Zhang <zhangyujun+...@gmail.com <zhangyujun%2b...@gmail.com>>*

2016-11-28 11:23
收件人
"OpenStack Development Mailing List (not for usage questions)" <
openstack-dev@lists.openstack.org>, "dong.wenj...@zte.com.cn" <
dong.wenj...@zte.com.cn>

抄送
"Hefetz, Idan (Nokia - IL)" <idan.hef...@nokia.com>, "
zhang.yuj...@zte.com.cn" <zhang.yuj...@zte.com.cn>

主题
Re: [openstack-dev] [vitrage] about aodh alarm notification




Hi, Alexey

My comments inline.

On Mon, Nov 28, 2016 at 1:52 AM Weyl, Alexey (Nokia - IL) <
*alexey.w...@nokia.com* <alexey.w...@nokia.com>> wrote:
Hi Dong,



I can think of 2 solutions for this problem:

1.   We can talk with the AODH developers and check if they can add
additional data for the aodh notifications.

I think it might not be easily accepted since sending only the changes on
update notification looks more reasonable to me.
2.   We can add a cache in the aodh driver, and call the get_all method
in the AodhDriver constructor or when the first notification happens and
fill the cache with the data. Then for each notification that arrives you
will update that cache in the aodh notification service, and send then
event with all the data you need.

I doubt this will degrade the performance for the first notification since
there will be additional conversation to retrieve the full data.

My proposal is to share a cache between snapshot service and listener
service and I think it is a common requirements for all datasource using
PUSH method.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] about aodh alarm notification

2016-11-27 Thread Yujun Zhang

Hi, Alexey

My comments inline.

On Mon, Nov 28, 2016 at 1:52 AM Weyl, Alexey (Nokia - IL) <
alexey.w...@nokia.com> wrote:

> Hi Dong,
>
>
>
> I can think of 2 solutions for this problem:
>
> 1.   We can talk with the AODH developers and check if they can add
> additional data for the aodh notifications.
>
I think it might not be easily accepted since sending only the changes on
update notification looks more reasonable to me.

> 2.   We can add a cache in the aodh driver, and call the get_all
> method in the AodhDriver constructor or when the first notification happens
> and fill the cache with the data. Then for each notification that arrives
> you  will update that cache in the aodh notification service, and send then
> event with all the data you need.
>
I doubt this will degrade the performance for the first notification since
there will be additional conversation to retrieve the full data.

My proposal is to share a cache between snapshot service and listener
service and I think it is a common requirements for all datasource using
PUSH method.

>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage]bp:static-datasource-config-formatworking items

2016-11-15 Thread Yujun Zhang

On Tue, Nov 15, 2016 at 8:44 PM Weyl, Alexey (Nokia - IL) <
alexey.w...@nokia.com> wrote:

> Ok, sounds good.
>
> We need to understand what is the common way in openstack to work with
> deprecated code.
>

Maybe it could be discussed in next weekly meeting. Anybody who has
experience on it please raise your hand :-)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ALU] [vitrage] bp:static-datasource-config-formatworking items

2016-11-15 Thread Yujun Zhang

Hi, Alexey

I plan to split the implementation to several steps, because it will take
weeks to complete. I'm afraid it would be too big a patch to review if I
submit all changes in one patch set.

Instead I want to get comments as earlier as possible. Each submit will be
covered by additional unit test and keep backward compatibility.

Detail replies inline.

On Tue, Nov 15, 2016 at 5:51 PM Weyl, Alexey (Nokia - IL) <
alexey.w...@nokia.com> wrote:

> Hi Yujun,
>
> Good job! This is a very important change for Vitrage.
>
> I have a couple of questions please:
> 1. Why do we want to create a new datasource ‘static’ and not rename the
> current ‘static_physical’ datasource and change it to work with the new
> format?
>
I don't want to break it during the evolution.

> 2. How are you planning to use the old 'static_physical' datasource  as a
> proxy if you said that you want to remove it?
>
Good point. Any suggestion on how we hide the deprecated modules? Move it
as a submodule in new module.

> 3. What kind of merge is needed in the evaluator?
>
Parsing of `definition` section would be a common module for both evaluator
and static data source

> 4. I saw the implementation of the driver.py in static, and it doesn't do
> anything at the moment? (if you are still working on one of the patches
> then please market it as -1 in the code-review that we will know that you
> are still working on it.
>
Yes, skeleton is skeleton. Since I'm working remotely with vitrage team. I
want to keep you updated on the progress. Still, it is complete with unit
test and backward compatibility and will reduce further review work.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [vitrage] bp:static-datasource-config-format working items

2016-11-14 Thread Yujun Zhang

Hi folks.

I have just started working on the blueprint about static datasource config
format[1]. The planned working items are as following.

   1. create new datasource `static` to parse new configuration format
   2. parse old configuration format in `static` with a proxy to existing
   `static_physical` module
   3. remove `static_physical` datasource and print deprecation warning in
   `static`
   4. merge common logic with scenario template evaluator

Requesting for comments.

P.S. I chose the name `static` since it is actually not limited to physical
entities. Virtual entities can also be described in `static` file if there
is no dynamic source.

[1]:
https://blueprints.launchpad.net/vitrage/+spec/static-datasource-config-format

--
Yujun
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [javascript] developer meetup at Barcelona Summit

2016-10-20 Thread Yujun Zhang

Dear JavaScript Ninjas,

We are planning a meetup of JavaScript developers in Barcelona Summit.

It is scheduled on Tuesday and we may talk about the javascript SDK for
openstack and the first MVP.

Anyone who is interested in it please contact me.

--
Yujun
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Vitrage] Installation

2016-10-05 Thread Yujun Zhang

Hi, Paul,

You may check whether all required services have been started normally.
Below is the example output from an instance created by
https://github.com/openzero-zte/vitrage-demo

The topology service is on `vitrage-graph` while the alarm service is on
`vitrage-notifier` if I understand it correctly.

ubuntu@vitrage-demo:~$ ps aux|grep vitrage
stack 5737  0.0  0.0 250496  5704 ?Sl   Oct02   0:08
(wsgi:vitrage-api -k start
stack 5738  0.0  0.0 250488  5704 ?Sl   Oct02   0:08
(wsgi:vitrage-api -k start
stack 7634  2.3  0.9 398724 74364 pts/25   S+   Aug30 1209:08
/usr/bin/python /usr/local/bin/vitrage-graph --config-file
/etc/vitrage/vitrage.conf
stack 8000  0.0  0.8 397696 70196 pts/25   S+   Aug30   3:16
/usr/bin/python /usr/local/bin/vitrage-graph --config-file
/etc/vitrage/vitrage.conf
stack 8001  0.0  0.8 399256 72144 pts/25   S+   Aug30  20:25
/usr/bin/python /usr/local/bin/vitrage-graph --config-file
/etc/vitrage/vitrage.conf
stack 8002  0.0  0.9 403712 76720 pts/25   S+   Aug30   3:55
/usr/bin/python /usr/local/bin/vitrage-graph --config-file
/etc/vitrage/vitrage.conf
stack 8003  0.0  0.8 397952 70676 pts/25   S+   Aug30   2:27
/usr/bin/python /usr/local/bin/vitrage-graph --config-file
/etc/vitrage/vitrage.conf
stack 8116  0.0  0.5 190668 43208 pts/26   Sl+  Aug30   1:17
/usr/bin/python /usr/local/bin/vitrage-notifier --config-file
/etc/vitrage/vitrage.conf
rabbitmq 25657  1.0  1.6 2344552 133980 ?  Sl   Aug30 559:24
/usr/lib/erlang/erts-5.10.4/bin/beam.smp -W w -K true -A30 -P 1048576 --
-root /usr/lib/erlang -progname erl -- -home /var/lib/rabbitmq -- -pa
/usr/lib/rabbitmq/lib/rabbitmq_server-3.2.4/sbin/../ebin -noshell -noinput
-s rabbit boot -sname rabbit@vitrage-demo -boot start_sasl -kernel
inet_default_connect_options [{nodelay,true}] -sasl errlog_type error -sasl
sasl_error_logger false -rabbit error_logger
{file,"/var/log/rabbitmq/rab...@vitrage-demo.log"} -rabbit
sasl_error_logger {file,"/var/log/rabbitmq/rab...@vitrage-demo-sasl.log"}
-rabbit enabled_plugins_file "/etc/rabbitmq/enabled_plugins" -rabbit
plugins_dir "/usr/lib/rabbitmq/lib/rabbitmq_server-3.2.4/sbin/../plugins"
-rabbit plugins_expand_dir
"/var/lib/rabbitmq/mnesia/rabbit@vitrage-demo-plugins-expand" -os_mon
start_cpu_sup false -os_mon start_disksup false -os_mon start_memsup false
-mnesia dir "/var/lib/rabbitmq/mnesia/rabbit@vitrage-demo"


On Wed, Oct 5, 2016 at 9:32 PM Paul Vaduva  wrote:

> Hi Ifat,
>
>
>
> Thanks you for the reply. I saw that this is for build on a virtual
> environment. I was courios if there is a way to install it on a deployed
> horizon. I tried to adapt the instructions to install it on my existing
> horizon installation (maily not use
>
> ./horizon/tools/with_venv.sh and copy cp -a vitragedashboard/enabled/* and
> the vitrageclient/api/* to my existing  horizon installation and the
> dashboard apeared on my web browser but
>
> · For Topology tab I get this error *Error:* Unable to fetch the
> Vitrage Topology service.
>
> · For Alarms tab I get *Error:* Unable to fetch the Vitrage
> Alarms service.
>
> · For Entity Graph I get *Error:* Unable to fetch the Vitrage
> Topology service.
>
>
>
> Best Regards,
>
> Paul
>
>
>
>
>
> *From:* Afek, Ifat (Nokia - IL) [mailto:ifat.a...@nokia.com]
> *Sent:* Wednesday, October 05, 2016 1:15 PM
> *To:* OpenStack Development Mailing List (not for usage questions) <
> openstack-dev@lists.openstack.org>
> *Subject:* Re: [openstack-dev] [Vitrage] Installation
>
>
>
> Hi Paul,
>
>
>
> Please find the installation and configuration instructions here:
> https://github.com/openstack/vitrage/blob/master/doc/source/installation-and-configuration.rst
>
> Let us know if you have any other questions, we’ll be happy to help.
>
>
>
> Best regards,
>
> Ifat.
>
>
>
>
>
> *From: *Paul Vaduva
> *Reply-To: *"OpenStack Development Mailing List (not for usage questions)"
> *Date: *Tuesday, 4 October 2016 at 16:03
> *To: *"openstack-dev@lists.openstack.org"
> *Subject: *[openstack-dev] [Vitrage] Installation
>
>
>
> Hi everybody,
>
>
>
> I am writing you to ask for guidance with the installation of Vitrage.
>
> I have a deployed opnfv environment based on openstack mitaka release and
> I want to ask you how I would go about installing vitrage (with horizon) on
> this existing environment
>
>
>
>
>
> *Paul Ionut Vaduva*
>
> Software Engineer
>
> Linux Team
>
>
>
> *Email*  *paul.vad...@enea.com *
>
>
>
> *Enea R*
>
> 319 Splaiul Independentei, OB403A
>
> Bucharest, 060044
>
> *Phone*  +4021311 43 00
>
> *Fax *+402131143 01
>
>
>
>
>
> www.enea.com
>
>
>
> [image: cid:image002.png@01D00576.7D651240]
>
> This message, including attachments, is CONFIDENTIAL. It may also be
> privileged or otherwise protected by law. If you received this email by
> mistake please let us know by reply and then delete it from your system;
> you should not copy it or disclose its contents

Re: [openstack-dev] [vitrage] inspecting external openstack environment

2016-09-06 Thread Yujun Zhang

I see. I'll draft a blueprint after essential study on this requirement.

And thank you for the information on add-new-datasource document.
--
Yujun

On Wed, Sep 7, 2016 at 1:31 PM Weyl, Alexey (Nokia - IL) <
alexey.w...@nokia.com> wrote:

> Hi Yujun,
>
> I don’t think we have a support for different versions of openstack at the
> same time at the moment.
>
> We have thought of such a requirement yet, but I think it can be done.
>
> If you can, please add a blueprint with all the details for such a feature.
>
> In addition, we now have a documentation for how to add a new datasource:
>
>
> https://github.com/openstack/vitrage/blob/master/doc/source/add-new-datasource.rst
>
>
>
> Alexey
>
>
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Wednesday, September 07, 2016 6:08 AM
>
>
> *To:* OpenStack Development Mailing List (not for usage questions)
>
> *Subject:* Re: [openstack-dev] [vitrage] inspecting external openstack
> environment
>
>
>
> One additional question about openstack datasource.
>
>
>
> Is it possible to connect *multiple* openstack data sources, furthermore,
> different version of openstack (liberty, newton), at the same time?
>
>
>
> To monitor existing systems, this might be a common requirement.
>
> On Thu, Sep 1, 2016 at 5:16 PM Yujun Zhang <zhangyujun+...@gmail.com>
> wrote:
>
> Thanks for explanation. I'll give it a try.
>
>
>
> Answer to BTW, it is Liberty on Ubuntu 14.04 deployed by Fuel 8.0.
>
> --
>
> Yujun
>
>
>
> On Wed, Aug 31, 2016 at 2:42 PM Afek, Ifat (Nokia - IL) <
> ifat.a...@nokia.com> wrote:
>
> Hi Yujun,
>
>
>
> Using Vitrage for an external openstack environment should be possible,
> but it has never been tested.
>
>
>
> There are some issues to consider:
>
>- You should set the ip of the keystone in vitrage.conf, and make sure
>you can access it
>- You will need to install, together with Vitrage, many dependent
>openstack libraries
>- In order to access Vitrage API, you will either need to install
>keystone on Vitrage environment, or to configure an endpoint for Vitrage in
>the existing openstack
>- For notifications, you need to connect Vitrage to the oslo bus of
>the existing openstack
>
> BTW, what is the version of this openstack environment?
>
>
>
> Best Regards,
>
> Ifat.
>
>
>
> *From: *Yujun Zhang
> *Reply-To: *"OpenStack Development Mailing List (not for usage questions)"
> *Date: *Tuesday, 30 August 2016 at 10:06
> *To: *"OpenStack Development Mailing List (not for usage questions)"
> *Subject: *[openstack-dev] [vitrage] inspecting external openstack
> environment
>
>
>
> My purpose is to inspect an **existing** openstack environment with
> vitrage.
>
>
>
> Do I have to install vitrage on the target environment or it can be done
> by proper configuration?
>
>
>
> --
>
> Yujun
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] inspecting external openstack environment

2016-09-06 Thread Yujun Zhang

One additional question about openstack datasource.

Is it possible to connect *multiple* openstack data sources, furthermore,
different version of openstack (liberty, newton), at the same time?

To monitor existing systems, this might be a common requirement.

On Thu, Sep 1, 2016 at 5:16 PM Yujun Zhang <zhangyujun+...@gmail.com> wrote:

> Thanks for explanation. I'll give it a try.
>
> Answer to BTW, it is Liberty on Ubuntu 14.04 deployed by Fuel 8.0.
> --
> Yujun
>
> On Wed, Aug 31, 2016 at 2:42 PM Afek, Ifat (Nokia - IL) <
> ifat.a...@nokia.com> wrote:
>
>> Hi Yujun,
>>
>> Using Vitrage for an external openstack environment should be possible,
>> but it has never been tested.
>>
>> There are some issues to consider:
>>
>>- You should set the ip of the keystone in vitrage.conf, and make
>>sure you can access it
>>- You will need to install, together with Vitrage, many dependent
>>openstack libraries
>>- In order to access Vitrage API, you will either need to install
>>keystone on Vitrage environment, or to configure an endpoint for Vitrage 
>> in
>>the existing openstack
>>- For notifications, you need to connect Vitrage to the oslo bus of
>>the existing openstack
>>
>> BTW, what is the version of this openstack environment?
>>
>> Best Regards,
>> Ifat.
>>
>> From: Yujun Zhang
>> Reply-To: "OpenStack Development Mailing List (not for usage questions)"
>> Date: Tuesday, 30 August 2016 at 10:06
>> To: "OpenStack Development Mailing List (not for usage questions)"
>> Subject: [openstack-dev] [vitrage] inspecting external openstack
>> environment
>>
>> My purpose is to inspect an **existing** openstack environment with
>> vitrage.
>>
>> Do I have to install vitrage on the target environment or it can be done
>> by proper configuration?
>>
>> --
>> Yujun
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] relationship_type in static_datasources

2016-09-05 Thread Yujun Zhang

On Mon, Sep 5, 2016 at 5:49 PM Afek, Ifat (Nokia - IL) <ifat.a...@nokia.com>
wrote:

>
> From: Yujun Zhang
> Date: Friday, 2 September 2016 at 08:47
> ...
> Cool.
> Just please note that you can’t push it to master at the moment, as we are
> in feature freeze. Once stable/newton is created we will be able to start
> using the master branch for Ocata development.
>

When will it probably be?

> Also note that it means datasource configurations from Ocata will be
> different than the older versions, so we might want to mark them somehow,
> and keep the old parser for backwards compatibility.
>

OK, I'll take this into consideration.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] relationship_type in static_datasources

2016-09-01 Thread Yujun Zhang

On Fri, Sep 2, 2016 at 2:44 AM Afek, Ifat (Nokia - IL) 
wrote:

> I think you have a point. We can indeed use the templates definitions for
> the static datasources as well.
>

If agreed by the team, I will get started to implement it.

@all, please share your opinions. All your comments are welcomed.

BTW, I noticed we already have a blueprint for this issue:
> https://blueprints.launchpad.net/vitrage/+spec/add-direction-to-physical-topology-relationships,
> but nobody has handled it so far.
>

This issue will be resolved without extra effort once we adopt the scenario
template for static resource. I can volunteer to handle it.

--
Yujun
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] inspecting external openstack environment

2016-09-01 Thread Yujun Zhang

Thanks for explanation. I'll give it a try.

Answer to BTW, it is Liberty on Ubuntu 14.04 deployed by Fuel 8.0.
--
Yujun

On Wed, Aug 31, 2016 at 2:42 PM Afek, Ifat (Nokia - IL) <ifat.a...@nokia.com>
wrote:

> Hi Yujun,
>
> Using Vitrage for an external openstack environment should be possible,
> but it has never been tested.
>
> There are some issues to consider:
>
>- You should set the ip of the keystone in vitrage.conf, and make sure
>you can access it
>- You will need to install, together with Vitrage, many dependent
>openstack libraries
>- In order to access Vitrage API, you will either need to install
>keystone on Vitrage environment, or to configure an endpoint for Vitrage in
>the existing openstack
>- For notifications, you need to connect Vitrage to the oslo bus of
>the existing openstack
>
> BTW, what is the version of this openstack environment?
>
> Best Regards,
> Ifat.
>
> From: Yujun Zhang
> Reply-To: "OpenStack Development Mailing List (not for usage questions)"
> Date: Tuesday, 30 August 2016 at 10:06
> To: "OpenStack Development Mailing List (not for usage questions)"
> Subject: [openstack-dev] [vitrage] inspecting external openstack
> environment
>
> My purpose is to inspect an **existing** openstack environment with
> vitrage.
>
> Do I have to install vitrage on the target environment or it can be done
> by proper configuration?
>
> --
> Yujun
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [vitrage] blueprint proposal: graph log inspector

2016-09-01 Thread Yujun Zhang

Dear all,

I proposed a new blueprint about graph log inspector on gerrit [1] and
would be happy to receive everyone's comments.

[1] https://review.openstack.org/#/c/364140

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] relationship_type in static_datasources

2016-09-01 Thread Yujun Zhang

Still failing after update the json file [1]

Yes, I think I shall need more help to understand how the mock_sync works.
@Elisha

And also the tox and tempest tests. @Alexey.

[1] https://review.openstack.org/#/c/362525/3

On Tue, Aug 30, 2016 at 4:02 PM Rosensweig, Elisha (Nokia - IL) <
elisha.rosensw...@nokia.com> wrote:

> Yes. Please add it to the file
>
>
>
>
> /workspace/dev/vitrage/vitrage/tests/resources/mock_configurations/driver/driver_switch_snapshot_dynamic.json
>
>
>
> under the "relationships" section, just like in your commit.
>
>
>
> If you need help understanding how to work with the mock_sync, let me know.
>
>
>
> Elisha
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Tuesday, August 30, 2016 9:59 AM
>
>
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* Re: [openstack-dev] [vitrage] relationship_type in
> static_datasources
>
>
>
> I added a new key 'is_source' to static physical configuration [1] and the
> test currently fails.
>
>
>
> Not sure we need to update mock_sync or not.
>
>
>
> [1]
> https://review.openstack.org/#/c/362525/1/vitrage/tests/resources/static_datasources/switch_to_host_datasource.yaml
>
>
>
> On Tue, Aug 30, 2016 at 2:53 PM Rosensweig, Elisha (Nokia - IL) <
> elisha.rosensw...@nokia.com> wrote:
>
> What is the problem you are running into with mock_sync?
>
> Elisha
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Tuesday, August 30, 2016 5:09 AM
>
>
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* Re: [openstack-dev] [vitrage] relationship_type in
> static_datasources
>
>
>
> Patch work in progress [1] but local test fails [2].
>
>
>
> It seems to be caused by the mock_sync.
>
>
>
> I'm still looking into it. Any help would be appreciated.
>
>
>
> [1] https://review.openstack.org/#/c/362525
>
> [2] http://pastebin.com/iepqxUAP
>
>
>
>
>
> On Mon, Aug 29, 2016 at 4:59 PM Yujun Zhang <zhangyujun+...@gmail.com>
> wrote:
>
> Thanks, Alexey. Point 1 and 3 are pretty clear.
>
>
>
> As for point 2, if I understand it correctly, you are suggesting to modify
> the static_physical.yaml as following
>
> entities:
>  - type: switch
>name: switch-1
>id: switch-1 # should be same as name
>state: available
>relationships:
>  - type: nova.host
>name: host-1
>id: host-1 # should be same as name
>
> *   is_source: true # entity is `source` in this relationship*
>
>relation_type: attached
>
>  - type: switch
>
>name: switch-2
>
>id: switch-2 # should be same as name
>
>
> *   is_source: false # entity is `target` in this relationship*   
> relation_type: backup
>
> But I wonder why the static physical configuration file use a different
> format from vitrage template definitions[1]
>
>
>
> [1]
> https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-template-format.rst
>
>
>
> On Sun, Aug 28, 2016 at 4:14 PM Weyl, Alexey (Nokia - IL) <
> alexey.w...@nokia.com> wrote:
>
> Hi Yujun,
>
>
>
> In order for the static_physical to work for different datasources without
> the restrictions, you need to do the following changes:
>
> Go to the static_physical transformer:
>
> 1.   Remove the methods: _register_relations_direction,
> _find_relation_direction_source.
>
> 2.   Add to the static_physical.yaml for each definition also a field
> for direction which will indicate the source and the destination between
> the datasources.
>
> 3.   In method: _create_neighbor, remove the usage of method
> _find_relation_direction_source, and use the new definition from the yaml
> file here to decide the edge direction.
>
>
>
> Is it ok?
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Friday, August 26, 2016 4:22 AM
>
>
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* Re: [openstack-dev] [vitrage] relationship_type in
> static_datasources
>
>
>
> Lost in the code...It seems the datasource just construct the entities and
> send them over event bus to entity graph processor. I need to dig further
> to find out the exact point the "backup" relationship is filtered.
>
>
>
> I think we should some how keep the validation of relationship type. It is
> so easy to make typo when creating the template manually (I did this quite
> often...).
>
>
>
> My idea is to delegate th

Re: [openstack-dev] [vitrage] relationship_type in static_datasources

2016-08-31 Thread Yujun Zhang

Hi, Ifat,

The static configuration contains definitions of `entities` and *their*
`relationships while the scenario templates contains a definition section
which includes `entities` and `relationships` *between them*. An outline of
these two format are as below.

static configuration

- entities
  - {entity}
  - {entity}

for each entity

- name:
  id:
  relationship:
- {relationship}
- {relationship}

scenario templates

- definitions
  - entities
- {entity}
- {entity}
  - relationships
- {relationship}
- {relationship}

Though serving different purpose, they both

   1. describe entities and relationships
   2. use a dedicated key (id/template_id) to reference the items
   3. include a source entity and target entity in relationship

The main differences between the two are


   - scenario *defines rules *(entity and relationship matching)*, *graph
   update is triggered when entities are added by datasource.
   - static configuration *defines rules* and also *add entities* to graph

The rule definition are common to these two modules. We may define the
static configuration using the same format as scenario template. And then
simulate an entity discovery from the same file.

By reusing the template parsing engine and workflow, we may reduce the work
in maintenance and bring in new features more easily.

We may discuss it further if anything unclear.

On Tue, Aug 30, 2016 at 11:07 PM Afek, Ifat (Nokia - IL) <
ifat.a...@nokia.com> wrote:

> Hi Yujun,
>
> From: Yujun Zhang
> Date: Monday, 29 August 2016 at 11:59
>
> entities:
>  - type: switch
>name: switch-1
>id: switch-1 # should be same as name
>state: available
>relationships:
>  - type: nova.host
>name: host-1
>id: host-1 # should be same as name*   is_source: true # entity is 
> `source` in this relationship
> *   relation_type: attached - type: switch   name: switch-2   
> id: switch-2 # should be same as name
> *   is_source: false # entity is `target` in this relationship*   
> relation_type: backup
>
>
> I think that’s the idea, instead of making this assumption in the code.
>
> But I wonder why the static physical configuration file use a different
> format from vitrage template definitions[1]
>
> [1]
> https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-template-format.rst
>
>
> What do you mean? The purpose of the templates is to describe the
> condition-action behaviour, wheres the purpose of the static configuration
> is to define resources to be added to vitrage graph. Can you please explain
> how you would make the formats more similar?
>
> Best Regards,
> Ifat.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [vitrage] inspecting external openstack environment

2016-08-30 Thread Yujun Zhang

My purpose is to inspect an **existing** openstack environment with vitrage.

Do I have to install vitrage on the target environment or it can be done by
proper configuration?

--
Yujun
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] relationship_type in static_datasources

2016-08-30 Thread Yujun Zhang

I added a new key 'is_source' to static physical configuration [1] and the
test currently fails.

Not sure we need to update mock_sync or not.

[1]
https://review.openstack.org/#/c/362525/1/vitrage/tests/resources/static_datasources/switch_to_host_datasource.yaml

On Tue, Aug 30, 2016 at 2:53 PM Rosensweig, Elisha (Nokia - IL) <
elisha.rosensw...@nokia.com> wrote:

> What is the problem you are running into with mock_sync?
>
> Elisha
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Tuesday, August 30, 2016 5:09 AM
>
>
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* Re: [openstack-dev] [vitrage] relationship_type in
> static_datasources
>
>
>
> Patch work in progress [1] but local test fails [2].
>
>
>
> It seems to be caused by the mock_sync.
>
>
>
> I'm still looking into it. Any help would be appreciated.
>
>
>
> [1] https://review.openstack.org/#/c/362525
>
> [2] http://pastebin.com/iepqxUAP
>
>
>
>
>
> On Mon, Aug 29, 2016 at 4:59 PM Yujun Zhang <zhangyujun+...@gmail.com>
> wrote:
>
> Thanks, Alexey. Point 1 and 3 are pretty clear.
>
>
>
> As for point 2, if I understand it correctly, you are suggesting to modify
> the static_physical.yaml as following
>
> entities:
>  - type: switch
>name: switch-1
>id: switch-1 # should be same as name
>state: available
>relationships:
>  - type: nova.host
>name: host-1
>id: host-1 # should be same as name
>
> *   is_source: true # entity is `source` in this relationship*
>
>relation_type: attached
>
>  - type: switch
>
>name: switch-2
>
>id: switch-2 # should be same as name
>
>
> *   is_source: false # entity is `target` in this relationship*   
> relation_type: backup
>
> But I wonder why the static physical configuration file use a different
> format from vitrage template definitions[1]
>
>
>
> [1]
> https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-template-format.rst
>
>
>
> On Sun, Aug 28, 2016 at 4:14 PM Weyl, Alexey (Nokia - IL) <
> alexey.w...@nokia.com> wrote:
>
> Hi Yujun,
>
>
>
> In order for the static_physical to work for different datasources without
> the restrictions, you need to do the following changes:
>
> Go to the static_physical transformer:
>
> 1.   Remove the methods: _register_relations_direction,
> _find_relation_direction_source.
>
> 2.   Add to the static_physical.yaml for each definition also a field
> for direction which will indicate the source and the destination between
> the datasources.
>
> 3.   In method: _create_neighbor, remove the usage of method
> _find_relation_direction_source, and use the new definition from the yaml
> file here to decide the edge direction.
>
>
>
> Is it ok?
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Friday, August 26, 2016 4:22 AM
>
>
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* Re: [openstack-dev] [vitrage] relationship_type in
> static_datasources
>
>
>
> Lost in the code...It seems the datasource just construct the entities and
> send them over event bus to entity graph processor. I need to dig further
> to find out the exact point the "backup" relationship is filtered.
>
>
>
> I think we should some how keep the validation of relationship type. It is
> so easy to make typo when creating the template manually (I did this quite
> often...).
>
>
>
> My idea is to delegate the validation to datasource instead of enumerating
> all constants it in evaluator. I think this will introduce better
> extensibility. Any comments?
>
>
>
> On Thu, Aug 25, 2016 at 1:32 PM Weyl, Alexey (Nokia - IL) <
> alexey.w...@nokia.com> wrote:
>
> Hi Yujun,
>
>
>
> You can find the names of the lables in the constants.py file.
>
>
>
> In addition, the restriction on the physical_static datasource is done in
> it’s driver.py.
>
>
>
> Alexey
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Thursday, August 25, 2016 4:50 AM
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* Re: [openstack-dev] [vitrage] relationship_type in
> static_datasources
>
>
>
> Hi, Ifat,
>
>
>
> I searched for edge_labels in the project. It seems it is validated only
> in `vitrage/evaluator/template_validation/template_syntax_validator.py`.
> Where is such restriction applied in static_datasources?
>
>
>
> --
>
> Yuju

Re: [openstack-dev] [vitrage] relationship_type in static_datasources

2016-08-29 Thread Yujun Zhang

Patch work in progress [1] but local test fails [2].

It seems to be caused by the mock_sync.

I'm still looking into it. Any help would be appreciated.

[1] https://review.openstack.org/#/c/362525
[2] http://pastebin.com/iepqxUAP


On Mon, Aug 29, 2016 at 4:59 PM Yujun Zhang <zhangyujun+...@gmail.com>
wrote:

> Thanks, Alexey. Point 1 and 3 are pretty clear.
>
> As for point 2, if I understand it correctly, you are suggesting to modify
> the static_physical.yaml as following
>
> entities:
>  - type: switch
>name: switch-1
>id: switch-1 # should be same as name
>state: available
>relationships:
>  - type: nova.host
>name: host-1
>id: host-1 # should be same as name*   is_source: true # entity is 
> `source` in this relationship
> *   relation_type: attached - type: switch   name: switch-2   
> id: switch-2 # should be same as name
> *   is_source: false # entity is `target` in this relationship*   
> relation_type: backup
>
> But I wonder why the static physical configuration file use a different
> format from vitrage template definitions[1]
>
> [1]
> https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-template-format.rst
>
> On Sun, Aug 28, 2016 at 4:14 PM Weyl, Alexey (Nokia - IL) <
> alexey.w...@nokia.com> wrote:
>
>> Hi Yujun,
>>
>>
>>
>> In order for the static_physical to work for different datasources
>> without the restrictions, you need to do the following changes:
>>
>> Go to the static_physical transformer:
>>
>> 1.   Remove the methods: _register_relations_direction,
>> _find_relation_direction_source.
>>
>> 2.   Add to the static_physical.yaml for each definition also a
>> field for direction which will indicate the source and the destination
>> between the datasources.
>>
>> 3.   In method: _create_neighbor, remove the usage of method
>> _find_relation_direction_source, and use the new definition from the yaml
>> file here to decide the edge direction.
>>
>>
>>
>> Is it ok?
>>
>>
>>
>> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
>> *Sent:* Friday, August 26, 2016 4:22 AM
>>
>>
>> *To:* OpenStack Development Mailing List (not for usage questions)
>> *Subject:* Re: [openstack-dev] [vitrage] relationship_type in
>> static_datasources
>>
>>
>>
>> Lost in the code...It seems the datasource just construct the entities
>> and send them over event bus to entity graph processor. I need to dig
>> further to find out the exact point the "backup" relationship is filtered.
>>
>>
>>
>> I think we should some how keep the validation of relationship type. It
>> is so easy to make typo when creating the template manually (I did this
>> quite often...).
>>
>>
>>
>> My idea is to delegate the validation to datasource instead of
>> enumerating all constants it in evaluator. I think this will introduce
>> better extensibility. Any comments?
>>
>>
>>
>> On Thu, Aug 25, 2016 at 1:32 PM Weyl, Alexey (Nokia - IL) <
>> alexey.w...@nokia.com> wrote:
>>
>> Hi Yujun,
>>
>>
>>
>> You can find the names of the lables in the constants.py file.
>>
>>
>>
>> In addition, the restriction on the physical_static datasource is done in
>> it’s driver.py.
>>
>>
>>
>> Alexey
>>
>>
>>
>> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
>> *Sent:* Thursday, August 25, 2016 4:50 AM
>> *To:* OpenStack Development Mailing List (not for usage questions)
>> *Subject:* Re: [openstack-dev] [vitrage] relationship_type in
>> static_datasources
>>
>>
>>
>> Hi, Ifat,
>>
>>
>>
>> I searched for edge_labels in the project. It seems it is validated only
>> in `vitrage/evaluator/template_validation/template_syntax_validator.py`.
>> Where is such restriction applied in static_datasources?
>>
>>
>>
>> --
>>
>> Yujun
>>
>>
>>
>> On Wed, Aug 24, 2016 at 3:19 PM Afek, Ifat (Nokia - IL) <
>> ifat.a...@nokia.com> wrote:
>>
>> Hi Yujun,
>>
>>
>>
>> Indeed, we have some restrictions on the relationship types that can be
>> used in the static datasources. I think we should remove these
>> restrictions, and allow any kind of relationship type.
>>
>>
>>
>> Best regards,
>>
>> Ifat.
>>
>>
>>
>> *From: *Yujun Zhang
>> *Date: *Monday, 22 A

Re: [openstack-dev] [vitrage] relationship_type in static_datasources

2016-08-29 Thread Yujun Zhang

Thanks, Alexey. Point 1 and 3 are pretty clear.

As for point 2, if I understand it correctly, you are suggesting to modify
the static_physical.yaml as following

entities:
 - type: switch
   name: switch-1
   id: switch-1 # should be same as name
   state: available
   relationships:
 - type: nova.host
   name: host-1
   id: host-1 # should be same as name*   is_source: true #
entity is `source` in this relationship
*   relation_type: attached - type: switch   name:
switch-2   id: switch-2 # should be same as name
*   is_source: false # entity is `target` in this relationship*
   relation_type: backup

But I wonder why the static physical configuration file use a different
format from vitrage template definitions[1]

[1]
https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-template-format.rst

On Sun, Aug 28, 2016 at 4:14 PM Weyl, Alexey (Nokia - IL) <
alexey.w...@nokia.com> wrote:

> Hi Yujun,
>
>
>
> In order for the static_physical to work for different datasources without
> the restrictions, you need to do the following changes:
>
> Go to the static_physical transformer:
>
> 1.   Remove the methods: _register_relations_direction,
> _find_relation_direction_source.
>
> 2.   Add to the static_physical.yaml for each definition also a field
> for direction which will indicate the source and the destination between
> the datasources.
>
> 3.   In method: _create_neighbor, remove the usage of method
> _find_relation_direction_source, and use the new definition from the yaml
> file here to decide the edge direction.
>
>
>
> Is it ok?
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Friday, August 26, 2016 4:22 AM
>
>
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* Re: [openstack-dev] [vitrage] relationship_type in
> static_datasources
>
>
>
> Lost in the code...It seems the datasource just construct the entities and
> send them over event bus to entity graph processor. I need to dig further
> to find out the exact point the "backup" relationship is filtered.
>
>
>
> I think we should some how keep the validation of relationship type. It is
> so easy to make typo when creating the template manually (I did this quite
> often...).
>
>
>
> My idea is to delegate the validation to datasource instead of enumerating
> all constants it in evaluator. I think this will introduce better
> extensibility. Any comments?
>
>
>
> On Thu, Aug 25, 2016 at 1:32 PM Weyl, Alexey (Nokia - IL) <
> alexey.w...@nokia.com> wrote:
>
> Hi Yujun,
>
>
>
> You can find the names of the lables in the constants.py file.
>
>
>
> In addition, the restriction on the physical_static datasource is done in
> it’s driver.py.
>
>
>
> Alexey
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Thursday, August 25, 2016 4:50 AM
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* Re: [openstack-dev] [vitrage] relationship_type in
> static_datasources
>
>
>
> Hi, Ifat,
>
>
>
> I searched for edge_labels in the project. It seems it is validated only
> in `vitrage/evaluator/template_validation/template_syntax_validator.py`.
> Where is such restriction applied in static_datasources?
>
>
>
> --
>
> Yujun
>
>
>
> On Wed, Aug 24, 2016 at 3:19 PM Afek, Ifat (Nokia - IL) <
> ifat.a...@nokia.com> wrote:
>
> Hi Yujun,
>
>
>
> Indeed, we have some restrictions on the relationship types that can be
> used in the static datasources. I think we should remove these
> restrictions, and allow any kind of relationship type.
>
>
>
> Best regards,
>
> Ifat.
>
>
>
> *From: *Yujun Zhang
> *Date: *Monday, 22 August 2016 at 08:37
>
> I'm following the sample configuration in docs [1] to verify how static
> datasources works.
>
>
>
> It seems `backup` relationship is not displayed in the entity graph view
> and neither is it included in topology show.
>
>
>
> There is an enumeration for edge labels [2]. Should relationship in static
> datasource be limited to it?
>
>
>
> [1]
> https://github.com/openstack/vitrage/blob/master/doc/source/static-physical-config.rst
>
> [2]
> https://github.com/openstack/vitrage/blob/master/vitrage/common/constants.py#L49
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>

Re: [openstack-dev] [vitrage] relationship_type in static_datasources

2016-08-28 Thread Yujun Zhang

Thanks for replying. See my comments inline.

On Sun, Aug 28, 2016 at 8:02 PM Afek, Ifat (Nokia - IL) 
wrote:

> Hi Yujun,
>
> Regarding the validation – I agree that we should implement it in another
> way, but as a first stage I think you can just remove it.
>

OK

> If you have some thoughts regarding the way to implement it, we will be
> happy to hear them. We thought about a yaml file per datasource, but didn’t
> design it yet.
>

No problem. I'll put it to open discuss once I have a proposal.

> BTW, did you notice the new API for template validation?
> https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-api.rst#template-validate
>
>

I'll have a look at it.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] relationship_type in static_datasources

2016-08-25 Thread Yujun Zhang

Lost in the code...It seems the datasource just construct the entities and
send them over event bus to entity graph processor. I need to dig further
to find out the exact point the "backup" relationship is filtered.

I think we should some how keep the validation of relationship type. It is
so easy to make typo when creating the template manually (I did this quite
often...).

My idea is to delegate the validation to datasource instead of enumerating
all constants it in evaluator. I think this will introduce better
extensibility. Any comments?

On Thu, Aug 25, 2016 at 1:32 PM Weyl, Alexey (Nokia - IL) <
alexey.w...@nokia.com> wrote:

> Hi Yujun,
>
>
>
> You can find the names of the lables in the constants.py file.
>
>
>
> In addition, the restriction on the physical_static datasource is done in
> it’s driver.py.
>
>
>
> Alexey
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Thursday, August 25, 2016 4:50 AM
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* Re: [openstack-dev] [vitrage] relationship_type in
> static_datasources
>
>
>
> Hi, Ifat,
>
>
>
> I searched for edge_labels in the project. It seems it is validated only
> in `vitrage/evaluator/template_validation/template_syntax_validator.py`.
> Where is such restriction applied in static_datasources?
>
>
>
> --
>
> Yujun
>
>
>
> On Wed, Aug 24, 2016 at 3:19 PM Afek, Ifat (Nokia - IL) <
> ifat.a...@nokia.com> wrote:
>
> Hi Yujun,
>
>
>
> Indeed, we have some restrictions on the relationship types that can be
> used in the static datasources. I think we should remove these
> restrictions, and allow any kind of relationship type.
>
>
>
> Best regards,
>
> Ifat.
>
>
>
> *From: *Yujun Zhang
> *Date: *Monday, 22 August 2016 at 08:37
>
> I'm following the sample configuration in docs [1] to verify how static
> datasources works.
>
>
>
> It seems `backup` relationship is not displayed in the entity graph view
> and neither is it included in topology show.
>
>
>
> There is an enumeration for edge labels [2]. Should relationship in static
> datasource be limited to it?
>
>
>
> [1]
> https://github.com/openstack/vitrage/blob/master/doc/source/static-physical-config.rst
>
> [2]
> https://github.com/openstack/vitrage/blob/master/vitrage/common/constants.py#L49
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] entity graph layout

2016-08-25 Thread Yujun Zhang

After the first investigation, I think cytoscape might be too heavy. There
would be lots of refactoring work to migrate all functions to new library.
So I would suspend this proposal for now.

However, it seems the layout could be improved by adjusting the parameters
applied to force layout, e.g. charge, gravity and etc. When a larger charge
is assigned to cluster, it will push away the other element to avoid
overlapping.

But currently it is difficult to tune such parameters since the scripts are
compressed. *Any idea to speed up debug process?*

--
Yujun

On Tue, Aug 23, 2016 at 9:29 AM Yujun Zhang <zhangyujun+...@gmail.com>
wrote:

> I'm considering to use Cytoscape.js [1] to improve the layout for entity
> graph view.
>
> Cytoscape.js is a graph theory (a.k.a. network) library for analysis and
> visualisation which under active maintenance (latest release 2.7.8 on Aug
> 18, 2016) [2], while the current library d3-dagre [3] is declared not being
> actively developed or maintained.
>
> Meanwhile, I'm building a proof of concept for visualizing the entity
> graph with Cytoscape.
>
> Could anybody give a list on the required features for this view? Any
> comments are welcome.
>
> [1] http://js.cytoscape.org/
> [2] https://github.com/cytoscape/cytoscape.js
> [3] https://github.com/cpettitt/dagre-d3
>
>
> On Mon, Aug 8, 2016 at 2:34 PM Afek, Ifat (Nokia - IL) <
> ifat.a...@nokia.com> wrote:
>
>> There is no such blueprint at the moment.
>> You are more than welcome to add one, in case you have some ideas for
>> improvements.
>>
>> Ifat.
>>
>> From: Yujun Zhang
>> Date: Monday, 8 August 2016 at 09:21
>>
>>
>> Great, it works.
>> But it would be better if we could improve the default layout. Is there
>> any blueprint in progress?
>> --
>> Yujun
>>
>> On Sun, Aug 7, 2016 at 1:09 PM Afek, Ifat (Nokia - IL) <
>> ifat.a...@nokia.com> wrote:
>>
>>> Hi,
>>>
>>> It is possible to adjust the layout of the graph. You can double-click
>>> on a vertex and it will remain pinned to its place. You can then move the
>>> pinned vertices around to adjust the graph layout.
>>>
>>> Hope it helped, and let us know if you need additional help with your
>>> demo.
>>>
>>> Best Regards,
>>> Ifat.
>>>
>>>
>>> From: Yujun Zhang
>>> Date: Friday, 5 August 2016 at 09:32
>>>
>>> Hi, all,
>>>
>>> I'm building a demo of vitrage. The dynamic entity graph looks
>>> interesting.
>>>
>>> But when more entities are added, things becomes crowded and the links
>>> screw over each other. Dragging the items will not help much.
>>>
>>> Is it possible to adjust the layout so I can get a more regular/stable
>>> tree view of the entities?
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] relationship_type in static_datasources

2016-08-24 Thread Yujun Zhang

Hi, Ifat,

I searched for edge_labels in the project. It seems it is validated only in
`vitrage/evaluator/template_validation/template_syntax_validator.py`. Where
is such restriction applied in static_datasources?

--
Yujun

On Wed, Aug 24, 2016 at 3:19 PM Afek, Ifat (Nokia - IL) <ifat.a...@nokia.com>
wrote:

> Hi Yujun,
>
> Indeed, we have some restrictions on the relationship types that can be
> used in the static datasources. I think we should remove these
> restrictions, and allow any kind of relationship type.
>
> Best regards,
> Ifat.
>
> From: Yujun Zhang
> Date: Monday, 22 August 2016 at 08:37
>
> I'm following the sample configuration in docs [1] to verify how static
> datasources works.
>
> It seems `backup` relationship is not displayed in the entity graph view
> and neither is it included in topology show.
>
> There is an enumeration for edge labels [2]. Should relationship in static
> datasource be limited to it?
>
> [1]
> https://github.com/openstack/vitrage/blob/master/doc/source/static-physical-config.rst
> [2]
> https://github.com/openstack/vitrage/blob/master/vitrage/common/constants.py#L49
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] entity graph layout

2016-08-22 Thread Yujun Zhang

I'm considering to use Cytoscape.js [1] to improve the layout for entity
graph view.

Cytoscape.js is a graph theory (a.k.a. network) library for analysis and
visualisation which under active maintenance (latest release 2.7.8 on Aug
18, 2016) [2], while the current library d3-dagre [3] is declared not being
actively developed or maintained.

Meanwhile, I'm building a proof of concept for visualizing the entity graph
with Cytoscape.

Could anybody give a list on the required features for this view? Any
comments are welcome.

[1] http://js.cytoscape.org/
[2] https://github.com/cytoscape/cytoscape.js
[3] https://github.com/cpettitt/dagre-d3


On Mon, Aug 8, 2016 at 2:34 PM Afek, Ifat (Nokia - IL) <ifat.a...@nokia.com>
wrote:

> There is no such blueprint at the moment.
> You are more than welcome to add one, in case you have some ideas for
> improvements.
>
> Ifat.
>
> From: Yujun Zhang
> Date: Monday, 8 August 2016 at 09:21
>
>
> Great, it works.
> But it would be better if we could improve the default layout. Is there
> any blueprint in progress?
> --
> Yujun
>
> On Sun, Aug 7, 2016 at 1:09 PM Afek, Ifat (Nokia - IL) <
> ifat.a...@nokia.com> wrote:
>
>> Hi,
>>
>> It is possible to adjust the layout of the graph. You can double-click on
>> a vertex and it will remain pinned to its place. You can then move the
>> pinned vertices around to adjust the graph layout.
>>
>> Hope it helped, and let us know if you need additional help with your
>> demo.
>>
>> Best Regards,
>> Ifat.
>>
>>
>> From: Yujun Zhang
>> Date: Friday, 5 August 2016 at 09:32
>>
>> Hi, all,
>>
>> I'm building a demo of vitrage. The dynamic entity graph looks
>> interesting.
>>
>> But when more entities are added, things becomes crowded and the links
>> screw over each other. Dragging the items will not help much.
>>
>> Is it possible to adjust the layout so I can get a more regular/stable
>> tree view of the entities?
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [vitrage] relationship_type in static_datasources

2016-08-21 Thread Yujun Zhang

I'm following the sample configuration in docs [1] to verify how static
datasources works.

It seems `backup` relationship is not displayed in the entity graph view
and neither is it included in topology show.

There is an enumeration for edge labels [2]. Should relationship in static
datasource be limited to it?

[1]
https://github.com/openstack/vitrage/blob/master/doc/source/static-physical-config.rst
[2]
https://github.com/openstack/vitrage/blob/master/vitrage/common/constants.py#L49
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [vitrage] gate-vitrage-dsvm-api FAILURE

2016-08-18 Thread Yujun Zhang

I submit a simple patch for additional log message but the CI job keeps
failure even after recheck.

It seems some configuration files are missing according to the console log
[2].

Does anybody encountering the same issue as me?

```
2016-08-18 19:59:15.155472

| 2016-08-18 19:59:15.155 | ++
/opt/stack/new/vitrage/devstack/post_test_hook.sh:source:L31: sudo
oslo-config-generator --config-file etc/config-generator.tempest.conf
--output-file etc/tempest.conf
2016-08-18 19:59:15.402397

| 2016-08-18 19:59:15.401 | Traceback (most recent call last):2016-08-18
19:59:15.403804

| 2016-08-18 19:59:15.403 | File "/usr/local/bin/oslo-config-generator",
line 11, in 
2016-08-18 19:59:15.405044

| 2016-08-18 19:59:15.404 | sys.exit(main())
2016-08-18 19:59:15.406367

| 2016-08-18 19:59:15.406 | File
"/usr/local/lib/python2.7/dist-packages/oslo_config/generator.py", line
478, in main
2016-08-18 19:59:15.407918

| 2016-08-18 19:59:15.407 | conf(args, version=version)
2016-08-18 19:59:15.409207

| 2016-08-18 19:59:15.408 | File
"/usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py", line 2256, in
__call__
2016-08-18 19:59:15.410630

| 2016-08-18 19:59:15.410 | raise
ConfigFilesNotFoundError(self._namespace._files_not_found)
*2016-08-18 19:59:15.412069

| 2016-08-18 19:59:15.411 | oslo_config.cfg.ConfigFilesNotFoundError:
Failed to find some config files:
/opt/stack/new/tempest/etc/config-generator.tempest.conf*
```


[1] https://review.openstack.org/#/c/357001/
[2]
http://logs.openstack.org/01/357001/1/check/gate-vitrage-dsvm-api/2e679a2/console.html
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage][vitrage-dashboard] cropped entity information

2016-08-18 Thread Yujun Zhang

Reported at https://bugs.launchpad.net/vitrage-dashboard/+bug/1614585
--
Yujun

On Thu, Aug 18, 2016 at 9:44 PM Afek, Ifat (Nokia - IL) <ifat.a...@nokia.com>
wrote:

> Hi Yujun,
>
> This is something that we need to handle. You can open a bug about it.
>
> Best Regards,
> Ifat.
>
> From: Yujun Zhang
> Reply-To: "OpenStack Development Mailing List (not for usage questions)"
> Date: Wednesday, 17 August 2016 at 07:08
> To: "OpenStack Development Mailing List (not for usage questions)"
> Subject: [openstack-dev] [vitrage][vitrage-dashboard] cropped entity
> information
>
> In the topology view, the content of entity information is sometimes
> cropped when there are too many rows.
>
> Is there a way to peek the cropped content?
>
> [image: xbm5sl.jpg]
> --
> Yujun
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] valid source and target for add_causal_relationship

2016-08-18 Thread Yujun Zhang

Fix proposed in https://review.openstack.org/#/c/356974/

To enable validation of definition content, I have also made a refactoring
to the validator as in https://review.openstack.org/#/c/356947/

--
Yujun

On Wed, Aug 17, 2016 at 10:29 PM Har-Tal, Liat (Nokia - IL) <
liat.har-...@nokia.com> wrote:

> Yes, it should be limited to ‘ALARM’
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Wednesday, August 17, 2016 9:00 AM
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* [openstack-dev] [vitrage] valid source and target for
> add_causal_relationship
>
>
>
> The issue comes from my carelessness that creating a causal relationship
> between two relationship instead of entities [1].
>
>
>
> But it seems not be detected by the template validator.
>
>
>
> I wonder what could be a valid `source` and `target` for causal
> relationship? Should it be limited to `ALARM`?
>
>
>
> [1]
> https://github.com/openzero-zte/vitrage-demo/commit/ce82f6f03e1b7168499233de431323e3cba43f9d#diff-aef3ec3ecbcccbad905bf3c57bb47e95R49
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] reference a type of alarm in template

2016-08-17 Thread Yujun Zhang

Thanks a lot Liat.

So if `name` is omitted, all alarms from that source will match the entity.
Otherwise the `name` is checked. The property mapped to `name` is defined
in datasource driver and transformer. Is that so?

P.S. I know that the ultimate answers can be digged outs from code and
document. It just sometimes consumes too much time.

Thanks again for the clarifications from the team. Your answers really
helped me a lot in the understanding of vitrage design.
--
Yujun

On Wed, Aug 17, 2016 at 10:03 PM Har-Tal, Liat (Nokia - IL) <
liat.har-...@nokia.com> wrote:

> Hi Yujun,
>
>
>
> There is no limitation for the number of alarms from the same data source
> you can define in the template.
>
>
>
> See the following examples:
>
>
>
> *Example 1*: two different alarms definitions from the same data source
>
>
>
> - entity:
>
> category: ALARM
>
> type: nagios
>
> name: CPU load
>
> template_id: nagios_alarm_1
>
> - entity:
>
> category: ALARM
>
> type: nagios
>
> name: Memory use
>
> template_id: nagios_alarm_2
>
>
>
> *Example 2*: definition for a general alarm from specific data source
>
>
>
> - entity:
>
>     category: ALARM
>
> type: nagios
>
> template_id: nagios_alarm
>
>
>
> Best regards,
>
> Liat
>
>
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Wednesday, August 17, 2016 3:51 AM
>
>
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* Re: [openstack-dev] [vitrage] reference a type of alarm in
> template
>
>
>
> Hi, Liat,
>
>
>
> Thanks for clarification. But my question still remains.
>
>
>
> Under this definition (type <=> datasource), we can only refer to ***one**
> *alarm from each datasource in the template. Is it a limitation?
>
>
>
> How should I refer two different aodh alarms in the template?
>
> --
>
> Yujun
>
>
>
> On Tue, Aug 16, 2016 at 10:24 PM Har-Tal, Liat (Nokia - IL) <
> liat.har-...@nokia.com> wrote:
>
> Hi Yujun,
>
>
>
> The template example you are looking at is invalid.
>
> I added a new valid example (see the following link:
> https://review.openstack.org/#/c/355840/)
>
>
>
> As Elisha wrote, the ‘type’ field means the alarm type itself or in simple
> words where it was generated (Vitrage/Nagios/Zabbix/AODH)
>
> The ‘name’ field is not mandatory and it describes the actual problem
> which the alarm was raised about.
>
>
>
> In the example you can see two alarm types:
>
>
>
> 1.   Zabbix alarm - No use of “name” field:
>
>  category: ALARM
>
>  type: zabbix
>
>  rawtext: Processor load is too high on {HOST.NAME}
>
>  template_id: zabbix_alarm
>
> 2.   Vitrage alarm
>
> category: ALARM
>
> type: vitrage
>
> name: CPU performance degradation
>
> template_id: instance_alarm
>
>
>
> One more point, in order to define an entity in the template, the only
> mandatory fields are:
>
> · template_id
>
> · category
>
> All the other fields are optional and they are designed so that you more
> accurately define the entity.
>
> Each alarm data source has its own set of fields you can use – we will add
> documentation for the in the future.
>
>
>
> Best regards,
>
> Liat Har-Tal
>
>
>
>
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Tuesday, August 16, 2016 5:18 AM
>
>
> *To:* OpenStack Development Mailing List (not for usage questions)
>
> *Subject:* Re: [openstack-dev] [vitrage] reference a type of alarm in
> template
>
>
>
> Hi, Elisha
>
>
>
> There is no `name` in the template [1], and type is not one of 'nagios',
> 'aodh' and 'vitrage' in the examples [2].
>
>
>
> - entity:
>
> category: ALARM
>
> type: Free disk space is less than 20% on volume /
>
> template_id: host_alarm
>
>
>
> [1]
> https://github.com/openstack/vitrage/blob/master/etc/vitrage/templates.sample/deduced_host_disk_space_to_instance_at_risk.yaml#L8
>
> [2]
> https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-template-format.rst#examples
>
>
>
>
>
> On Tue, Aug 16, 2016 at 2:21 AM Rosensweig, Elisha (Nokia - IL) <
> elisha.rosensw...@nokia.com> wrote:
>
> Hi,
>
>
>
> The "ty

[openstack-dev] [vitrage] valid source and target for add_causal_relationship

2016-08-17 Thread Yujun Zhang

The issue comes from my carelessness that creating a causal relationship
between two relationship instead of entities [1].

But it seems not be detected by the template validator.

I wonder what could be a valid `source` and `target` for causal
relationship? Should it be limited to `ALARM`?

[1]
https://github.com/openzero-zte/vitrage-demo/commit/ce82f6f03e1b7168499233de431323e3cba43f9d#diff-aef3ec3ecbcccbad905bf3c57bb47e95R49
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [vitrage][vitrage-dashboard] cropped entity information

2016-08-16 Thread Yujun Zhang

In the topology view, the content of entity information is sometimes
cropped when there are too many rows.

Is there a way to peek the cropped content?

[image: xbm5sl.jpg]
--
Yujun
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [javascript] library api reference publishing

2016-08-16 Thread Yujun Zhang

As an action from the javascript meeting, below is the summary of how to
integrate jsdoc [1] generated api reference with sphinx document.

Feel free to comment and discuss in today's meeting.


   - findings


   - openstack api ref is dedicated to describe RESTful interface
   http://developer.openstack.org/api-ref.html


   - no example of library api reference found


   - Proposal


   - hosting api reference on independent site and add external link in
   openstack document site


   - api reference: http://js-generator-openstack.openzero.net/


   - infra document: http://docs.openstack.org/infra/


   - TODO


   - hosting of api reference (static HTML pages)


   - github pages?


   - publish api reference with CI


[1] http://usejsdoc.org/
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] reference a type of alarm in template

2016-08-16 Thread Yujun Zhang

Hi, Liat,

Thanks for clarification. But my question still remains.

Under this definition (type <=> datasource), we can only refer to
***one** *alarm
from each datasource in the template. Is it a limitation?

How should I refer two different aodh alarms in the template?
--
Yujun

On Tue, Aug 16, 2016 at 10:24 PM Har-Tal, Liat (Nokia - IL) <
liat.har-...@nokia.com> wrote:

> Hi Yujun,
>
>
>
> The template example you are looking at is invalid.
>
> I added a new valid example (see the following link:
> https://review.openstack.org/#/c/355840/)
>
>
>
> As Elisha wrote, the ‘type’ field means the alarm type itself or in simple
> words where it was generated (Vitrage/Nagios/Zabbix/AODH)
>
> The ‘name’ field is not mandatory and it describes the actual problem
> which the alarm was raised about.
>
>
>
> In the example you can see two alarm types:
>
>
>
> 1.   Zabbix alarm - No use of “name” field:
>
>  category: ALARM
>
>  type: zabbix
>
>  rawtext: Processor load is too high on {HOST.NAME}
>
>  template_id: zabbix_alarm
>
> 2.   Vitrage alarm
>
> category: ALARM
>
> type: vitrage
>
> name: CPU performance degradation
>
> template_id: instance_alarm
>
>
>
> One more point, in order to define an entity in the template, the only
> mandatory fields are:
>
> · template_id
>
> · category
>
> All the other fields are optional and they are designed so that you more
> accurately define the entity.
>
> Each alarm data source has its own set of fields you can use – we will add
> documentation for the in the future.
>
>
>
> Best regards,
>
> Liat Har-Tal
>
>
>
>
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Tuesday, August 16, 2016 5:18 AM
>
>
> *To:* OpenStack Development Mailing List (not for usage questions)
>
> *Subject:* Re: [openstack-dev] [vitrage] reference a type of alarm in
> template
>
>
>
> Hi, Elisha
>
>
>
> There is no `name` in the template [1], and type is not one of 'nagios',
> 'aodh' and 'vitrage' in the examples [2].
>
>
>
> - entity:
>
> category: ALARM
>
> type: Free disk space is less than 20% on volume /
>
> template_id: host_alarm
>
>
>
> [1]
> https://github.com/openstack/vitrage/blob/master/etc/vitrage/templates.sample/deduced_host_disk_space_to_instance_at_risk.yaml#L8
>
> [2]
> https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-template-format.rst#examples
>
>
>
>
>
> On Tue, Aug 16, 2016 at 2:21 AM Rosensweig, Elisha (Nokia - IL) <
> elisha.rosensw...@nokia.com> wrote:
>
> Hi,
>
>
>
> The "type" means where it was generated - aodh, vitrage, nagios...
>
>
>
> I think you are looking for"name", a field that describes the actual
> problem. We should add that to our documentation to clarify.
>
>
>
> Sent from Nine <http://www.9folders.com/>
> --
>
> *From:* Yujun Zhang <zhangyujun+...@gmail.com>
> *Sent:* Aug 15, 2016 16:10
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* [openstack-dev] [vitrage] reference a type of alarm in template
>
>
>
> I have a question on how to reference a type of alarm in template so that
> we can build scenarios.
>
>
>
> In the template sample [1], an alarm entity has three keys: `category`,
> `type` and `template_id`. It seems `type` is the only information to
> distinguish different alarms. However, when an alarm is raised by aodh, it
> seems all alarms are assigned entity type `aodh` [2], so are they shown in
> dashboard.
>
>
>
> Suppose we have two different types of alarms from `aodh`, e.g.
> `volume.corrupt` and `volume.deleted`. How should I reference them
> separately in a template?
>
>
>
> ×
>
> [image: Screen Shot 2016-08-15 at 8.44.56 PM.png]
>
> ×
>
>
>
> ×
>
>
>
> [1]
> https://github.com/openstack/vitrage/blob/master/etc/vitrage/templates.sample/deduced_host_disk_space_to_instance_at_risk.yaml#L8
>
> [2]
> https://github.com/openstack/vitrage/blob/master/vitrage/datasources/aodh/transformer.py#L75
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] scenario evaluator not enabled by default

2016-08-16 Thread Yujun Zhang

Found it at
https://github.com/openstack/vitrage/blob/master/vitrage/entity_graph/consistency/consistency_enforcer.py#L58

Thanks, Nofar.

On Tue, Aug 16, 2016 at 3:09 PM Schnider, Nofar (EXT - IL) <
nofar.schnider@nokia.com> wrote:

> Hi Yujun,
>
> I think you will find what you are looking for in
> “consistency_enforcer.py”.
>
> Hope it helps.
>
>
>
>
>
> *Nofar Schnider*
>
> Senior Software Engineer
>
> Applications & Analytics, Nokia
>
> 16 Atir Yeda
>
> Kfar Saba, Israel 44643
>
> M: +972 52 4326023
>
> F: +972 9 793 3036
>
> [image: download]
>
>
>
> *From:* Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> *Sent:* Monday, August 15, 2016 8:49 AM
> *To:* OpenStack Development Mailing List (not for usage questions) <
> openstack-dev@lists.openstack.org>; Rosensweig, Elisha (Nokia - IL) <
> elisha.rosensw...@nokia.com>
>
>
> *Subject:* Re: [openstack-dev] [vitrage] scenario evaluator not enabled
> by default
>
>
>
> Thanks for the expanation, Elisha. I understand the design now.
>
>
>
> But I could not find the statement which enables the evaluator after
> initial phase.
>
>
>
> Could you help to point it out?
>
> --
>
> Yujun
>
>
>
> On Thu, Aug 11, 2016 at 11:42 PM Rosensweig, Elisha (Nokia - IL) <
> elisha.rosensw...@nokia.com> wrote:
>
> This is on purpose.
>
> When Vitrage is started, it first runs a "consistency" round where it gets
> all the resources from its datasources and inserts them into the entity
> graph. Once this initial phase is over, the evaluator is run over all the
> entity graph to check for meaningful patterns based on it's templates.
>
> The reason for this process is to avoid too much churn during the initial
> phase when Vitrage comes up. With so many changes done to the entity graph,
> it's best to wait for the initial collection phase to finish and then to do
> the analysis.
>
> Elisha
>
> > From: Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> > Sent: Thursday, August 11, 2016 5:49 PM
> > To: OpenStack Development Mailing List (not for usage questions)
> > Subject: Re: [openstack-dev] [vitrage] scenario evaluator not enabled by
> default
> >
> > Sorry for having put a url from forked repo. It should be
> https://github.com/openstack/vitrage/commit/bdba10cb71b2fa3744e4178494fa860303ae0bbe#diff>
> -6f1a277a2f6e9a567b38d646f19728bcL36
>
> > But the content is the same
> > --
> > Yujun
>
> > On Thu, Aug 11, 2016 at 10:43 PM Yujun Zhang <zhangyujun+...@gmail.com>
> wrote:
> > It seems the scenario evaluator is not enabled when vitrage is started
> in devstack installer.
> >
> > I dig a bit in the history, it seems the default value for the evaluator
> is changed from True to False > in a history commit [1].
> >
> > Is it breaking the starting of evaluator or I have missed some steps to
> enable it explictily?
> >
> > - [1]
> https://github.com/openzero-zte/vitrage/commit/bdba10cb71b2fa3744e4178494fa860303ae0bbe#diff-
> 6f1a277a2f6e9a567b38d646f19728bcL36
>
> > --
> > Yujun
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [vitrage][vitrage-dashboard] missing icon/font in alarm list under material theme

2016-08-16 Thread Yujun Zhang

The alarm list is not displaying some icon/font correctly under Material
theme. See attached screenshots.

Could anybody have a look?

Tracked in https://bugs.launchpad.net/vitrage-dashboard/+bug/1613574

Material:

[image: Screen Shot 2016-08-16 at 3.14.26 PM.png]
×
[image: Screen Shot 2016-08-16 at 3.14.34 PM.png]
×


Default:

[image: Screen Shot 2016-08-16 at 3.13.58 PM.png][image: Screen Shot
2016-08-16 at 3.14.13 PM.png]
×

× *[image: Screen Shot 2016-08-16 at 3.14.26 PM.png]*
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [vitrage] reference a type of alarm in template

2016-08-15 Thread Yujun Zhang

I have a question on how to reference a type of alarm in template so that
we can build scenarios.

In the template sample [1], an alarm entity has three keys: `category`,
`type` and `template_id`. It seems `type` is the only information to
distinguish different alarms. However, when an alarm is raised by aodh, it
seems all alarms are assigned entity type `aodh` [2], so are they shown in
dashboard.

Suppose we have two different types of alarms from `aodh`, e.g.
`volume.corrupt` and `volume.deleted`. How should I reference them
separately in a template?

×
[image: Screen Shot 2016-08-15 at 8.44.56 PM.png]
×

×

[1]
https://github.com/openstack/vitrage/blob/master/etc/vitrage/templates.sample/deduced_host_disk_space_to_instance_at_risk.yaml#L8
[2]
https://github.com/openstack/vitrage/blob/master/vitrage/datasources/aodh/transformer.py#L75
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] scenario evaluator not enabled by default

2016-08-14 Thread Yujun Zhang

Thanks for the expanation, Elisha. I understand the design now.

But I could not find the statement which enables the evaluator after
initial phase.

Could you help to point it out?
--
Yujun

On Thu, Aug 11, 2016 at 11:42 PM Rosensweig, Elisha (Nokia - IL) <
elisha.rosensw...@nokia.com> wrote:

> This is on purpose.
>
> When Vitrage is started, it first runs a "consistency" round where it gets
> all the resources from its datasources and inserts them into the entity
> graph. Once this initial phase is over, the evaluator is run over all the
> entity graph to check for meaningful patterns based on it's templates.
>
> The reason for this process is to avoid too much churn during the initial
> phase when Vitrage comes up. With so many changes done to the entity graph,
> it's best to wait for the initial collection phase to finish and then to do
> the analysis.
>
> Elisha
>
> > From: Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> > Sent: Thursday, August 11, 2016 5:49 PM
> > To: OpenStack Development Mailing List (not for usage questions)
> > Subject: Re: [openstack-dev] [vitrage] scenario evaluator not enabled by
> default
> >
> > Sorry for having put a url from forked repo. It should be
> https://github.com/openstack/vitrage/commit/bdba10cb71b2fa3744e4178494fa860303ae0bbe#diff>
> -6f1a277a2f6e9a567b38d646f19728bcL36
>
> > But the content is the same
> > --
> > Yujun
>
> > On Thu, Aug 11, 2016 at 10:43 PM Yujun Zhang <zhangyujun+...@gmail.com>
> wrote:
> > It seems the scenario evaluator is not enabled when vitrage is started
> in devstack installer.
> >
> > I dig a bit in the history, it seems the default value for the evaluator
> is changed from True to False > in a history commit [1].
> >
> > Is it breaking the starting of evaluator or I have missed some steps to
> enable it explictily?
> >
> > - [1]
> https://github.com/openzero-zte/vitrage/commit/bdba10cb71b2fa3744e4178494fa860303ae0bbe#diff-
> 6f1a277a2f6e9a567b38d646f19728bcL36
>
> > --
> > Yujun
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] scenario evaluator not enabled by default

2016-08-11 Thread Yujun Zhang

Sorry for having put a url from forked repo. It should be
https://github.com/openstack/vitrage/commit/bdba10cb71b2fa3744e4178494fa860303ae0bbe#diff-6f1a277a2f6e9a567b38d646f19728bcL36

But the content is the same
--
Yujun

On Thu, Aug 11, 2016 at 10:43 PM Yujun Zhang <zhangyujun+...@gmail.com>
wrote:

> It seems the scenario evaluator is not enabled when vitrage is started in
> devstack installer.
>
> I dig a bit in the history, it seems the default value for the evaluator
> is changed from True to False in a history commit [1].
>
> Is it breaking the starting of evaluator or I have missed some steps to
> enable it explictily?
>
> - [1]
> https://github.com/openzero-zte/vitrage/commit/bdba10cb71b2fa3744e4178494fa860303ae0bbe#diff-6f1a277a2f6e9a567b38d646f19728bcL36
>
> --
> Yujun
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [vitrage] scenario evaluator not enabled by default

2016-08-11 Thread Yujun Zhang

It seems the scenario evaluator is not enabled when vitrage is started in
devstack installer.

I dig a bit in the history, it seems the default value for the evaluator is
changed from True to False in a history commit [1].

Is it breaking the starting of evaluator or I have missed some steps to
enable it explictily?

- [1]
https://github.com/openzero-zte/vitrage/commit/bdba10cb71b2fa3744e4178494fa860303ae0bbe#diff-6f1a277a2f6e9a567b38d646f19728bcL36

--
Yujun
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] aodh datasource properties

2016-08-11 Thread Yujun Zhang

OK, I have assigned myself for this issue and hope to resolve it soon.
--
Yujun

On Thu, Aug 11, 2016 at 6:30 PM Afek, Ifat (Nokia - IL) <ifat.a...@nokia.com>
wrote:

> Hi Yujun,
>
> There are few ways to create Aodh alarms. You are referring to
> event-alarms, and I believe you are right about the bug.
>
> Note, however, that for threshold-alarm, the resource_id appears without
> the “traits” prefix, so Vitrage Aodh datasource should be able to handle
> both cases. An example for creating a threshold alarm:
> ceilometer alarm-threshold-create --name 'cpu_alarm' --description 'CPU
> utilization is above 70%' -m 'cpu_util' --period 60 --threshold 0.7
>  --comparison-operator gt --query
> 'resource_id=5f6db701-19d6-4a98-895b-8094f2bd7304'
>
> Hope it helped,
> Ifat.
>
> From: Yujun Zhang
> Date: Thursday, 11 August 2016 at 12:36
>
>
> It seems the aodh properties definition in vitrage [1] is not consistent
> with the latest ceilometer spec [2].
>
> The response_id is now encapsulated in `traits` and we must prepend the
> scope in query to make it a valid alarm condition, e.g. `query: [{u'field':
> u'traits.resource_id', u'type': u'string', u'value': u'...'`
>
> But in vitrage, the resource id is parsed from key 'resource_id' [3] and
> always get an empty result.
>
> Could anybody confirm whether this is a bug or not? Tracked in
> https://bugs.launchpad.net/vitrage/+bug/1612152
>
> - [1]
> https://github.com/openstack/vitrage/blob/master/vitrage/datasources/aodh/properties.py#L27
> - [2]
>
> https://github.com/openstack/ceilometer/blob/master/etc/ceilometer/event_definitions.yaml#L76
> - [3]
> https://github.com/openstack/vitrage/blob/master/vitrage/datasources/aodh/driver.py#L66
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [vitrage] aodh datasource properties

2016-08-11 Thread Yujun Zhang

It seems the aodh properties definition in vitrage [1] is not consistent
with the latest ceilometer spec [2].

The response_id is now encapsulated in `traits` and we must prepend the
scope in query to make it a valid alarm condition, e.g. `query: [{u'field':
u'traits.resource_id', u'type': u'string', u'value': u'...'`

But in vitrage, the resource id is parsed from key 'resource_id' [3] and
always get an empty result.

Could anybody confirm whether this is a bug or not? Tracked in
https://bugs.launchpad.net/vitrage/+bug/1612152

- [1]
https://github.com/openstack/vitrage/blob/master/vitrage/datasources/aodh/properties.py#L27
- [2]
https://github.com/openstack/ceilometer/blob/master/etc/ceilometer/event_definitions.yaml#L76
- [3]
https://github.com/openstack/vitrage/blob/master/vitrage/datasources/aodh/driver.py#L66
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] spec for datasource

2016-08-09 Thread Yujun Zhang

Hi lfat,

Thank you for the answers, please see my reply inline.

On Tue, Aug 9, 2016 at 6:51 PM Afek, Ifat (Nokia - IL) <ifat.a...@nokia.com>
wrote:

> Hi Yujun,
>
> Please see my answers below.
>
> Best Regards,
> Ifat.
>
> From: Yujun Zhang
> Date: Tuesday, 9 August 2016 at 12:06
>
>
> For proprietary datasource, I'm considering to adapt the api to common
> protocol/interface, e.g. RESTful, SNMP and etc and I wish to know how to
> add support for these interface.
>
> [Ifat]: Do you mean that you want to write a datasource that gets its
> information from SNMP/REST API? Note that for alarm datasources, we have in
> our roadmap to support OPNFV Doctor SB REST API[1]. Will it be relevant for
> your use cases?
>
[yujunz] Great, this could also be a good start point for us

> Some documents are found in https://github.com/openstack/vitrage-specs and
>> datasource seems to be related to synchronizer but I didn't find a
>> dedicated spec.
>>
> [Ifat]: We started documenting the process of adding a new datasource, but
> the document is not final. We will try to finish it shortly. BTW, there are
> many other documents in Vitrage wiki page[2].
>
[yujunz] Thanks.

> 1. How do I register a new datasource in an existing system?
>>
> [yujunz] It seems to be in
https://github.com/openstack/vitrage/blob/master/vitrage/datasources/__init__.py


> 2. Is the type of datasource (ALARM/RESOURCE) configured in
>> `/etc/vitrage/datasource_values/.yaml` ?
>>
> [Ifat]: No, it is configured in the datasource code. For information about
> datasource_values please see [3]
>
[yujunz] which code file? I found `category: RESOURCE` in the datasource
configuration file as
https://github.com/openstack/vitrage/blob/master/doc/source/resource-state-config.rst#format

> 3. Is there any other datasource type besides ALARM/RESOURCE?
>
> [Ifat]: No, and at the moment we don’t see a need for that. Vitrage can
> hold resources of any type in a topology graph, and manage alarms that are
> raised on these resources. If you see a use case for other datasource
> types, let us know.
>
[yujunz]  I agree. Just ask to confirm my guess :-)

> 4. What does `aggregated values` and `priority` mean
>>
> [Ifat]: Detailed in [3]
>
[yujunz]: Clear now.

> 5. What is the required data format for the datasource driver api? The
>> inline comments give some brief description but didn't specify the api
>> signature.
>> Many thanks.
>>
> [Ifat]: This should be part of the datasource documentation that we need
> to add. But basically, the driver should return a dictionary with few
> predefined fields (like datasource type, timestamp), and add whatever data
> is needed for the datasource. Then, the datasource transformer will
> transform this data into a vertex and decide where to connect it to the
> graph.
>
[yujunz] So if I understand it correctly, driver and transformer works as a
pair and the intermediate data format is not exposed to vitrage. It
consumes data from the sources and convert them into graph

> [1] https://gerrit.opnfv.org/gerrit/12179
> [2] https://wiki.openstack.org/wiki/Vitrage
> *[3] *
> https://github.com/openstack/vitrage/blob/master/doc/source/resource-state-config.rst
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [vitrage] questions and answers

2016-08-09 Thread Yujun Zhang

Dear all,

Below is a digest of conversation between me and vitrage PTL lfat. It
clarifies some of my questions and I hope it will also be useful for
developers who are watching vitrage project.

relationship between entities Examples of "on" and "contain" given in
document. But not sure how to extend it, e.g. adding a new relationship
such as "equivalent"

[Ifat] Right now we have template validation code, which checks that
the relationship type is one of a few predefined types. I don’t think this
is the right behaviour. It is very reasonable for a datasource to add its
own relationship types. I think we should modify the code, and maybe have
a configuration file(s) with all possible relationship types. This way, we
can still do the validation but it will be easy to support new relationship
types. You are welcome to add a blueprint for that.

rule live update

Is it possible to modify the template on line without interrupting the
service?

[Ifat] Not yet, but this is in our roadmap, see
https://blueprints.launchpad.net/vitrage/+spec/crud-templates-api

condition format

Example of "not" is given in

https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-template-format.rst#condition-format
but
negation is marked to be done in

https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-template-format.rst#negation
.
So what is the current situation? Is multiple ( more than three )
conditions combination supported?

[Ifat] The negation is not marked as done. It says: “We *need to* support
‘not' operator”. Again, this is in our roadmap. Regarding multiple
conditions, we don’t limit their number.

performance

How many rules can be evaluate per second? Is there a benchmark?

[Ifat] We did not perform such a benchmark yet.

supported data source

Examples of nagios and nova are given. Is there any limitation on the
required protocol? e.g. retrieving data from http, snmp, odbc.

[Ifat] You can add whatever datasource that you want, and get data in any
format. It is the datasource responsibility to convert this data into
Vitrage format, so it can be added to the graph.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] spec for datasource

2016-08-09 Thread Yujun Zhang

Hi, Eylon,

It is not decided yet what datasources will be required. But we may assume
there will be both open and proprietary ones.

The example of the former could be include service status of nova host in
the topology (currently only host, instance and zone are found in the code).

For proprietary datasource, I'm considering to adapt the api to common
protocol/interface, e.g. RESTful, SNMP and etc and I wish to know how to
add support for these interface.

At the moment, I'm making an evaluation on the extensibility of vitrage
architecture and estimate the workload for new datasource. Any idea how I
can proceed?

--
Yujun

On Tue, Aug 9, 2016 at 4:34 PM Malin, Eylon (Nokia - IL) <
eylon.ma...@nokia.com> wrote:

> Hi,
>
> There are different instruction for datasource that are part of openstack
> vitrage upstream, and for propriety datasource.
> So for better understanding the case, do you want to add new datasource
> that would be contributed to openstack, or is it propriety one ?
> I'm meaning do you plan to push the new datasource to vitrage upstream or
> leave it private ?
>
> Eylon
>
>
> From: Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> Sent: Tuesday, August 09, 2016 10:22 AM
> To: OpenStack Development Mailing List (not for usage questions) <
> openstack-dev@lists.openstack.org>
> Subject: [openstack-dev] [vitrage] spec for datasource
>
> Dear all,
>
> Is there a guide on how to understand the design of datasource? I want to
> extend the existing one and also try to create a custom datasource from
> scratch.
>
> Some documents are found in https://github.com/openstack/vitrage-specs and
> datasource seems to be related to synchronizer but I didn't find a
> dedicated spec.
>
> Currently I have the following questions
> 1. How do I register a new datasource in an existing system?
> 2. Is the type of datasource (ALARM/RESOURCE) configured in
> `/etc/vitrage/datasource_values/.yaml` ?
> 3. Is there any other datasource type besides ALARM/RESOURCE?
> 4. What does `aggregated values` and `priority` mean
> 5. What is the required data format for the datasource driver api? The
> inline comments give some brief description but didn't specify the api
> signature.
> Many thanks.
>
> --
> Yujun
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [vitrage] spec for datasource

2016-08-09 Thread Yujun Zhang

Dear all,

Is there a guide on how to understand the design of datasource? I want to
extend the existing one and also try to create a custom datasource from
scratch.

Some documents are found in https://github.com/openstack/vitrage-specs and
datasource seems to be related to synchronizer but I didn't find a
dedicated spec.

Currently I have the following questions

   1. How do I register a new datasource in an existing system?
   2. Is the type of datasource (ALARM/RESOURCE) configured in
   `/etc/vitrage/datasource_values/.yaml` ?
   3. Is there any other datasource type besides ALARM/RESOURCE?
   4. What does `aggregated values` and `priority` mean
   5. What is the required data format for the datasource driver api? The
   inline comments give some brief description but didn't specify the api
   signature.

Many thanks.

--
Yujun
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [vitrage] Nova host list api - performance

2016-08-08 Thread Yujun Zhang

Is the data transfer compressed?

If there are lots of repeated pattern in the payload, compressing the
content may result in great improvement in performance.
--
Yujun

On Mon, Aug 8, 2016 at 3:19 PM Weyl, Alexey (Nokia - IL) <
alexey.w...@nokia.com> wrote:

> Hi,
>
> I am running the "client.hosts.list()" command in Vitrage to get all the
> hosts in the system (we also run similar commands for availability zones
> and instances). Vitrage does this to gain a holistic view of system
> resources[1].
>
> As part of our performance checks we noticed that when calling
> client.hosts.list(), the info on each host includes the "HostManager"
> object, which weighs about 840KB on the oslo messaging bus, which is quite
> heavy.
>
> We would like to reduce that amount of data transferred, to improve
> performance. Is it possible to pass some parameter in "client.hosts.list()"
> so the manager property won't be sent on the bus?
>
> Thanks in advance,
> Alexey
>
> [1] https://wiki.openstack.org/wiki/Vitrage
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] entity graph layout

2016-08-08 Thread Yujun Zhang

Great, it works.
But it would be better if we could improve the default layout. Is there any
blueprint in progress?
--
Yujun

On Sun, Aug 7, 2016 at 1:09 PM Afek, Ifat (Nokia - IL) <ifat.a...@nokia.com>
wrote:

> Hi,
>
> It is possible to adjust the layout of the graph. You can double-click on
> a vertex and it will remain pinned to its place. You can then move the
> pinned vertices around to adjust the graph layout.
>
> Hope it helped, and let us know if you need additional help with your demo.
>
> Best Regards,
> Ifat.
>
>
> From: Yujun Zhang
> Date: Friday, 5 August 2016 at 09:32
>
> Hi, all,
>
> I'm building a demo of vitrage. The dynamic entity graph looks
> interesting.
>
> But when more entities are added, things becomes crowded and the links
> screw over each other. Dragging the items will not help much.
>
> Is it possible to adjust the layout so I can get a more regular/stable
> tree view of the entities?
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

1 2 >

1 - 100 of 106 matches

Mail list logo