subject:"\[openstack\-dev\] \[vitrage\] spec for datasource"

Re: [openstack-dev] [vitrage] spec for datasource

2016-08-10 Thread Afek, Ifat (Nokia - IL)

From: Yujun Zhang
Date: Tuesday, 9 August 2016 at 14:48

1. How do I register a new datasource in an existing system?
[yujunz] It seems to be in 
https://github.com/openstack/vitrage/blob/master/vitrage/datasources/__init__.py
Not exactly. This file defines the default datasources to be used. You can 
overwrite this list in /etc/vitrage/vitrage.conf. For example:

[datasources]
types = 
nova.host,nova.instance,nova.zone,cinder.volume,neutron.network,neutron.port

2. Is the type of datasource (ALARM/RESOURCE) configured in 
`/etc/vitrage/datasource_values/.yaml` ?

[Ifat]: No, it is configured in the datasource code. For information about 
datasource_values please see [3]
[yujunz] which code file? I found `category: RESOURCE` in the datasource 
configuration file as 
https://github.com/openstack/vitrage/blob/master/doc/source/resource-state-config.rst#format
EntityCategory is defined in 
https://github.com/openstack/vitrage/blob/master/vitrage/common/constants.py
It is used in the transformer files of each datasource.

5. What is the required data format for the datasource driver api? The inline 
comments give some brief description but didn't specify the api signature.
Many thanks.
[Ifat]: This should be part of the datasource documentation that we need to 
add. But basically, the driver should return a dictionary with few predefined 
fields (like datasource type, timestamp), and add whatever data is needed for 
the datasource. Then, the datasource transformer will transform this data into 
a vertex and decide where to connect it to the graph.
[yujunz] So if I understand it correctly, driver and transformer works as a 
pair and the intermediate data format is not exposed to vitrage. It consumes 
data from the sources and convert them into graph
Right.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] spec for datasource

2016-08-09 Thread Yujun Zhang

Hi lfat,

Thank you for the answers, please see my reply inline.

On Tue, Aug 9, 2016 at 6:51 PM Afek, Ifat (Nokia - IL) 
wrote:

> Hi Yujun,
>
> Please see my answers below.
>
> Best Regards,
> Ifat.
>
> From: Yujun Zhang
> Date: Tuesday, 9 August 2016 at 12:06
>
>
> For proprietary datasource, I'm considering to adapt the api to common
> protocol/interface, e.g. RESTful, SNMP and etc and I wish to know how to
> add support for these interface.
>
> [Ifat]: Do you mean that you want to write a datasource that gets its
> information from SNMP/REST API? Note that for alarm datasources, we have in
> our roadmap to support OPNFV Doctor SB REST API[1]. Will it be relevant for
> your use cases?
>
[yujunz] Great, this could also be a good start point for us

> Some documents are found in https://github.com/openstack/vitrage-specs and
>> datasource seems to be related to synchronizer but I didn't find a
>> dedicated spec.
>>
> [Ifat]: We started documenting the process of adding a new datasource, but
> the document is not final. We will try to finish it shortly. BTW, there are
> many other documents in Vitrage wiki page[2].
>
[yujunz] Thanks.

> 1. How do I register a new datasource in an existing system?
>>
> [yujunz] It seems to be in
https://github.com/openstack/vitrage/blob/master/vitrage/datasources/__init__.py


> 2. Is the type of datasource (ALARM/RESOURCE) configured in
>> `/etc/vitrage/datasource_values/.yaml` ?
>>
> [Ifat]: No, it is configured in the datasource code. For information about
> datasource_values please see [3]
>
[yujunz] which code file? I found `category: RESOURCE` in the datasource
configuration file as
https://github.com/openstack/vitrage/blob/master/doc/source/resource-state-config.rst#format

> 3. Is there any other datasource type besides ALARM/RESOURCE?
>
> [Ifat]: No, and at the moment we don’t see a need for that. Vitrage can
> hold resources of any type in a topology graph, and manage alarms that are
> raised on these resources. If you see a use case for other datasource
> types, let us know.
>
[yujunz]  I agree. Just ask to confirm my guess :-)

> 4. What does `aggregated values` and `priority` mean
>>
> [Ifat]: Detailed in [3]
>
[yujunz]: Clear now.

> 5. What is the required data format for the datasource driver api? The
>> inline comments give some brief description but didn't specify the api
>> signature.
>> Many thanks.
>>
> [Ifat]: This should be part of the datasource documentation that we need
> to add. But basically, the driver should return a dictionary with few
> predefined fields (like datasource type, timestamp), and add whatever data
> is needed for the datasource. Then, the datasource transformer will
> transform this data into a vertex and decide where to connect it to the
> graph.
>
[yujunz] So if I understand it correctly, driver and transformer works as a
pair and the intermediate data format is not exposed to vitrage. It
consumes data from the sources and convert them into graph

> [1] https://gerrit.opnfv.org/gerrit/12179
> [2] https://wiki.openstack.org/wiki/Vitrage
> *[3] *
> https://github.com/openstack/vitrage/blob/master/doc/source/resource-state-config.rst
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] spec for datasource

2016-08-09 Thread Malin, Eylon (Nokia - IL)

Hi Yujun,

Regards your questions :
For example, let's say you have 2 packages for your datasources named : 
yujun.ds.snmp ,  yujun.ds.my_alerts

1. For proprietary datasources: 
a. You need that your datasource packages would be in the python path 
or installed in your python site-packages 
   (.e.g in RHEL 7 : under /usr/lib/python2.7/site-packages/)
b. You need to edit the path value pf datasource section in 
vitrage.conf to include the base path of your datasource. 
In our example, you need to edit under [datasource] section, 
the path value to: vitrage.datasources, yujun.ds
c. You need to have in each __init__.py of your datasources an OPTS for 
transformer and driver. See vitrage/vitrage/datasources/aodh/__init__.py for 
example.

2.   The type of datasource can be anything. 
   Best practice is declare it as Constant in the __init__.py of the 
package as some string. For example SNMP=’snmp’.
   And use that Constant as the sync_type in make_pickleable, and as 
entity_type= in the transformer while creating vertex.
   Anyway in  `/etc/vitrage/datasource_values/ you need to put file with 
the same name as the type (string) in the entity_type of the vertex that 
created in the transformer.
   In this example, create `/etc/vitrage/datasource_values/snmp.yaml

3. Can be a lot of datasource types. ALARM/RESOURCE are Entity Category, and 
there are no more categories yet. (see vitrage.common.constants.EntityCategory)
4.  I will let Alexy Weyl to explain about that.
5. What do you mean in API format ? In your driver python class, you need to 
inherit from DriverBase and implement all the abstract methods.
 You can see a lot of examples in current code.
 The Datasource itself (the external system) can use any API, and your 
driver need to communicate with the datasource in the API that the datasource 
support (for example SNMP protocol if it is SNMP datasource)

Hopes that my answers helps you...

BR

Eylon


From: Yujun Zhang [mailto:zhangyujun+...@gmail.com] 
Sent: Tuesday, August 09, 2016 12:07 PM
To: OpenStack Development Mailing List (not for usage questions) 
<openstack-dev@lists.openstack.org>
Subject: Re: [openstack-dev] [vitrage] spec for datasource

Hi, Eylon,

It is not decided yet what datasources will be required. But we may assume 
there will be both open and proprietary ones.

The example of the former could be include service status of nova host in the 
topology (currently only host, instance and zone are found in the code).

For proprietary datasource, I'm considering to adapt the api to common 
protocol/interface, e.g. RESTful, SNMP and etc and I wish to know how to add 
support for these interface.

At the moment, I'm making an evaluation on the extensibility of vitrage 
architecture and estimate the workload for new datasource. Any idea how I can 
proceed?

--
Yujun

On Tue, Aug 9, 2016 at 4:34 PM Malin, Eylon (Nokia - IL) 
<eylon.ma...@nokia.com> wrote:
Hi,

There are different instruction for datasource that are part of openstack 
vitrage upstream, and for propriety datasource.
So for better understanding the case, do you want to add new datasource that 
would be contributed to openstack, or is it propriety one ?
I'm meaning do you plan to push the new datasource to vitrage upstream or leave 
it private ?

Eylon


From: Yujun Zhang [mailto:zhangyujun+...@gmail.com]
Sent: Tuesday, August 09, 2016 10:22 AM
To: OpenStack Development Mailing List (not for usage questions) 
<openstack-dev@lists.openstack.org>
Subject: [openstack-dev] [vitrage] spec for datasource

Dear all,

Is there a guide on how to understand the design of datasource? I want to 
extend the existing one and also try to create a custom datasource from scratch.

Some documents are found in https://github.com/openstack/vitrage-specs and 
datasource seems to be related to synchronizer but I didn't find a dedicated 
spec.

Currently I have the following questions
1. How do I register a new datasource in an existing system?
2. Is the type of datasource (ALARM/RESOURCE) configured in 
`/etc/vitrage/datasource_values/.yaml` ?
3. Is there any other datasource type besides ALARM/RESOURCE?
4. What does `aggregated values` and `priority` mean
5. What is the required data format for the datasource driver api? The inline 
comments give some brief description but didn't specify the api signature.
Many thanks.

--
Yujun
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/

Re: [openstack-dev] [vitrage] spec for datasource

2016-08-09 Thread Afek, Ifat (Nokia - IL)

Hi Yujun,

Please see my answers below.

Best Regards,
Ifat.

From: Yujun Zhang
Date: Tuesday, 9 August 2016 at 12:06

For proprietary datasource, I'm considering to adapt the api to common 
protocol/interface, e.g. RESTful, SNMP and etc and I wish to know how to add 
support for these interface.
[Ifat]: Do you mean that you want to write a datasource that gets its 
information from SNMP/REST API? Note that for alarm datasources, we have in our 
roadmap to support OPNFV Doctor SB REST API[1]. Will it be relevant for your 
use cases?

Some documents are found in https://github.com/openstack/vitrage-specs and 
datasource seems to be related to synchronizer but I didn't find a dedicated 
spec.
[Ifat]: We started documenting the process of adding a new datasource, but the 
document is not final. We will try to finish it shortly. BTW, there are many 
other documents in Vitrage wiki page[2].

1. How do I register a new datasource in an existing system?
2. Is the type of datasource (ALARM/RESOURCE) configured in 
`/etc/vitrage/datasource_values/.yaml` ?
[Ifat]: No, it is configured in the datasource code. For information about 
datasource_values please see [3]

3. Is there any other datasource type besides ALARM/RESOURCE?
[Ifat]: No, and at the moment we don’t see a need for that. Vitrage can hold 
resources of any type in a topology graph, and manage alarms that are raised on 
these resources. If you see a use case for other datasource types, let us know.

4. What does `aggregated values` and `priority` mean
[Ifat]: Detailed in [3]

5. What is the required data format for the datasource driver api? The inline 
comments give some brief description but didn't specify the api signature.
Many thanks.
[Ifat]: This should be part of the datasource documentation that we need to 
add. But basically, the driver should return a dictionary with few predefined 
fields (like datasource type, timestamp), and add whatever data is needed for 
the datasource. Then, the datasource transformer will transform this data into 
a vertex and decide where to connect it to the graph.

[1] https://gerrit.opnfv.org/gerrit/12179
[2] https://wiki.openstack.org/wiki/Vitrage
[3] 
https://github.com/openstack/vitrage/blob/master/doc/source/resource-state-config.rst

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] spec for datasource

2016-08-09 Thread Yujun Zhang

Hi, Eylon,

It is not decided yet what datasources will be required. But we may assume
there will be both open and proprietary ones.

The example of the former could be include service status of nova host in
the topology (currently only host, instance and zone are found in the code).

For proprietary datasource, I'm considering to adapt the api to common
protocol/interface, e.g. RESTful, SNMP and etc and I wish to know how to
add support for these interface.

At the moment, I'm making an evaluation on the extensibility of vitrage
architecture and estimate the workload for new datasource. Any idea how I
can proceed?

--
Yujun

On Tue, Aug 9, 2016 at 4:34 PM Malin, Eylon (Nokia - IL) <
eylon.ma...@nokia.com> wrote:

> Hi,
>
> There are different instruction for datasource that are part of openstack
> vitrage upstream, and for propriety datasource.
> So for better understanding the case, do you want to add new datasource
> that would be contributed to openstack, or is it propriety one ?
> I'm meaning do you plan to push the new datasource to vitrage upstream or
> leave it private ?
>
> Eylon
>
>
> From: Yujun Zhang [mailto:zhangyujun+...@gmail.com]
> Sent: Tuesday, August 09, 2016 10:22 AM
> To: OpenStack Development Mailing List (not for usage questions) <
> openstack-dev@lists.openstack.org>
> Subject: [openstack-dev] [vitrage] spec for datasource
>
> Dear all,
>
> Is there a guide on how to understand the design of datasource? I want to
> extend the existing one and also try to create a custom datasource from
> scratch.
>
> Some documents are found in https://github.com/openstack/vitrage-specs and
> datasource seems to be related to synchronizer but I didn't find a
> dedicated spec.
>
> Currently I have the following questions
> 1. How do I register a new datasource in an existing system?
> 2. Is the type of datasource (ALARM/RESOURCE) configured in
> `/etc/vitrage/datasource_values/.yaml` ?
> 3. Is there any other datasource type besides ALARM/RESOURCE?
> 4. What does `aggregated values` and `priority` mean
> 5. What is the required data format for the datasource driver api? The
> inline comments give some brief description but didn't specify the api
> signature.
> Many thanks.
>
> --
> Yujun
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] spec for datasource

2016-08-09 Thread Malin, Eylon (Nokia - IL)

Hi,

There are different instruction for datasource that are part of openstack 
vitrage upstream, and for propriety datasource.
So for better understanding the case, do you want to add new datasource that 
would be contributed to openstack, or is it propriety one ?
I'm meaning do you plan to push the new datasource to vitrage upstream or leave 
it private ?

Eylon


From: Yujun Zhang [mailto:zhangyujun+...@gmail.com] 
Sent: Tuesday, August 09, 2016 10:22 AM
To: OpenStack Development Mailing List (not for usage questions) 
<openstack-dev@lists.openstack.org>
Subject: [openstack-dev] [vitrage] spec for datasource

Dear all,

Is there a guide on how to understand the design of datasource? I want to 
extend the existing one and also try to create a custom datasource from scratch.

Some documents are found in https://github.com/openstack/vitrage-specs and 
datasource seems to be related to synchronizer but I didn't find a dedicated 
spec.

Currently I have the following questions
1. How do I register a new datasource in an existing system?
2. Is the type of datasource (ALARM/RESOURCE) configured in 
`/etc/vitrage/datasource_values/.yaml` ?
3. Is there any other datasource type besides ALARM/RESOURCE?
4. What does `aggregated values` and `priority` mean
5. What is the required data format for the datasource driver api? The inline 
comments give some brief description but didn't specify the api signature.
Many thanks.

--
Yujun
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [vitrage] spec for datasource

2016-08-09 Thread Yujun Zhang

Dear all,

Is there a guide on how to understand the design of datasource? I want to
extend the existing one and also try to create a custom datasource from
scratch.

Some documents are found in https://github.com/openstack/vitrage-specs and
datasource seems to be related to synchronizer but I didn't find a
dedicated spec.

Currently I have the following questions

   1. How do I register a new datasource in an existing system?
   2. Is the type of datasource (ALARM/RESOURCE) configured in
   `/etc/vitrage/datasource_values/.yaml` ?
   3. Is there any other datasource type besides ALARM/RESOURCE?
   4. What does `aggregated values` and `priority` mean
   5. What is the required data format for the datasource driver api? The
   inline comments give some brief description but didn't specify the api
   signature.

Many thanks.

--
Yujun
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] spec for datasource

Re: [openstack-dev] [vitrage] spec for datasource

Re: [openstack-dev] [vitrage] spec for datasource

Re: [openstack-dev] [vitrage] spec for datasource

Re: [openstack-dev] [vitrage] spec for datasource

Re: [openstack-dev] [vitrage] spec for datasource

[openstack-dev] [vitrage] spec for datasource

7 matches

Site Navigation

Mail list logo

Footer information