[openstack-dev] [congress] generic push driver
We've been discussing generic push drivers for Congress for quite a while. Finally sketching out something concrete and looking for some preliminary feedback. Below are sample interactions with a proposed generic push driver. A generic push driver could be used to receive push updates from vitrage, monasca, and many other sources. 1. creating a datasource: congress datasource create generic_push_driver vitrage --config schema=' { "tables":[ { "name":"alarms", "columns":[ "id", "name", "state", "severity", ] } ] } ' 2. Update an entire table: PUT '/v1/data-sources/vitrage/tables/alarms' with body: { "rows":[ { "id":"1-1", "name":"name1", "state":"active", "severity":1 }, [ "1-2", "name2", "active", 2 ] ] } Note that a row can be either a {} or [] 3. perform differential update: PUT '/v1/data-sources/vitrage/tables/alarms' with body: { "addrows":[ { "id":"1-1", "name":"name1", "state":"active", "severity":1 }, [ "1-2", "name2", "active", 2 ] ] } OR { "deleterows":[ { "id":"1-1", "name":"name1", "state":"active", "severity":1 }, [ "1-2", "name2", "active", 2 ] ] } Note 1: we may allow 'rows', 'addrows', and 'deleterows' to be used together with some well defined semantics. Alternatively we may mandate that each request can have only one of the three pieces. Note 2: we leave it as the responsibility of the sender to send and confirm the requests for differential updates in correct order. We could add sequencing in future work. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [congress] generic push driver
Hi Eric, I have two questions: 1. An alarm is usually raised on a resource, and in Vitrage we can send you the details of that resource. Is there a way in Congress for the alarm to reference a resource that exists in another table? And what if the resource does not exist in Congress? 2. Do you plan to support also updateRows? This can be useful for alarm state changes. Thanks, Ifat From: Eric K Reply-To: "OpenStack Development Mailing List (not for usage questions)" Date: Saturday, 6 January 2018 at 3:50 To: "OpenStack Development Mailing List (not for usage questions)" Subject: [openstack-dev] [congress] generic push driver We've been discussing generic push drivers for Congress for quite a while. Finally sketching out something concrete and looking for some preliminary feedback. Below are sample interactions with a proposed generic push driver. A generic push driver could be used to receive push updates from vitrage, monasca, and many other sources. 1. creating a datasource: congress datasource create generic_push_driver vitrage --config schema=' { "tables":[ { "name":"alarms", "columns":[ "id", "name", "state", "severity", ] } ] } ' 2. Update an entire table: PUT '/v1/data-sources/vitrage/tables/alarms' with body: { "rows":[ { "id":"1-1", "name":"name1", "state":"active", "severity":1 }, [ "1-2", "name2", "active", 2 ] ] } Note that a row can be either a {} or [] 3. perform differential update: PUT '/v1/data-sources/vitrage/tables/alarms' with body: { "addrows":[ { "id":"1-1", "name":"name1", "state":"active", "severity":1 }, [ "1-2", "name2", "active", 2 ] ] } OR { "deleterows":[ { "id":"1-1", "name":"name1", "state":"active", "severity":1 }, [ "1-2", "name2", "active", 2 ] ] } Note 1: we may allow 'rows', 'addrows', and 'deleterows' to be used together with some well defined semantics. Alternatively we may mandate that each request can have only one of the three pieces. Note 2: we leave it as the responsibility of the sender to send and confirm the requests for differential updates in correct order. We could add sequencing in future work. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [congress] generic push driver
It's probably worth considering PATCH instead of PUT for updating the table. http://restcookbook.com/HTTP%20Methods/patch/ You could also think about using JSON-patch to describe the requested update. It provides fine-grained update semantics: https://tools.ietf.org/html/rfc6902 Tim On Fri, Jan 5, 2018 at 5:50 PM Eric K wrote: > We've been discussing generic push drivers for Congress for quite a while. > Finally sketching out something concrete and looking for some preliminary > feedback. Below are sample interactions with a proposed generic push > driver. A generic push driver could be used to receive push updates from > vitrage, monasca, and many other sources. > > 1. creating a datasource: > > congress datasource create generic_push_driver vitrage --config schema=' > { > "tables":[ > { > "name":"alarms", > "columns":[ > "id", > "name", > "state", > "severity", > ] > } > ] > } > ' > > 2. Update an entire table: > > PUT '/v1/data-sources/vitrage/tables/alarms' with body: > { > "rows":[ > { > "id":"1-1", > "name":"name1", > "state":"active", > "severity":1 > }, > [ > "1-2", > "name2", > "active", > 2 > ] > ] > } > Note that a row can be either a {} or [] > > > 3. perform differential update: > > PUT '/v1/data-sources/vitrage/tables/alarms' with body: > { > "addrows":[ > { > "id":"1-1", > "name":"name1", > "state":"active", > "severity":1 > }, > [ > "1-2", > "name2", > "active", > 2 > ] > ] > } > > OR > > { > "deleterows":[ > { > "id":"1-1", > "name":"name1", > "state":"active", > "severity":1 > }, > [ > "1-2", > "name2", > "active", > 2 > ] > ] > } > > Note 1: we may allow 'rows', 'addrows', and 'deleterows' to be used > together with some well defined semantics. Alternatively we may mandate > that each request can have only one of the three pieces. > > Note 2: we leave it as the responsibility of the sender to send and > confirm the requests for differential updates in correct order. We could > add sequencing in future work. > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [congress] generic push driver
Hi Ifat, From: "Afek, Ifat (Nokia - IL/Kfar Sava)" Reply-To: "OpenStack Development Mailing List (not for usage questions)" Date: Sunday, January 7, 2018 at 4:00 AM To: "OpenStack Development Mailing List (not for usage questions)" Subject: Re: [openstack-dev] [congress] generic push driver > Hi Eric, > > I have two questions: > > 1. An alarm is usually raised on a resource, and in Vitrage we can send > you the details of that resource. Is there a way in Congress for the alarm to > reference a resource that exists in another table? And what if the resource > does not exist in Congress? First, the columns I chose are just a minimal sample to illustrate the generic nature of the driver. In use with vitrage, we would probably also want to include columns such as `resource_id`. Does that address the need to reference a resource? That resource referenced by ID may or may not exist in another part of Congress. It would be the job of the policy to resolve references when taking appropriate actions. If referential integrity is needed, additional policy rules can be specified to catch breakage. This brings up a related question I had about vitrage: Looking at the vertex properties listed here: https://github.com/openstack/vitrage/blob/master/vitrage/common/constants.py #L17 Where can I find more information about the type and content of data in each property? Exapmle: - is the `resource` property an ID string or a python object reference? - what does the property `is_real_vitrage_id` represent? - what is the difference between `resource_id` and `vitrage_resource_id` ? > 2. Do you plan to support also updateRows? This can be useful for alarm > state changes. Are you thinking about updating an entire row or updating a specific field of a row? That is, update Row {"id":"1-1", "name":"name1", "state":"active", "severity":1} to become {"id":"1-1", "name":"name1", "state":"active", "severity":100} Vs Update the severity field of row with id "1-1" to severity 100. Both could be supported, but the second one is more complex to support efficiently. Thanks! Eric > > Thanks, > Ifat > > > > From: Eric K > Reply-To: "OpenStack Development Mailing List (not for usage questions)" > > Date: Saturday, 6 January 2018 at 3:50 > To: "OpenStack Development Mailing List (not for usage questions)" > > Subject: [openstack-dev] [congress] generic push driver > > > > We've been discussing generic push drivers for Congress for quite a while. > Finally sketching out something concrete and looking for some preliminary > feedback. Below are sample interactions with a proposed generic push driver. A > generic push driver could be used to receive push updates from vitrage, > monasca, and many other sources. > > > > 1. creating a datasource: > > > > congress datasource create generic_push_driver vitrage --config schema=' > > { > > "tables":[ > > { > > "name":"alarms", > > "columns":[ > > "id", > > "name", > > "state", > > "severity", > > ] > > } > > ] > > } > > ' > > > > 2. Update an entire table: > > > > PUT '/v1/data-sources/vitrage/tables/alarms' with body: > > { > > "rows":[ > > { > > "id":"1-1", > > "name":"name1", > > "state":"active", > > "severity":1 > > }, > > [ > > "1-2", > > "name2", > > "active", > > 2 > > ] > > ] > > } > > Note that a row can be either a {} or [] > > > > > > 3. perform differential update: > > > > PUT '/v1/data-sources/vitrage/tables/alarms' with body: > > { > > "addrows":[ > > { > > "id":"1-1", > > "name":"name1", > > "state":"active", > > "severity":1 > > }, > > [ > > "1-2", > > "name2", > > "active", > > 2 > > ] > > ] > > } > > > > OR > > > > { > > "deleterows":[ > > { >
Re: [openstack-dev] [congress] generic push driver
From: Tim Hinrichs Date: Monday, January 8, 2018 at 7:31 AM To: Eric Kao Cc: "OpenStack Development Mailing List (not for usage questions)" Subject: Re: [congress] generic push driver > It's probably worth considering PATCH instead of PUT for updating the table. Ah right of course. PATCH makes more sense here. > > http://restcookbook.com/HTTP%20Methods/patch/ > > You could also think about using JSON-patch to describe the requested update. > It provides fine-grained update semantics: > > https://tools.ietf.org/html/rfc6902 Hmm it would be very nice to follow an existing standard. Unfortunately the json patch path specifications seem like an awkward fit with the set semantics of congress tables. Removal, for example, must be done by specifying the array index of the row to be removed. But perhaps we can borrow the style of json patch for patching sets. For example: PATCH '/v1/data-sources/vitrage/tables/alarms' with body: [ { "op":"add", "path":"/", "value":{ "id":"1-1", "name":"name1", "state":"active", "severity":1 } }, { "op":"add", "path":"/", "value":[ "1-2", "name2", "active", 2 ] }, { "op":"remove", "path":"/", "value":[ "1-2", "name2", "active", 2 ] } ] Would that work well? At least there will be well-defined semantic based on sequential operation. > > Tim > > On Fri, Jan 5, 2018 at 5:50 PM Eric K wrote: >> We've been discussing generic push drivers for Congress for quite a while. >> Finally sketching out something concrete and looking for some preliminary >> feedback. Below are sample interactions with a proposed generic push driver. >> A generic push driver could be used to receive push updates from vitrage, >> monasca, and many other sources. >> >> 1. creating a datasource: >> >> congress datasource create generic_push_driver vitrage --config schema=' >> { >> "tables":[ >> { >> "name":"alarms", >> "columns":[ >> "id", >> "name", >> "state", >> "severity", >> ] >> } >> ] >> } >> ' >> >> 2. Update an entire table: >> >> PUT '/v1/data-sources/vitrage/tables/alarms' with body: >> { >> "rows":[ >> { >> "id":"1-1", >> "name":"name1", >> "state":"active", >> "severity":1 >> }, >> [ >> "1-2", >> "name2", >> "active", >> 2 >> ] >> ] >> } >> Note that a row can be either a {} or [] >> >> >> 3. perform differential update: >> >> PUT '/v1/data-sources/vitrage/tables/alarms' with body: >> { >> "addrows":[ >> { >> "id":"1-1", >> "name":"name1", >> "state":"active", >> "severity":1 >> }, >> [ >> "1-2", >> "name2", >> "active", >> 2 >> ] >> ] >> } >> >> OR >> >> { >> "deleterows":[ >> { >> "id":"1-1", >> "name":"name1", >> "state":"active", >> "severity":1 >> }, >> [ >> "1-2", >> "name2", >> "active", >> 2 >> ] >> ] >> } >> >> Note 1: we may allow 'rows', 'addrows', and 'deleterows' to be used together >> with some well defined semantics. Alternatively we may mandate that each >> request can have only one of the three pieces. >> >> Note 2: we leave it as the responsibility of the sender to send and confirm >> the requests for differential updates in correct order. We could add >> sequencing in future work. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [congress] generic push driver
From: Eric K Date: Tuesday, 9 January 2018 at 0:43 Hi Ifat, From: "Afek, Ifat (Nokia - IL/Kfar Sava)" mailto:ifat.a...@nokia.com>> Date: Sunday, January 7, 2018 at 4:00 AM Hi Eric, I have two questions: 1. An alarm is usually raised on a resource, and in Vitrage we can send you the details of that resource. Is there a way in Congress for the alarm to reference a resource that exists in another table? And what if the resource does not exist in Congress? First, the columns I chose are just a minimal sample to illustrate the generic nature of the driver. In use with vitrage, we would probably also want to include columns such as `resource_id`. Does that address the need to reference a resource? That resource referenced by ID may or may not exist in another part of Congress. It would be the job of the policy to resolve references when taking appropriate actions. If referential integrity is needed, additional policy rules can be specified to catch breakage. [Ifat] Ok, sounds good. This brings up a related question I had about vitrage: Looking at the vertex properties listed here: https://github.com/openstack/vitrage/blob/master/vitrage/common/constants.py#L17 Where can I find more information about the type and content of data in each property? Exapmle: - is the `resource` property an ID string or a python object reference? [Ifat] Most of the properties are key-value strings on the vertex in the entity graph. The RESOURCE is a special property that is added on an alarm for the use of the notifier. It holds the entire resource object, so the notifier could use its properties when sending notifications. - what does the property `is_real_vitrage_id` represent? [Ifat] It represents old code that should be deleted ;-) please ignore it - what is the difference between `resource_id` and `vitrage_resource_id` ? [Ifat] resource_id is the id of the resource as retrieved by the datasource, e.g. the Nova instance id vitrage_id is the id of the resource inside Vitrage. This is the id that Vitrage uses to identify its resources. For a Nova instance, vitrage_id will be different from its resource_id. vitrage_resource_id is used only on alarms, and holds the vitrage_id of the resource of the alarm. 2. Do you plan to support also updateRows? This can be useful for alarm state changes. Are you thinking about updating an entire row or updating a specific field of a row? That is, update Row {"id":"1-1", "name":"name1", "state":"active", "severity":1} to become {"id":"1-1", "name":"name1", "state":"active", "severity":100} Vs Update the severity field of row with id "1-1" to severity 100. Both could be supported, but the second one is more complex to support efficiently. [Ifat] It’s really up to you, I think both would satisfy the use case. The Congress notifier will be written based on your selected implementation. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev