[opnfv-tech-discuss] [dovetail]if the l3vpn feature is completed fully in C release
Hi Jose: Last dovetail weekly meeting, we have reviewed dovetail test cases and requirements Here is the action item from the last dovetail weekly meeting: Vpn or l3vpn: unclear if the l3vpn feature is completed fully in C release (action: to collect definitive answer for next review). Commented that it should be excluded on the ground of not yet mature and have adoption (see test case reqs & cvp description). Could you please help to get the feedback to the dovetail for the next review? The minutes is attached by the end for email. Best regards hongbo --- Begin Message --- Hi all: Thanks Chris Price to help make the IRC minutes. Chris Price had to drop off early and so IRC didn’t capture the second half of minutes: below is manually taken minutes by wenjing. So the minutes is sent by the email. This is the dovetail weekly meeting minutes: - http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2017/opnfv-meeting.2017-01-13-14.00.log.txt -Chris Price had to drop off early and so IRC didn’t capture the second half of minutes: below is manually taken minutes by wenjing, please correct if you find anything not accurate -Dovetail test case review: wiki https://wiki.opnfv.org/display/dovetail/Dovetail+Test+Areas+and+Test+Cases -Vim operation 5 areas: included -Nfvi and mini-use cases: included, need complete more info in wiki to fully understand (action item: fill in more details) -HA: included. description from HA project. Comment: we should put info in one place for easier access (action item: gather info and fill in here) -Ipv6: included. Same as above to gather info in one place. -Vpn or l3vpn: unclear if the l3vpn feature is completed fully in C release (action: to collect definitive answer for next review). Commented that it should be excluded on the ground of not yet mature and have adoption (see test case reqs & cvp description). -Fault mgmt.: noted that the current feature has dependency on modules such as ceilometer, congress, vitrage etc. that have not yet maturity. To exclude/postpone. -The other test case areas to be reviewed next time. -Topic changed to process and time schedule. -Documentation to-do: https://etherpad.opnfv.org/p/collabrationofdovetail. High level content agreed. Comment: it should have a self-contained format. -Agreed gerrit review and commit process -Test suite: compliance_set.yml: patches (one patch per area) -Test case: e.g. ipv6_tc001.yml: 1 patch per case -Tool: normal software patches -Each area/feature: request project rep to review & cc TWG -Agreed: report to TSC: next Tuesday -Proposed time schedule If I missed something, please do not hesitate to contact me. Thanks Best regards hongbo ___ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss --- End Message --- ___ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
Re: [opnfv-tech-discuss] [dovetail]if the l3vpn feature is completed fully in C release
Hi, Adding Tim who is the PTL of the SDNVPN project. We had a discussion yesterday about it during the weekly team meeting. We would like to remark that the test cases were mature and gave success in Colorado release. However, there were some performance penalties in ODL when activating the bgpvpn plugin in Fuel. When doing so, some other test cases took longer execution time than in non-bgpvpn scenarios. This is being addressed for Danube. Apart from that, the test cases were stable enough to be considered as part of Dovetail test plan. Regards, Jose On 17 Jan 2017, at 03:33, Tianhongbo mailto:hongbo.tianhon...@huawei.com>> wrote: Hi Jose and Chris: Could you please help me to check this action item? Best regards hongbo 发件人: Tianhongbo 发送时间: 2017年1月14日 15:32 收件人: Jose Lausuch 抄送: 'TECH-DISCUSS OPNFV' 主题: [dovetail]if the l3vpn feature is completed fully in C release Hi Jose: Last dovetail weekly meeting, we have reviewed dovetail test cases and requirements Here is the action item from the last dovetail weekly meeting: Vpn or l3vpn: unclear if the l3vpn feature is completed fully in C release (action: to collect definitive answer for next review). Commented that it should be excluded on the ground of not yet mature and have adoption (see test case reqs & cvp description). Could you please help to get the feedback to the dovetail for the next review? The minutes is attached by the end for email. Best regards hongbo ___ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
Re: [opnfv-tech-discuss] [dovetail]if the l3vpn feature is completed fully in C release
Hi Hongbo, Jose, As I was saying on the Dovetail calls, I have some concerns about moving tests into the Dovetail test suite too early. In the Dovetail test requirements, we have: "* Test cases must pass on OPNFV reference deployments * Tests must not require a specific NFVi platform composition or installation tool * Tests must not require unmerged patches to the relevant upstream projects" And in the CVP requirements, we have the following section: "The overall CVP compliance verification scope tied to an OPNFV release is determined by the Committee. The OPNFV TSC defines and maintains the compliance verification procedures and associated tools. The scope is constrained to features, capabilities, components, and interfaces included in an OPNFV release that are generally available in the industry (e.g., through adoption by an upstream community)." I wonder if this functionality is sufficiently widely adopted in commercial NFVi and VIM solutions to pass this bar. Thanks, Dave. On 01/14/2017 02:32 AM, Tianhongbo wrote: > Hi Jose: > > > > Last dovetail weekly meeting, we have reviewed dovetail test cases and > requirements > > > > Here is the action item from the last dovetail weekly meeting: > > > > Vpn or l3vpn: unclear if the l3vpn feature is completed fully in C > release (action: to collect definitive answer for next review). > Commented that it should be excluded on the ground of not yet mature and > have adoption (see test case reqs & cvp description). > > > > Could you please help to get the feedback to the dovetail for the next > review? > > The minutes is attached by the end for email. > > > > Best regards > > > > hongbo > > > > ___ > opnfv-tech-discuss mailing list > opnfv-tech-discuss@lists.opnfv.org > https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss > -- Dave Neary - NFV/SDN Community Strategy Open Source and Standards, Red Hat - http://community.redhat.com Ph: +1-978-399-2182 / Cell: +1-978-799-3338 ___ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
Re: [opnfv-tech-discuss] [dovetail]if the l3vpn feature is completed fully in C release
On 01/18/2017 12:53 AM, Dave Neary wrote: Hi Hongbo, Jose, As I was saying on the Dovetail calls, I have some concerns about moving tests into the Dovetail test suite too early. In the Dovetail test requirements, we have: "* Test cases must pass on OPNFV reference deployments * Tests must not require a specific NFVi platform composition or installation tool * Tests must not require unmerged patches to the relevant upstream projects" And in the CVP requirements, we have the following section: "The overall CVP compliance verification scope tied to an OPNFV release is determined by the Committee. The OPNFV TSC defines and maintains the compliance verification procedures and associated tools. The scope is constrained to features, capabilities, components, and interfaces included in an OPNFV release that are generally available in the industry (e.g., through adoption by an upstream community)." I wonder if this functionality is sufficiently widely adopted in commercial NFVi and VIM solutions to pass this bar. Thanks, Dave. I have no opinion about L3VPN as such, but I read this to mean that the code should be part of a released upstream projects and that OPNFV installers should all support it. What would be the best way to discuss these? Currently, the test cases are on a wiki page which makes it a little difficult to comment them. Would it make sense to copy the whole test areas and test cases wiki page to an Etherpad? Or should the whole page be put to gerrit for commenting? -Tapio ___ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
Re: [opnfv-tech-discuss] [dovetail]if the l3vpn feature is completed fully in C release
Hmm, I was not aware that “all installers must support” a feature for there to be a dovetail suite to validate it. Maybe we should review the “qualification criteria” again on Friday’s call. Completely agree that we need to do this in Gerrit. / chris On 2017-01-18, 08:59, "Tapio Tallgren" wrote: On 01/18/2017 12:53 AM, Dave Neary wrote: > Hi Hongbo, Jose, > > As I was saying on the Dovetail calls, I have some concerns about moving > tests into the Dovetail test suite too early. > > In the Dovetail test requirements, we have: > > "* Test cases must pass on OPNFV reference deployments >* Tests must not require a specific NFVi platform composition or > installation tool >* Tests must not require unmerged patches to the relevant upstream > projects" > > And in the CVP requirements, we have the following section: > > "The overall CVP compliance verification scope tied to an OPNFV release > is determined by the Committee. The OPNFV TSC defines and maintains the > compliance verification procedures and associated tools. The scope is > constrained to features, capabilities, components, and interfaces > included in an OPNFV release that are generally available in the > industry (e.g., through adoption by an upstream community)." > > > I wonder if this functionality is sufficiently widely adopted in > commercial NFVi and VIM solutions to pass this bar. > > Thanks, > Dave. I have no opinion about L3VPN as such, but I read this to mean that the code should be part of a released upstream projects and that OPNFV installers should all support it. What would be the best way to discuss these? Currently, the test cases are on a wiki page which makes it a little difficult to comment them. Would it make sense to copy the whole test areas and test cases wiki page to an Etherpad? Or should the whole page be put to gerrit for commenting? -Tapio ___ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss ___ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
Re: [opnfv-tech-discuss] [dovetail]if the l3vpn feature is completed fully in C release
Me neither. If that were the case, that feature Was tested only in Fuel during Colorado. Let's follow up on Friday. - Jose - -Original Message- From: opnfv-tech-discuss-boun...@lists.opnfv.org [mailto:opnfv-tech-discuss-boun...@lists.opnfv.org] On Behalf Of Christopher Price Sent: Wednesday, January 18, 2017 9:32 AM To: Tapio Tallgren; opnfv-tech-discuss@lists.opnfv.org Subject: Re: [opnfv-tech-discuss] [dovetail]if the l3vpn feature is completed fully in C release Hmm, I was not aware that “all installers must support” a feature for there to be a dovetail suite to validate it. Maybe we should review the “qualification criteria” again on Friday’s call. Completely agree that we need to do this in Gerrit. / chris On 2017-01-18, 08:59, "Tapio Tallgren" wrote: On 01/18/2017 12:53 AM, Dave Neary wrote: > Hi Hongbo, Jose, > > As I was saying on the Dovetail calls, I have some concerns about moving > tests into the Dovetail test suite too early. > > In the Dovetail test requirements, we have: > > "* Test cases must pass on OPNFV reference deployments >* Tests must not require a specific NFVi platform composition or > installation tool >* Tests must not require unmerged patches to the relevant upstream > projects" > > And in the CVP requirements, we have the following section: > > "The overall CVP compliance verification scope tied to an OPNFV release > is determined by the Committee. The OPNFV TSC defines and maintains the > compliance verification procedures and associated tools. The scope is > constrained to features, capabilities, components, and interfaces > included in an OPNFV release that are generally available in the > industry (e.g., through adoption by an upstream community)." > > > I wonder if this functionality is sufficiently widely adopted in > commercial NFVi and VIM solutions to pass this bar. > > Thanks, > Dave. I have no opinion about L3VPN as such, but I read this to mean that the code should be part of a released upstream projects and that OPNFV installers should all support it. What would be the best way to discuss these? Currently, the test cases are on a wiki page which makes it a little difficult to comment them. Would it make sense to copy the whole test areas and test cases wiki page to an Etherpad? Or should the whole page be put to gerrit for commenting? -Tapio ___ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss ___ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss ___ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
Re: [opnfv-tech-discuss] [dovetail]if the l3vpn feature is completed fully in C release
Good topic, I also felt that the criteria were not too clear. My interpretation was that if we are testing a feature that should be in all OPNFV platforms and which is generally available in the industry, and which does not require a specific installation tool, then many OPNFV installers would support it. Perhaps even all of them. -Tapio On 01/18/2017 11:38 AM, Jose Lausuch wrote: Me neither. If that were the case, that feature Was tested only in Fuel during Colorado. Let's follow up on Friday. - Jose - -Original Message- From: opnfv-tech-discuss-boun...@lists.opnfv.org [mailto:opnfv-tech-discuss-boun...@lists.opnfv.org] On Behalf Of Christopher Price Sent: Wednesday, January 18, 2017 9:32 AM To: Tapio Tallgren; opnfv-tech-discuss@lists.opnfv.org Subject: Re: [opnfv-tech-discuss] [dovetail]if the l3vpn feature is completed fully in C release Hmm, I was not aware that “all installers must support” a feature for there to be a dovetail suite to validate it. Maybe we should review the “qualification criteria” again on Friday’s call. Completely agree that we need to do this in Gerrit. / chris On 2017-01-18, 08:59, "Tapio Tallgren" wrote: On 01/18/2017 12:53 AM, Dave Neary wrote: > Hi Hongbo, Jose, > > As I was saying on the Dovetail calls, I have some concerns about moving > tests into the Dovetail test suite too early. > > In the Dovetail test requirements, we have: > > "* Test cases must pass on OPNFV reference deployments >* Tests must not require a specific NFVi platform composition or > installation tool >* Tests must not require unmerged patches to the relevant upstream > projects" > > And in the CVP requirements, we have the following section: > > "The overall CVP compliance verification scope tied to an OPNFV release > is determined by the Committee. The OPNFV TSC defines and maintains the > compliance verification procedures and associated tools. The scope is > constrained to features, capabilities, components, and interfaces > included in an OPNFV release that are generally available in the > industry (e.g., through adoption by an upstream community)." > > > I wonder if this functionality is sufficiently widely adopted in > commercial NFVi and VIM solutions to pass this bar. > > Thanks, > Dave. I have no opinion about L3VPN as such, but I read this to mean that the code should be part of a released upstream projects and that OPNFV installers should all support it. What would be the best way to discuss these? Currently, the test cases are on a wiki page which makes it a little difficult to comment them. Would it make sense to copy the whole test areas and test cases wiki page to an Etherpad? Or should the whole page be put to gerrit for commenting? -Tapio ___ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss ___ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss ___ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
Re: [opnfv-tech-discuss] [dovetail]if the l3vpn feature is completed fully in C release
Le 18/01/2017 à 11:07, Tapio Tallgren a écrit : > Good topic, I also felt that the criteria were not too clear. > > My interpretation was that if we are testing a feature that should be > in all OPNFV platforms and which is generally available in the > industry, and which does not require a specific installation tool, > then many OPNFV installers would support it. Perhaps even all of them. part of the feature alignment for "mature" features evoked in the discussion on priorities not realistic for Danube, but could be for E and somehow linked to the discussion on scenario refactoring it is a richness to have several installers until a feature is not mature, it makes fully sense to focus on only 1 installer in specific scenario(s) but when the integration is done and available since 1 or 2 OPNFV versions, the feature should be adopted by most of/all the installers in generic scenario(s) it will be useful for certification (and we are back to an old thread... when we say we cannot certify a feature that is not supported by 80% of the scenarios we are releasing...today that is the case of lots of features that are installer dependant) /Morgan > > -Tapio > > > On 01/18/2017 11:38 AM, Jose Lausuch wrote: >> Me neither. If that were the case, that feature Was tested only in >> Fuel during Colorado. >> >> Let's follow up on Friday. >> >> - Jose - >> >> >> -Original Message- >> From: opnfv-tech-discuss-boun...@lists.opnfv.org >> [mailto:opnfv-tech-discuss-boun...@lists.opnfv.org] On Behalf Of >> Christopher Price >> Sent: Wednesday, January 18, 2017 9:32 AM >> To: Tapio Tallgren; opnfv-tech-discuss@lists.opnfv.org >> Subject: Re: [opnfv-tech-discuss] [dovetail]if the l3vpn feature is >> completed fully in C release >> >> Hmm, >> >> I was not aware that “all installers must support” a feature for >> there to be a dovetail suite to validate it. >> Maybe we should review the “qualification criteria” again on Friday’s >> call. >> >> Completely agree that we need to do this in Gerrit. >> >> / chris >> >> On 2017-01-18, 08:59, "Tapio Tallgren" >> > tapio.tallg...@nokia.com> wrote: >> >> On 01/18/2017 12:53 AM, Dave Neary wrote: >> > Hi Hongbo, Jose, >> > >> > As I was saying on the Dovetail calls, I have some concerns >> about moving >> > tests into the Dovetail test suite too early. >> > >> > In the Dovetail test requirements, we have: >> > >> > "* Test cases must pass on OPNFV reference deployments >> >* Tests must not require a specific NFVi platform >> composition or >> > installation tool >> >* Tests must not require unmerged patches to the relevant >> upstream >> > projects" >> > >> > And in the CVP requirements, we have the following section: >> > >> > "The overall CVP compliance verification scope tied to an >> OPNFV release >> > is determined by the Committee. The OPNFV TSC defines and >> maintains the >> > compliance verification procedures and associated tools. The >> scope is >> > constrained to features, capabilities, components, and interfaces >> > included in an OPNFV release that are generally available in the >> > industry (e.g., through adoption by an upstream community)." >> > >> > >> > I wonder if this functionality is sufficiently widely adopted in >> > commercial NFVi and VIM solutions to pass this bar. >> > >> > Thanks, >> > Dave. >> I have no opinion about L3VPN as such, but I read this to >> mean that the >> code should be part of a released upstream projects and that OPNFV >> installers should all support it. >> What would be the best way to discuss these? Currently, the >> test cases >> are on a wiki page which makes it a little difficult to comment >> them. >> Would it make sense to copy the whole test areas and test cases >> wiki >> page to an Etherpad? Or should the whole page be put to gerrit for >> commenting? >> -Tapio >> ___ >> opnfv-tech-discuss mailing list >> opnfv-tech-discuss@lists.opnfv.org >> https://lists.opnfv.org/mailman/
Re: [opnfv-tech-discuss] [dovetail]if the l3vpn feature is completed fully in C release
Hi, On 01/18/2017 03:31 AM, Christopher Price wrote: > I was not aware that “all installers must support” a feature for there to be > a dovetail suite to validate it. > Maybe we should review the “qualification criteria” again on Friday’s call. The wording we came up with in the test requirements was: * Tests must not require a specific NFVi platform composition or installation tool In other words, not all, but at least 2 installers and 2 SDN controllers should support the feature. Dave. > Completely agree that we need to do this in Gerrit. > > / chris > > On 2017-01-18, 08:59, "Tapio Tallgren" > tapio.tallg...@nokia.com> wrote: > > On 01/18/2017 12:53 AM, Dave Neary wrote: > > Hi Hongbo, Jose, > > > > As I was saying on the Dovetail calls, I have some concerns about moving > > tests into the Dovetail test suite too early. > > > > In the Dovetail test requirements, we have: > > > > "* Test cases must pass on OPNFV reference deployments > >* Tests must not require a specific NFVi platform composition or > > installation tool > >* Tests must not require unmerged patches to the relevant upstream > > projects" > > > > And in the CVP requirements, we have the following section: > > > > "The overall CVP compliance verification scope tied to an OPNFV release > > is determined by the Committee. The OPNFV TSC defines and maintains the > > compliance verification procedures and associated tools. The scope is > > constrained to features, capabilities, components, and interfaces > > included in an OPNFV release that are generally available in the > > industry (e.g., through adoption by an upstream community)." > > > > > > I wonder if this functionality is sufficiently widely adopted in > > commercial NFVi and VIM solutions to pass this bar. > > > > Thanks, > > Dave. > > I have no opinion about L3VPN as such, but I read this to mean that the > code should be part of a released upstream projects and that OPNFV > installers should all support it. > > What would be the best way to discuss these? Currently, the test cases > are on a wiki page which makes it a little difficult to comment them. > Would it make sense to copy the whole test areas and test cases wiki > page to an Etherpad? Or should the whole page be put to gerrit for > commenting? > > -Tapio > > ___ > opnfv-tech-discuss mailing list > opnfv-tech-discuss@lists.opnfv.org > https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss > > > > ___ > opnfv-tech-discuss mailing list > opnfv-tech-discuss@lists.opnfv.org > https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss > -- Dave Neary - NFV/SDN Community Strategy Open Source and Standards, Red Hat - http://community.redhat.com Ph: +1-978-399-2182 / Cell: +1-978-799-3338 ___ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
Re: [opnfv-tech-discuss] [dovetail]if the l3vpn feature is completed fully in C release
Maybe wordsmithing, but I think "2 installers and 2 SDN controllers" still means "specific". It is just slightly loosened from "specific one" to "specific two". I believe that the essence of, and also logically, "must not require a specific ..." really means "any currently available" in OPNFV. Thanks Bin -Original Message- From: opnfv-tech-discuss-boun...@lists.opnfv.org [mailto:opnfv-tech-discuss-boun...@lists.opnfv.org] On Behalf Of Dave Neary Sent: Wednesday, January 18, 2017 12:20 PM To: Christopher Price ; Tapio Tallgren ; opnfv-tech-discuss@lists.opnfv.org Subject: Re: [opnfv-tech-discuss] [dovetail]if the l3vpn feature is completed fully in C release Hi, On 01/18/2017 03:31 AM, Christopher Price wrote: > I was not aware that “all installers must support” a feature for there to be > a dovetail suite to validate it. > Maybe we should review the “qualification criteria” again on Friday’s call. The wording we came up with in the test requirements was: * Tests must not require a specific NFVi platform composition or installation tool In other words, not all, but at least 2 installers and 2 SDN controllers should support the feature. Dave. > Completely agree that we need to do this in Gerrit. > > / chris > > On 2017-01-18, 08:59, "Tapio Tallgren" > tapio.tallg...@nokia.com> wrote: > > On 01/18/2017 12:53 AM, Dave Neary wrote: > > Hi Hongbo, Jose, > > > > As I was saying on the Dovetail calls, I have some concerns about moving > > tests into the Dovetail test suite too early. > > > > In the Dovetail test requirements, we have: > > > > "* Test cases must pass on OPNFV reference deployments > >* Tests must not require a specific NFVi platform composition or > > installation tool > >* Tests must not require unmerged patches to the relevant upstream > > projects" > > > > And in the CVP requirements, we have the following section: > > > > "The overall CVP compliance verification scope tied to an OPNFV release > > is determined by the Committee. The OPNFV TSC defines and maintains the > > compliance verification procedures and associated tools. The scope is > > constrained to features, capabilities, components, and interfaces > > included in an OPNFV release that are generally available in the > > industry (e.g., through adoption by an upstream community)." > > > > > > I wonder if this functionality is sufficiently widely adopted in > > commercial NFVi and VIM solutions to pass this bar. > > > > Thanks, > > Dave. > > I have no opinion about L3VPN as such, but I read this to mean that the > code should be part of a released upstream projects and that OPNFV > installers should all support it. > > What would be the best way to discuss these? Currently, the test cases > are on a wiki page which makes it a little difficult to comment them. > Would it make sense to copy the whole test areas and test cases wiki > page to an Etherpad? Or should the whole page be put to gerrit for > commenting? > > -Tapio > > ___ > opnfv-tech-discuss mailing list > opnfv-tech-discuss@lists.opnfv.org > https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss > > > > ___ > opnfv-tech-discuss mailing list > opnfv-tech-discuss@lists.opnfv.org > https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss > -- Dave Neary - NFV/SDN Community Strategy Open Source and Standards, Red Hat - http://community.redhat.com Ph: +1-978-399-2182 / Cell: +1-978-799-3338 ___ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss ___ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
Re: [opnfv-tech-discuss] [dovetail]if the l3vpn feature is completed fully in C release
Hi, For my part, my understanding of this when Chris, Hongbo and myself wrote it in New Hampshire, our intention was to communicate that a feature was not elligible if it was only done for one specific stack or one installer - this was our best guess at communicating "widely available". I think all installers and all possible stack combinations would be excessive. Thanks, Dave. On 01/18/2017 03:37 PM, HU, BIN wrote: > Maybe wordsmithing, but I think "2 installers and 2 SDN controllers" still > means "specific". It is just slightly loosened from "specific one" to > "specific two". > > I believe that the essence of, and also logically, "must not require a > specific ..." really means "any currently available" in OPNFV. > > Thanks > Bin > > -Original Message- > From: opnfv-tech-discuss-boun...@lists.opnfv.org > [mailto:opnfv-tech-discuss-boun...@lists.opnfv.org] On Behalf Of Dave Neary > Sent: Wednesday, January 18, 2017 12:20 PM > To: Christopher Price ; Tapio Tallgren > ; opnfv-tech-discuss@lists.opnfv.org > Subject: Re: [opnfv-tech-discuss] [dovetail]if the l3vpn feature is completed > fully in C release > > Hi, > > On 01/18/2017 03:31 AM, Christopher Price wrote: >> I was not aware that “all installers must support” a feature for there to be >> a dovetail suite to validate it. >> Maybe we should review the “qualification criteria” again on Friday’s call. > > The wording we came up with in the test requirements was: > * Tests must not require a specific NFVi platform composition or installation > tool > > In other words, not all, but at least 2 installers and 2 SDN controllers > should support the feature. > > Dave. > >> Completely agree that we need to do this in Gerrit. >> >> / chris >> >> On 2017-01-18, 08:59, "Tapio Tallgren" >> > tapio.tallg...@nokia.com> wrote: >> >> On 01/18/2017 12:53 AM, Dave Neary wrote: >> > Hi Hongbo, Jose, >> > >> > As I was saying on the Dovetail calls, I have some concerns about >> moving >> > tests into the Dovetail test suite too early. >> > >> > In the Dovetail test requirements, we have: >> > >> > "* Test cases must pass on OPNFV reference deployments >> >* Tests must not require a specific NFVi platform composition or >> > installation tool >> >* Tests must not require unmerged patches to the relevant upstream >> > projects" >> > >> > And in the CVP requirements, we have the following section: >> > >> > "The overall CVP compliance verification scope tied to an OPNFV release >> > is determined by the Committee. The OPNFV TSC defines and maintains the >> > compliance verification procedures and associated tools. The scope is >> > constrained to features, capabilities, components, and interfaces >> > included in an OPNFV release that are generally available in the >> > industry (e.g., through adoption by an upstream community)." >> > >> > >> > I wonder if this functionality is sufficiently widely adopted in >> > commercial NFVi and VIM solutions to pass this bar. >> > >> > Thanks, >> > Dave. >> >> I have no opinion about L3VPN as such, but I read this to mean that the >> code should be part of a released upstream projects and that OPNFV >> installers should all support it. >> >> What would be the best way to discuss these? Currently, the test cases >> are on a wiki page which makes it a little difficult to comment them. >> Would it make sense to copy the whole test areas and test cases wiki >> page to an Etherpad? Or should the whole page be put to gerrit for >> commenting? >> >> -Tapio >> >> ___ >> opnfv-tech-discuss mailing list >> opnfv-tech-discuss@lists.opnfv.org >> https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss >> >> >> >> ___ >> opnfv-tech-discuss mailing list >> opnfv-tech-discuss@lists.opnfv.org >> https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss >> > > -- > Dave Neary - NFV/SDN Community Strategy > Open Source and Standards, Red Hat - http://community.redhat.com > Ph: +1-978-399-2182 / Cell: +1-978-799-3338 > ___ > opnfv-tech-discuss mailing list > opnfv-tech-discuss@lists.opnfv.org > https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss > -- Dave Neary - NFV/SDN Community Strategy Open Source and Standards, Red Hat - http://community.redhat.com Ph: +1-978-399-2182 / Cell: +1-978-799-3338 ___ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
Re: [opnfv-tech-discuss] [dovetail]if the l3vpn feature is completed fully in C release
Thanks for background information. I think the challenge is how to quantify "widely available". 50% or less is certainly not "widely available". Maybe 80%? 90%? If 100% is excessive. Then it tranlates to 4 installers (given current 5 in total), and relevant combinations. Thanks Bin -Original Message- From: Dave Neary [mailto:dne...@redhat.com] Sent: Wednesday, January 18, 2017 12:41 PM To: HU, BIN ; Christopher Price ; Tapio Tallgren ; opnfv-tech-discuss@lists.opnfv.org Subject: Re: [opnfv-tech-discuss] [dovetail]if the l3vpn feature is completed fully in C release Hi, For my part, my understanding of this when Chris, Hongbo and myself wrote it in New Hampshire, our intention was to communicate that a feature was not elligible if it was only done for one specific stack or one installer - this was our best guess at communicating "widely available". I think all installers and all possible stack combinations would be excessive. Thanks, Dave. On 01/18/2017 03:37 PM, HU, BIN wrote: > Maybe wordsmithing, but I think "2 installers and 2 SDN controllers" still > means "specific". It is just slightly loosened from "specific one" to > "specific two". > > I believe that the essence of, and also logically, "must not require a > specific ..." really means "any currently available" in OPNFV. > > Thanks > Bin > > -Original Message- > From: opnfv-tech-discuss-boun...@lists.opnfv.org > [mailto:opnfv-tech-discuss-boun...@lists.opnfv.org] On Behalf Of Dave > Neary > Sent: Wednesday, January 18, 2017 12:20 PM > To: Christopher Price ; Tapio Tallgren > ; opnfv-tech-discuss@lists.opnfv.org > Subject: Re: [opnfv-tech-discuss] [dovetail]if the l3vpn feature is > completed fully in C release > > Hi, > > On 01/18/2017 03:31 AM, Christopher Price wrote: >> I was not aware that “all installers must support” a feature for there to be >> a dovetail suite to validate it. >> Maybe we should review the “qualification criteria” again on Friday’s call. > > The wording we came up with in the test requirements was: > * Tests must not require a specific NFVi platform composition or > installation tool > > In other words, not all, but at least 2 installers and 2 SDN controllers > should support the feature. > > Dave. > >> Completely agree that we need to do this in Gerrit. >> >> / chris >> >> On 2017-01-18, 08:59, "Tapio Tallgren" >> > tapio.tallg...@nokia.com> wrote: >> >> On 01/18/2017 12:53 AM, Dave Neary wrote: >> > Hi Hongbo, Jose, >> > >> > As I was saying on the Dovetail calls, I have some concerns about >> moving >> > tests into the Dovetail test suite too early. >> > >> > In the Dovetail test requirements, we have: >> > >> > "* Test cases must pass on OPNFV reference deployments >> >* Tests must not require a specific NFVi platform composition or >> > installation tool >> >* Tests must not require unmerged patches to the relevant upstream >> > projects" >> > >> > And in the CVP requirements, we have the following section: >> > >> > "The overall CVP compliance verification scope tied to an OPNFV release >> > is determined by the Committee. The OPNFV TSC defines and maintains the >> > compliance verification procedures and associated tools. The scope is >> > constrained to features, capabilities, components, and interfaces >> > included in an OPNFV release that are generally available in the >> > industry (e.g., through adoption by an upstream community)." >> > >> > >> > I wonder if this functionality is sufficiently widely adopted in >> > commercial NFVi and VIM solutions to pass this bar. >> > >> > Thanks, >> > Dave. >> >> I have no opinion about L3VPN as such, but I read this to mean that the >> code should be part of a released upstream projects and that OPNFV >> installers should all support it. >> >> What would be the best way to discuss these? Currently, the test cases >> are on a wiki page which makes it a little difficult to comment them. >> Would it make sense to copy the whole test areas and test cases wiki >> page to an Etherpad? Or should the whole page be put to gerrit for >> commenting? >> >> -Tapio >> >> ___ >
Re: [opnfv-tech-discuss] [dovetail]if the l3vpn feature is completed fully in C release
Good discussions. Another area that we need to consider is the availability of good quality test cases in OPNFV. It is one thing to consider if a particular feature itself is "mature', we also need a set of test cases that can adequately evaluate the feature as part of Dovetail. Projects (and testing projects functest/yardstick) should also evaluate the "maturity" of their test suites as one of the criteria. Morgan, The old thread on percentage of scenarios is not without valid points, but the relevance/significance of scenarios is a disqualifying issue IMO. I'll illustrate with an example, E.g. I looked into all the scenarios in Colorado in the wiki, and filter them with 1) participates in at least one of C-1.0, 2.0, 3.0 2) supports ha 2) supports 2 or more installers If the above hypothetical criteria sounds reasonable, then there are only 6 scenarios remain. One can imagine a case where someone pass all the other scenarios (39 out of total 45 scenarios = 87%) , but not really address what we think as important. This is the state we found ourselves in at Colorado, which, I hope, future releases can evolve/improve. Here are the 6 scenarios I was able to find. I used this table for source (may have inaccuracy in it - I didn’t try to verify): https://wiki.opnfv.org/display/SWREL/Colorado+Scenario+Status os-odl_l3-nofeature-ha Apex,Compass,Fuel os-nosdn-nofeature-ha Apex,Compass,Fuel, Joid os-odl_l2-nofeature-ha Apex,Compass,Fuel, Joid os-onos-nofeature-haApex,Compass,Joid os-odl_l2-bgpvpn-ha Apex,Fuel os-onos-sfc-ha Compass,Fuel,Joid Regards, Wenjing -Original Message- From: opnfv-tech-discuss-boun...@lists.opnfv.org [mailto:opnfv-tech-discuss-boun...@lists.opnfv.org] On Behalf Of morgan.richo...@orange.com Sent: Wednesday, January 18, 2017 6:50 PM To: Tapio Tallgren ; Jose Lausuch ; Christopher Price ; opnfv-tech-discuss@lists.opnfv.org Subject: Re: [opnfv-tech-discuss] [dovetail]if the l3vpn feature is completed fully in C release Le 18/01/2017 à 11:07, Tapio Tallgren a écrit : > Good topic, I also felt that the criteria were not too clear. > > My interpretation was that if we are testing a feature that should be > in all OPNFV platforms and which is generally available in the > industry, and which does not require a specific installation tool, > then many OPNFV installers would support it. Perhaps even all of them. part of the feature alignment for "mature" features evoked in the discussion on priorities not realistic for Danube, but could be for E and somehow linked to the discussion on scenario refactoring it is a richness to have several installers until a feature is not mature, it makes fully sense to focus on only 1 installer in specific scenario(s) but when the integration is done and available since 1 or 2 OPNFV versions, the feature should be adopted by most of/all the installers in generic scenario(s) it will be useful for certification (and we are back to an old thread... when we say we cannot certify a feature that is not supported by 80% of the scenarios we are releasing...today that is the case of lots of features that are installer dependant) /Morgan > > -Tapio > > > On 01/18/2017 11:38 AM, Jose Lausuch wrote: >> Me neither. If that were the case, that feature Was tested only in >> Fuel during Colorado. >> >> Let's follow up on Friday. >> >> - Jose - >> >> >> -Original Message- >> From: opnfv-tech-discuss-boun...@lists.opnfv.org >> [mailto:opnfv-tech-discuss-boun...@lists.opnfv.org] On Behalf Of >> Christopher Price >> Sent: Wednesday, January 18, 2017 9:32 AM >> To: Tapio Tallgren; opnfv-tech-discuss@lists.opnfv.org >> Subject: Re: [opnfv-tech-discuss] [dovetail]if the l3vpn feature is >> completed fully in C release >> >> Hmm, >> >> I was not aware that “all installers must support” a feature for >> there to be a dovetail suite to validate it. >> Maybe we should review the “qualification criteria” again on Friday’s >> call. >> >> Completely agree that we need to do this in Gerrit. >> >> / chris >> >> On 2017-01-18, 08:59, "Tapio Tallgren" >> > tapio.tallg...@nokia.com> wrote: >> >> On 01/18/2017 12:53 AM, Dave Neary wrote: >> > Hi Hongbo, Jose, >> > >> > As I was saying on the Dovetail calls, I have some concerns >> about moving >> > tests into the Dovetail test suite too early. >> > >> > In the Dovetail test requirements, we have: >> > >> > "* Test cases must pass on OPNFV reference deployments >> >* Tests must not require a specific NFVi platform >> composition or >>
