Re: Review Request 15247: [DOC] CLOUDSTACK-4967: Add a section to describe VNI allocation matter
It's already committed :) > commit 002c2b22f43f7ace0ade652e92c2d46b92c3136f in 4.3 -- Toshiaki 2014-01-24 Radhika Puthiyetath > Please apply this patch to 4.3 branch as well. > > -Original Message- > From: Toshiaki Hatano [mailto:nore...@reviews.apache.org] On Behalf Of > Toshiaki Hatano > Sent: Thursday, January 23, 2014 10:53 PM > To: Marcus Sorensen; Toshiaki Hatano > Cc: Yoshikazu Nojima; cloudstack > Subject: Re: Review Request 15247: [DOC] CLOUDSTACK-4967: Add a section to > describe VNI allocation matter > > > > > On Jan. 23, 2014, 5:08 p.m., Toshiaki Hatano wrote: > > > Ship It! > > applied to master and 4.3. > commit c9b8bc0884e9e1166046bfdd1354a991d39d7d71 in master commit > 002c2b22f43f7ace0ade652e92c2d46b92c3136f in 4.3 > > > - Toshiaki > > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/15247/#review32625 > --- > > > On Jan. 22, 2014, 5:44 a.m., Yoshikazu Nojima wrote: > > > > --- > > This is an automatically generated e-mail. To reply, visit: > > https://reviews.apache.org/r/15247/ > > ------- > > > > (Updated Jan. 22, 2014, 5:44 a.m.) > > > > > > Review request for cloudstack, Marcus Sorensen and Toshiaki Hatano. > > > > > > Bugs: CLOUDSTACK-4967 > > https://issues.apache.org/jira/browse/CLOUDSTACK-4967 > > > > > > Repository: cloudstack-docs > > > > > > Description > > --- > > > > This patch updates CloudStack Plugin Guide for the VXLAN Plugin. > > > > - Add a section to describe VNIs allocation matter > > - Remove section that explain how to configure the bridge for traffic > label because the bridge configuration is no longer necessary. > > - Update screen shots to catch up new UI visual appearance. > > > > > > Diffs > > - > > > > vxlan/en-US/images/vxlan-physicalnetwork.png > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > > vxlan/en-US/images/vxlan-trafficlabel.png > 956d5f4f58f592ed3f260d6e1982c238c2ddf06a > > vxlan/en-US/images/vxlan-vniconfig.png PRE-CREATION > > vxlan/en-US/plugin-vxlan-config-hypervisor.xml 2c5e138 > > vxlan/en-US/plugin-vxlan-config-management.xml 21f5461 > > vxlan/en-US/plugin-vxlan-requirements.xml c2e04a6 > > vxlan/en-US/plugin-vxlan-revision-history.xml ec04d11 > > > > Diff: https://reviews.apache.org/r/15247/diff/ > > > > > > Testing > > --- > > > > I generated and confirmed the PDF. > > > > > > Thanks, > > > > Yoshikazu Nojima > > > > > >
Re: Review Request 15247: [DOC] CLOUDSTACK-4967: Add a section to describe VNI allocation matter
> On Jan. 23, 2014, 5:08 p.m., Toshiaki Hatano wrote: > > Ship It! applied to master and 4.3. commit c9b8bc0884e9e1166046bfdd1354a991d39d7d71 in master commit 002c2b22f43f7ace0ade652e92c2d46b92c3136f in 4.3 - Toshiaki --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15247/#review32625 --- On Jan. 22, 2014, 5:44 a.m., Yoshikazu Nojima wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/15247/ > --- > > (Updated Jan. 22, 2014, 5:44 a.m.) > > > Review request for cloudstack, Marcus Sorensen and Toshiaki Hatano. > > > Bugs: CLOUDSTACK-4967 > https://issues.apache.org/jira/browse/CLOUDSTACK-4967 > > > Repository: cloudstack-docs > > > Description > --- > > This patch updates CloudStack Plugin Guide for the VXLAN Plugin. > > - Add a section to describe VNIs allocation matter > - Remove section that explain how to configure the bridge for traffic label > because the bridge configuration is no longer necessary. > - Update screen shots to catch up new UI visual appearance. > > > Diffs > - > > vxlan/en-US/images/vxlan-physicalnetwork.png > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > vxlan/en-US/images/vxlan-trafficlabel.png > 956d5f4f58f592ed3f260d6e1982c238c2ddf06a > vxlan/en-US/images/vxlan-vniconfig.png PRE-CREATION > vxlan/en-US/plugin-vxlan-config-hypervisor.xml 2c5e138 > vxlan/en-US/plugin-vxlan-config-management.xml 21f5461 > vxlan/en-US/plugin-vxlan-requirements.xml c2e04a6 > vxlan/en-US/plugin-vxlan-revision-history.xml ec04d11 > > Diff: https://reviews.apache.org/r/15247/diff/ > > > Testing > --- > > I generated and confirmed the PDF. > > > Thanks, > > Yoshikazu Nojima > >
Re: Review Request 15247: [DOC] CLOUDSTACK-4967: Add a section to describe VNI allocation matter
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15247/#review32625 --- Ship it! Ship It! - Toshiaki Hatano On Jan. 22, 2014, 5:44 a.m., Yoshikazu Nojima wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/15247/ > --- > > (Updated Jan. 22, 2014, 5:44 a.m.) > > > Review request for cloudstack, Marcus Sorensen and Toshiaki Hatano. > > > Bugs: CLOUDSTACK-4967 > https://issues.apache.org/jira/browse/CLOUDSTACK-4967 > > > Repository: cloudstack-docs > > > Description > --- > > This patch updates CloudStack Plugin Guide for the VXLAN Plugin. > > - Add a section to describe VNIs allocation matter > - Remove section that explain how to configure the bridge for traffic label > because the bridge configuration is no longer necessary. > - Update screen shots to catch up new UI visual appearance. > > > Diffs > - > > vxlan/en-US/images/vxlan-physicalnetwork.png > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > vxlan/en-US/images/vxlan-trafficlabel.png > 956d5f4f58f592ed3f260d6e1982c238c2ddf06a > vxlan/en-US/images/vxlan-vniconfig.png PRE-CREATION > vxlan/en-US/plugin-vxlan-config-hypervisor.xml 2c5e138 > vxlan/en-US/plugin-vxlan-config-management.xml 21f5461 > vxlan/en-US/plugin-vxlan-requirements.xml c2e04a6 > vxlan/en-US/plugin-vxlan-revision-history.xml ec04d11 > > Diff: https://reviews.apache.org/r/15247/diff/ > > > Testing > --- > > I generated and confirmed the PDF. > > > Thanks, > > Yoshikazu Nojima > >
Re: VXLAN problems
Hi Nux, Marcus, It's FYI: There's compiled vxlan document in Jenkins. http://jenkins.buildacloud.org/job/build-docs-vxlan-master/ -- Toshiaki Hatano 2014/1/20 Nux! > On 19.01.2014 14:37, Marcus Sorensen wrote: > >> You probably need an IP address on the physical network that you're >> running vxlan on, since this host needs to communicate with other >> hosts to send vxlan packets. Where prevously you could just bring up a >> bridge on an ethernet interface with no config, you can put all of >> your hosts on one untagged vlan, give them all ip addresses, and then >> the vxlan networks will flow over that. You can also use your >> physical interface name for the traffic label, if you have no other >> need for that bridge. >> >> It looks like there's been a bit of documentation checked in for 4.3, >> so there should be at least some published along with the release of >> the feature, and here are some links to the design docs and such. I >> haven't reviewed the docs in detail. >> >> http://www.slideshare.net/haeenajp/asfccc2013-toshiaki-release >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/ >> Linux+native+VXLAN+support+on+KVM+hypervisor >> > > Thanks a lot, Marcus. I did not expect I needed an IP on that interface, > too used with the old vlans. > I'll try to fix my setup. > > > Lucian > > -- > Sent from the Delta quadrant using Borg technology! > > Nux! > www.nux.ro >
Re: Review Request 15247: [DOC] CLOUDSTACK-4967: Add a section to describe VNI allocation matter
> On Nov. 7, 2013, 4:32 p.m., Toshiaki Hatano wrote: > > This modification basically subtract contents from the doc but the patch > > actually add a option to use physical interface name for traffic label. > > Since we still allow user to use bridge (even if it's not optimal) as > > traffic label, why don't we add more documentation to tell options and > > pros/cons instead of concealing things. Or, just make bridge-named traffic label invalid for vxlan isolation and write so. (it's better option for users, might be) - Toshiaki --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15247/#review28374 --- On Nov. 5, 2013, 9:31 p.m., Yoshikazu Nojima wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/15247/ > --- > > (Updated Nov. 5, 2013, 9:31 p.m.) > > > Review request for cloudstack and Marcus Sorensen. > > > Bugs: CLOUDSTACK-4967 > https://issues.apache.org/jira/browse/CLOUDSTACK-4967 > > > Repository: cloudstack-docs > > > Description > --- > > This patch updates CloudStack Plugin Guide for the VXLAN Plugin. > > - Add a section to describe VNIs allocation matter > - Remove section that explain how to configure the bridge for traffic label > because the bridge configuration is no longer necessary. > - Update screen shots to catch up new UI visual appearance. > > > Diffs > - > > vxlan/en-US/images/vxlan-trafficlabel.png > 956d5f4f58f592ed3f260d6e1982c238c2ddf06a > vxlan/en-US/images/vxlan-vniconfig.png PRE-CREATION > vxlan/en-US/plugin-vxlan-config-hypervisor.xml 2c5e138 > vxlan/en-US/plugin-vxlan-config-management.xml 21f5461 > vxlan/en-US/plugin-vxlan-revision-history.xml ec04d11 > > Diff: https://reviews.apache.org/r/15247/diff/ > > > Testing > --- > > I generated and confirmed the PDF. > > > Thanks, > > Yoshikazu Nojima > >
Re: Review Request 15247: [DOC] CLOUDSTACK-4967: Add a section to describe VNI allocation matter
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15247/#review28374 --- This modification basically subtract contents from the doc but the patch actually add a option to use physical interface name for traffic label. Since we still allow user to use bridge (even if it's not optimal) as traffic label, why don't we add more documentation to tell options and pros/cons instead of concealing things. vxlan/en-US/plugin-vxlan-config-hypervisor.xml <https://reviews.apache.org/r/15247/#comment55191> s/XVLAN/VXLAN/ ? :) vxlan/en-US/plugin-vxlan-config-hypervisor.xml <https://reviews.apache.org/r/15247/#comment55194> This paragraph excluding line 32 is still valid after the patch. IMO it's better to explain why we need to open UDP/8472 port. I also feel it's must to explain this plugin require multicast reachability between underlying interfaces, at least somewhere in this document. - Toshiaki Hatano On Nov. 5, 2013, 9:31 p.m., Yoshikazu Nojima wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/15247/ > --- > > (Updated Nov. 5, 2013, 9:31 p.m.) > > > Review request for cloudstack and Marcus Sorensen. > > > Bugs: CLOUDSTACK-4967 > https://issues.apache.org/jira/browse/CLOUDSTACK-4967 > > > Repository: cloudstack-docs > > > Description > --- > > This patch updates CloudStack Plugin Guide for the VXLAN Plugin. > > - Add a section to describe VNIs allocation matter > - Remove section that explain how to configure the bridge for traffic label > because the bridge configuration is no longer necessary. > - Update screen shots to catch up new UI visual appearance. > > > Diffs > - > > vxlan/en-US/images/vxlan-trafficlabel.png > 956d5f4f58f592ed3f260d6e1982c238c2ddf06a > vxlan/en-US/images/vxlan-vniconfig.png PRE-CREATION > vxlan/en-US/plugin-vxlan-config-hypervisor.xml 2c5e138 > vxlan/en-US/plugin-vxlan-config-management.xml 21f5461 > vxlan/en-US/plugin-vxlan-revision-history.xml ec04d11 > > Diff: https://reviews.apache.org/r/15247/diff/ > > > Testing > --- > > I generated and confirmed the PDF. > > > Thanks, > > Yoshikazu Nojima > >
Re: Review Request 15068: Change labels for VLAN to vNet
> On Nov. 1, 2013, 1:28 a.m., Toshiaki Hatano wrote: > > I suppose this should be reviewed by UI devs too. > > Could you add UI devs in reviewers? Other than that, this patch looks OK for me. But this patch makes changes in UI, so I suppose it's better to check with UI ppl. - Toshiaki --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15068/#review27992 --- On Oct. 30, 2013, 8:46 p.m., Chris Cameron wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/15068/ > --- > > (Updated Oct. 30, 2013, 8:46 p.m.) > > > Review request for cloudstack and Toshiaki Hatano. > > > Repository: cloudstack-git > > > Description > --- > > We would like to change the labels for VLAN to vNet to make the term more > generic for VXLAN and VLAN. This relates to the work being done to add in > VXLAN support to Cloudstack. > > > Diffs > - > > client/WEB-INF/classes/resources/messages.properties 3210aca > ui/dictionary.jsp 35cba22 > ui/scripts/network.js 12e5389 > ui/scripts/system.js 479883c > > Diff: https://reviews.apache.org/r/15068/diff/ > > > Testing > --- > > > Thanks, > > Chris Cameron > >
Re: Review Request 15068: Change labels for VLAN to vNet
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15068/#review27992 --- I suppose this should be reviewed by UI devs too. Could you add UI devs in reviewers? - Toshiaki Hatano On Oct. 30, 2013, 8:46 p.m., Chris Cameron wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/15068/ > --- > > (Updated Oct. 30, 2013, 8:46 p.m.) > > > Review request for cloudstack and Toshiaki Hatano. > > > Repository: cloudstack-git > > > Description > --- > > We would like to change the labels for VLAN to vNet to make the term more > generic for VXLAN and VLAN. This relates to the work being done to add in > VXLAN support to Cloudstack. > > > Diffs > - > > client/WEB-INF/classes/resources/messages.properties 3210aca > ui/dictionary.jsp 35cba22 > ui/scripts/network.js 12e5389 > ui/scripts/system.js 479883c > > Diff: https://reviews.apache.org/r/15068/diff/ > > > Testing > --- > > > Thanks, > > Chris Cameron > >
Re: Review Request 14868: CLOUDSTACK-4932: Bugfix: listNetworks API doesn't return vlan id / VNI ( Virtual Network Identifier in VXLAN )
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14868/#review27780 --- Ship it! Ship It! - Toshiaki Hatano On Oct. 23, 2013, 6:38 a.m., Yoshikazu Nojima wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/14868/ > --- > > (Updated Oct. 23, 2013, 6:38 a.m.) > > > Review request for cloudstack, Alena Prokharchyk, Chiradeep Vittal, Murali > Reddy, Hugo Trippaers, and Toshiaki Hatano. > > > Bugs: CLOUDSTACK-4932 > https://issues.apache.org/jira/browse/CLOUDSTACK-4932 > > > Repository: cloudstack-git > > > Description > --- > > Issue to resolve: > When listNetworks API is called with root admin account, it must return vlan > id, but it returns "N/A". > I tested with the branch "master". > > http://cloudstack.apache.org/docs/api/apidocs-4.2/root_admin/listNetworks.html > > This patch fixed comparison bug of BroadCastDomainType instance to make this > api return vlan id, and made this api returns VNI ( Virtual Network > Identifier in VXLAN ) when network is configured to use VXLAN. > > > Diffs > - > > server/src/com/cloud/api/ApiResponseHelper.java f4ca112 > > Diff: https://reviews.apache.org/r/14868/diff/ > > > Testing > --- > > I confirmed appropriate VLAN ID is returned by listNetworks API, and it is > displayed in the "Network"/"Guest networks"/"details" tab in CloudStack > WebGUI. > > > Thanks, > > Yoshikazu Nojima > >
Re: Review Request 15001: CLOUDSTACK-4967: Bugfix: Guest network isn't created correctly in rare case when vxlan is used for isolation method
> On Oct. 29, 2013, 4:35 a.m., Toshiaki Hatano wrote: > > plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/BridgeVifDriver.java, > > line 158 > > <https://reviews.apache.org/r/15001/diff/2/?file=372880#file372880line158> > > > > If we checked length here, generated brName should be checked, not a > > pifName. > > > > We shouldn't block valid brName by this check, e.g. eth0.1000-9000. > > Toshiaki Hatano wrote: > s/eth0.1000-9000/breth0.100-9000/ > > Yoshikazu Nojima wrote: > It validates pifName, not brName. It validates pifName in the point of > whether it is appropriate to be used for generating bridge name here. My apology if it was lack of talk. What I trid to mean is, brName like "breth0.100-9000" are still valid bridge name despite pifName longer than 5, thus this check is too strong. - Toshiaki --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15001/#review27657 --- On Oct. 29, 2013, 12:51 a.m., Yoshikazu Nojima wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/15001/ > --- > > (Updated Oct. 29, 2013, 12:51 a.m.) > > > Review request for cloudstack, Marcus Sorensen and Toshiaki Hatano. > > > Bugs: CLOUDSTACK-4967 > https://issues.apache.org/jira/browse/CLOUDSTACK-4967 > > > Repository: cloudstack-git > > > Description > --- > > Guest network isn't created correctly when vxlan is used for isolation > method, VNI is bigger than 1000, and physical network interface name > length is longer than 4. > This patch fix this issue by raising error when long interface name is > specified. > > Issue remained: document that physical interface name length must be shorter > than 5. > > > Diffs > - > > > plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/BridgeVifDriver.java > f945b74 > > plugins/hypervisors/kvm/test/com/cloud/hypervisor/kvm/resource/BridgeVifDriverTest.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/15001/diff/ > > > Testing > --- > > I confirmed it passes the unit tests I add. > > > Thanks, > > Yoshikazu Nojima > >
Re: Review Request 15001: CLOUDSTACK-4967: Bugfix: Guest network isn't created correctly in rare case when vxlan is used for isolation method
> On Oct. 29, 2013, 4:35 a.m., Toshiaki Hatano wrote: > > > > Toshiaki Hatano wrote: > Actual issue is not "the physical interface name length must be shorter > than 5" but "generated bridge name cannot exceed 15 character". > > > Yoshikazu Nojima wrote: > If we check generated bridge name length, users will not face the > exception before their cloudstack environment grow up. They will face the > exception in production environment suddenly. > If we check physical interface name length, users can face the exception > whenever bridge name length is not enough. > That's why we would note in document that such a situation can be happened when VXLAN isolated zone grows up, isn't it? It's too much to completely disable VXLAN isolation for pifName >=5, especially when we won't provide any workaround. IMO when we try to introduce fool proof mechanism, like this patch, we first try to avoid creating restriction in non-fools' usage. Let's check Marcus's patch. - Toshiaki --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15001/#review27657 --- On Oct. 29, 2013, 12:51 a.m., Yoshikazu Nojima wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/15001/ > ------- > > (Updated Oct. 29, 2013, 12:51 a.m.) > > > Review request for cloudstack, Marcus Sorensen and Toshiaki Hatano. > > > Bugs: CLOUDSTACK-4967 > https://issues.apache.org/jira/browse/CLOUDSTACK-4967 > > > Repository: cloudstack-git > > > Description > --- > > Guest network isn't created correctly when vxlan is used for isolation > method, VNI is bigger than 1000, and physical network interface name > length is longer than 4. > This patch fix this issue by raising error when long interface name is > specified. > > Issue remained: document that physical interface name length must be shorter > than 5. > > > Diffs > - > > > plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/BridgeVifDriver.java > f945b74 > > plugins/hypervisors/kvm/test/com/cloud/hypervisor/kvm/resource/BridgeVifDriverTest.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/15001/diff/ > > > Testing > --- > > I confirmed it passes the unit tests I add. > > > Thanks, > > Yoshikazu Nojima > >
Re: Review Request 15012: CLOUDSTACK-4984: decrement MAX_VXLAN_VNI to be aligned with Linux kernel
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15012/#review27660 --- Ship it! Ship It! - Toshiaki Hatano On Oct. 29, 2013, 5 a.m., Yoshikazu Nojima wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/15012/ > --- > > (Updated Oct. 29, 2013, 5 a.m.) > > > Review request for cloudstack, Marcus Sorensen and Toshiaki Hatano. > > > Repository: cloudstack-git > > > Description > --- > > Linux vxlan interface doesn't accept VNI:16777215. > https://github.com/torvalds/linux/blob/master/drivers/net/vxlan.c#L2140 > > As far as I read internet draft ( > http://tools.ietf.org/html/draft-mahalingam-dutt-dcops-vxlan-05 ), > 16777215 is a valid VNI, but it cannot be used in Linux now. > > This patch decrements MAX_VXLAN_VNI to 16777214. > > > Diffs > - > > server/src/com/cloud/network/NetworkServiceImpl.java 61c070a > > Diff: https://reviews.apache.org/r/15012/diff/ > > > Testing > --- > > I confirmed VNI validation works by specifying 16777215-16777215 to id range > via add zone wizard. > > > Thanks, > > Yoshikazu Nojima > >
Re: Review Request 15001: CLOUDSTACK-4967: Bugfix: Guest network isn't created correctly in rare case when vxlan is used for isolation method
> On Oct. 29, 2013, 4:35 a.m., Toshiaki Hatano wrote: > > Actual issue is not "the physical interface name length must be shorter than 5" but "generated bridge name cannot exceed 15 character". > On Oct. 29, 2013, 4:35 a.m., Toshiaki Hatano wrote: > > plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/BridgeVifDriver.java, > > line 158 > > <https://reviews.apache.org/r/15001/diff/2/?file=372880#file372880line158> > > > > If we checked length here, generated brName should be checked, not a > > pifName. > > > > We shouldn't block valid brName by this check, e.g. eth0.1000-9000. s/eth0.1000-9000/breth0.100-9000/ - Toshiaki --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15001/#review27657 --- On Oct. 29, 2013, 12:51 a.m., Yoshikazu Nojima wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/15001/ > --- > > (Updated Oct. 29, 2013, 12:51 a.m.) > > > Review request for cloudstack, Marcus Sorensen and Toshiaki Hatano. > > > Bugs: CLOUDSTACK-4967 > https://issues.apache.org/jira/browse/CLOUDSTACK-4967 > > > Repository: cloudstack-git > > > Description > --- > > Guest network isn't created correctly when vxlan is used for isolation > method, VNI is bigger than 1000, and physical network interface name > length is longer than 4. > This patch fix this issue by raising error when long interface name is > specified. > > Issue remained: document that physical interface name length must be shorter > than 5. > > > Diffs > - > > > plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/BridgeVifDriver.java > f945b74 > > plugins/hypervisors/kvm/test/com/cloud/hypervisor/kvm/resource/BridgeVifDriverTest.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/15001/diff/ > > > Testing > --- > > I confirmed it passes the unit tests I add. > > > Thanks, > > Yoshikazu Nojima > >
Re: Review Request 15001: CLOUDSTACK-4967: Bugfix: Guest network isn't created correctly in rare case when vxlan is used for isolation method
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15001/#review27657 --- plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/BridgeVifDriver.java <https://reviews.apache.org/r/15001/#comment53719> If we checked length here, generated brName should be checked, not a pifName. We shouldn't block valid brName by this check, e.g. eth0.1000-9000. - Toshiaki Hatano On Oct. 29, 2013, 12:51 a.m., Yoshikazu Nojima wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/15001/ > --- > > (Updated Oct. 29, 2013, 12:51 a.m.) > > > Review request for cloudstack, Marcus Sorensen and Toshiaki Hatano. > > > Bugs: CLOUDSTACK-4967 > https://issues.apache.org/jira/browse/CLOUDSTACK-4967 > > > Repository: cloudstack-git > > > Description > --- > > Guest network isn't created correctly when vxlan is used for isolation > method, VNI is bigger than 1000, and physical network interface name > length is longer than 4. > This patch fix this issue by raising error when long interface name is > specified. > > Issue remained: document that physical interface name length must be shorter > than 5. > > > Diffs > - > > > plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/BridgeVifDriver.java > f945b74 > > plugins/hypervisors/kvm/test/com/cloud/hypervisor/kvm/resource/BridgeVifDriverTest.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/15001/diff/ > > > Testing > --- > > I confirmed it passes the unit tests I add. > > > Thanks, > > Yoshikazu Nojima > >
Re: Review Request 12623: CLOUDSTACK-2328: Linux native VXLAN support on KVM hypervisor
ge/vxlan device deleted on the host (B) 8. Migrate VM from (C) to empty host (A) 9. Confirm it's pingable from VM to public network #) Test case 5: plug/unplug Nic 1. Add an instance from UI, create network during wizard. 2. Create additional network 3. Add NIC for network created in step 2 to the VM 4. Confirm it's pingable from VM to public network by using both side of NICs 5. Delete NIC created in step 3 6. Confirm bridge/vxlan device deleted on the host Thanks, Toshiaki Hatano
RE: MS fails to start due - Incorrect code version
It looks chicken and egg situation. Bean injection requires 4.3 upgraded database. DB upgrade process executed by DatabaseUpgradeChecker runs after bean injection. At least I found a workaround, upgrade DB by hand like below. $ cloudstack-setup-databases cloud:cloud@localhost --deploy-as root $ cloudstack-setup-management $ service cloudstack-management stop $ mysql -u root cloud < setup/db/db/schema-40to410.sql $ mysql -u root cloud < setup/db/db/schema-40to410-cleanup.sql $ mysql -u root cloud < setup/db/db/schema-410to420.sql $ mysql -u root cloud < setup/db/db/schema-410to420-cleanup.sql $ mysql -u root cloud < setup/db/db/schema-420to430.sql $ mysql -u root cloud < setup/db/db/schema-420to430-cleanup.sql $ mysql -u root cloud -e 'UPDATE version SET version="4.3.0";' $ service cloudstack-management start I hope it helps. Thanks, -- Toshiaki -----Original Message- From: Toshiaki Hatano [mailto:toshiaki.hat...@verio.net] Sent: Tuesday, August 20, 2013 15:59 To: dev@cloudstack.apache.org Subject: RE: MS fails to start due - Incorrect code version I'm on 61c5b4bf7520080d88ae5a9aad38dee7e0348fa2 and got same error. I've wiped and redeployed DB by using 'cloud-setup-database' command, then I confirmed there're no 'default_value' column in table 'configuration'. The column should be created by schema-420to430.sql. So, for some reason, the SQL isn't executed by 'cloud-setup-database'. -- Toshiaki -Original Message- From: SuichII, Christopher [mailto:chris.su...@netapp.com] Sent: Tuesday, August 20, 2013 09:00 To: Subject: Re: MS fails to start due - Incorrect code version I was on cc18ca79fc6d58fb639ffbb455791caeb021589a and tried rolling back to a commit I have been able to deploy with before (b727001f483012012c061e8c352c1ebfe7d3fecd) and got the same result. On Aug 20, 2013, at 10:43 AM, Prasanna Santhanam wrote: > What's your HEAD? I'm on 3a29c734475184cf28135acaca271fea1c90554a and > don't see a problm starting up the server. I haven't configured my > zones/pods etc. Also I started with a fresh DB. > > On Tue, Aug 20, 2013 at 01:47:50PM +, SuichII, Christopher wrote: >> Master. >> >> Then, after commenting out the offending code, rebuilding, redeploying the >> db and rerunning, this exception is thrown when attempting to start the MS: >> >> ERROR [o.s.w.c.ContextLoader] (main:null) Context initialization >> failed >> org.springframework.beans.factory.BeanCreationException: Error creating bean >> with name 'dataMotionServiceImpl': Injection of autowired dependencies >> failed; nested exception is >> org.springframework.beans.factory.BeanCreationException: Could not autowire >> field: java.util.List >> org.apache.cloudstack.storage.motion.DataMotionServiceImpl.strategies; >> nested exception is org.springframework.beans.factory.BeanCreationException: >> Error creating bean with name 'ancientDataMotionStrategy': Injection of >> autowired dependencies failed; nested exception is >> org.springframework.beans.factory.BeanCreationException: Could not autowire >> field: >> org.apache.cloudstack.engine.subsystem.api.storage.StorageCacheManager >> org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.cacheMgr; >> nested exception is org.springframework.beans.factory.BeanCreationException: >> Error creating bean with name 'storageCacheManagerImpl': Injection of >> autowired dependencies failed; nested exception is >> org.springframework.beans.factory.BeanCreationException: Could not autowire >> field: >> org.apache.cloudstack.storage.cache.manager.StorageCacheReplacementAlgorithm >> org.apache.cloudstack.storage.cache.manager.StorageCacheManagerImpl.cacheReplacementAlgorithm; >> nested exception is >> org.springframework.beans.factory.BeanCreationException: Error creating bean >> with name 'StorageCacheReplacementAlgorithm': Invocation of init method >> failed; nested exception is com.cloud.utils.exception.CloudRuntimeException: >> DB Exception on: com.mysql.jdbc.JDBC4PreparedStatement@29f9aaf0: SELECT >> configuration.instance, configuration.component, configuration.name, >> configuration.value, configuration.default_value, configuration.description, >> configuration.category, configuration.is_dynamic, configuration.scope, >> configuration.updated FROM configuration WHERE configuration.name = >> _binary'storage.cache.replacement.lru.interval' ORDER BY RAND() LIMIT 1 >> at >> org.springframework.beans.factory.annotation.Auto
RE: MS fails to start due - Incorrect code version
I'm on 61c5b4bf7520080d88ae5a9aad38dee7e0348fa2 and got same error. I've wiped and redeployed DB by using 'cloud-setup-database' command, then I confirmed there're no 'default_value' column in table 'configuration'. The column should be created by schema-420to430.sql. So, for some reason, the SQL isn't executed by 'cloud-setup-database'. -- Toshiaki -Original Message- From: SuichII, Christopher [mailto:chris.su...@netapp.com] Sent: Tuesday, August 20, 2013 09:00 To: Subject: Re: MS fails to start due - Incorrect code version I was on cc18ca79fc6d58fb639ffbb455791caeb021589a and tried rolling back to a commit I have been able to deploy with before (b727001f483012012c061e8c352c1ebfe7d3fecd) and got the same result. On Aug 20, 2013, at 10:43 AM, Prasanna Santhanam wrote: > What's your HEAD? I'm on 3a29c734475184cf28135acaca271fea1c90554a and > don't see a problm starting up the server. I haven't configured my > zones/pods etc. Also I started with a fresh DB. > > On Tue, Aug 20, 2013 at 01:47:50PM +, SuichII, Christopher wrote: >> Master. >> >> Then, after commenting out the offending code, rebuilding, redeploying the >> db and rerunning, this exception is thrown when attempting to start the MS: >> >> ERROR [o.s.w.c.ContextLoader] (main:null) Context initialization >> failed >> org.springframework.beans.factory.BeanCreationException: Error creating bean >> with name 'dataMotionServiceImpl': Injection of autowired dependencies >> failed; nested exception is >> org.springframework.beans.factory.BeanCreationException: Could not autowire >> field: java.util.List >> org.apache.cloudstack.storage.motion.DataMotionServiceImpl.strategies; >> nested exception is org.springframework.beans.factory.BeanCreationException: >> Error creating bean with name 'ancientDataMotionStrategy': Injection of >> autowired dependencies failed; nested exception is >> org.springframework.beans.factory.BeanCreationException: Could not autowire >> field: >> org.apache.cloudstack.engine.subsystem.api.storage.StorageCacheManager >> org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.cacheMgr; >> nested exception is org.springframework.beans.factory.BeanCreationException: >> Error creating bean with name 'storageCacheManagerImpl': Injection of >> autowired dependencies failed; nested exception is >> org.springframework.beans.factory.BeanCreationException: Could not autowire >> field: >> org.apache.cloudstack.storage.cache.manager.StorageCacheReplacementAlgorithm >> org.apache.cloudstack.storage.cache.manager.StorageCacheManagerImpl.cacheReplacementAlgorithm; >> nested exception is >> org.springframework.beans.factory.BeanCreationException: Error creating bean >> with name 'StorageCacheReplacementAlgorithm': Invocation of init method >> failed; nested exception is com.cloud.utils.exception.CloudRuntimeException: >> DB Exception on: com.mysql.jdbc.JDBC4PreparedStatement@29f9aaf0: SELECT >> configuration.instance, configuration.component, configuration.name, >> configuration.value, configuration.default_value, configuration.description, >> configuration.category, configuration.is_dynamic, configuration.scope, >> configuration.updated FROM configuration WHERE configuration.name = >> _binary'storage.cache.replacement.lru.interval' ORDER BY RAND() LIMIT 1 >> at >> org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:287) >> at >> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1106) >> at >> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:517) >> at >> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456) >> at >> org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294) >> at >> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225) >> at >> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291) >> at >> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193) >> at >> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:284) >> at >> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193) >> at >> org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:609) >> at >> org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918) >> at >> org.sprin
RE: [ANNOUNCE] New Committer: Toshiaki Hatano
Thanks, Marcus, Dave, Chiba-san, Hugo, Sebastian, Nitin, Koushik, Jaypal! :) -- Toshiaki -Original Message- From: Jayapal Reddy Uradi [mailto:jayapalreddy.ur...@citrix.com] Sent: Tuesday, August 20, 2013 08:03 To: Subject: Re: [ANNOUNCE] New Committer: Toshiaki Hatano Congratulations! On 20-Aug-2013, at 1:06 PM, Hugo Trippaers wrote: > Congratulations :-) > > Sent from my iPhone > > On 20 aug. 2013, at 02:55, Go Chiba wrote: > >> Congrats Hatano-san! >> >> >> On Tue, Aug 20, 2013 at 7:56 AM, Marcus Sorensen wrote: >> >>> The Project Management Committee (PMC) for Apache CloudStack has >>> asked Toshiaki Hatano to become a committer and we are pleased to >>> announce that he has accepted. >>> >>> Being a committer enables easier contribution to the project since >>> there is no need to go via the patch submission process. This should >>> enable better productivity. >>> >>> Please join us in congratulating Toshiaki! >> >> >> >> -- >> 千葉 豪 Go Chiba >> E-mail:go.ch...@gmail.com This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio Inc. makes no warranty that this email is error or virus free. Thank you.
RE: Did we already started developing support VxLAN?
Hello Mr. 曹, I'm currently working on VXLAN specifically to KVM with Linux Bridge (not OVS) and Multicast mode. I think some network plugin also supports (or will support) VXLAN as tunneling protocol. For what environment (KVM, Xen, VMware (Nexus 1kV maybe?), etc.) and which type of VXLAN (Multicast mode/Unicast mode) are you planning to develop? Sincerely, -- Toshiaki -Original Message- From: David Nalley [mailto:da...@gnsa.us] Sent: 2013/08/12 (月) 10:40 To: dev@cloudstack.apache.org Subject: Re: Did we already started developing support VxLAN? On Mon, Aug 12, 2013 at 9:44 AM, 曹建林 wrote: > > Hi guys > > My company has a project to delivery support of VxLAN network , using > CloudStack. > I know the version 4.x does not support vxlan. > So I wonder what the plan to support vxlan network . > > We have a team about 5 guys in charge of developing the feature, > > So if there is anybody in this thing . please send me email , thank you very > much. > Toshiaki from Verio has been working on VXLAN in KVM. You can see his patch here: https://reviews.apache.org/r/12623/ https://issues.apache.org/jira/browse/CLOUDSTACK-2328 I'm sure he'd welcome help. --David This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio Inc. makes no warranty that this email is error or virus free. Thank you.
Re: Review Request 12849: added backwards compatibility code to Networks enums
> On Aug. 9, 2013, 5:23 p.m., Toshiaki Hatano wrote: > > api/src/com/cloud/network/Networks.java, line 172 > > <https://reviews.apache.org/r/12849/diff/9/?file=338752#file338752line172> > > > > Why don't we merge getValueFrom() and getValue()? > > > > public String getValue(URI uri) { > > if(uri.isOpaque()) { > > return uri.getSchemeSpecificPart(); > > } else { > > return uri.getHost(); > > } > > } > > Toshiaki Hatano wrote: > Or, maybe make getValueFrom private. > > I think it will make confuse if we provide 2 public method to get value > of URI. Sorry I missed the point, getValueFrom is static method. Please disregard and close this issue. - Toshiaki --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/12849/#review24925 --- On Aug. 8, 2013, 11:51 a.m., daan Hoogland wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/12849/ > --- > > (Updated Aug. 8, 2013, 11:51 a.m.) > > > Review request for cloudstack, Chiradeep Vittal, Dave Cahill, Koushik Das, > and Sheng Yang. > > > Repository: cloudstack-git > > > Description > --- > > Both BroadcastDomainType and IsolationType needed some extra code for > backwards compatibility. > > All over the code calls are done to URI.getHost() to retrieve ids of > broadcastdomains. These id obviously are not hosts so this call is confusing > and requires maintenance all over the code base. Also for different types the > value returned by getHost has different meaning. vlan://1 is id 1 of course > but others might be ranges of vlans or colon separated values. To make things > worse a NiciraNvp has an uri of the form lswitch: without the forward > slashes. > > To make the system more maintainable in this perspect the changes in this > patch were made. It is my intention to replace the calls to getHost by the > member call getValueFrom or the static method getValue in time. In this way > maintenance is centralized and an overview of differnces and quirks is easily > found > > > Diffs > - > > api/src/com/cloud/network/Networks.java c76c3d4 > api/test/com/cloud/network/NetworksTest.java 31114e8 > > Diff: https://reviews.apache.org/r/12849/diff/ > > > Testing > --- > > unit tests for different kind of BroadcastDomainType.values > > > Thanks, > > daan Hoogland > >
Re: Review Request 12849: added backwards compatibility code to Networks enums
> On Aug. 9, 2013, 5:23 p.m., Toshiaki Hatano wrote: > > api/src/com/cloud/network/Networks.java, line 172 > > <https://reviews.apache.org/r/12849/diff/9/?file=338752#file338752line172> > > > > Why don't we merge getValueFrom() and getValue()? > > > > public String getValue(URI uri) { > > if(uri.isOpaque()) { > > return uri.getSchemeSpecificPart(); > > } else { > > return uri.getHost(); > > } > > } Or, maybe make getValueFrom private. I think it will make confuse if we provide 2 public method to get value of URI. - Toshiaki --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/12849/#review24925 --- On Aug. 8, 2013, 11:51 a.m., daan Hoogland wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/12849/ > --- > > (Updated Aug. 8, 2013, 11:51 a.m.) > > > Review request for cloudstack, Chiradeep Vittal, Dave Cahill, Koushik Das, > and Sheng Yang. > > > Repository: cloudstack-git > > > Description > --- > > Both BroadcastDomainType and IsolationType needed some extra code for > backwards compatibility. > > All over the code calls are done to URI.getHost() to retrieve ids of > broadcastdomains. These id obviously are not hosts so this call is confusing > and requires maintenance all over the code base. Also for different types the > value returned by getHost has different meaning. vlan://1 is id 1 of course > but others might be ranges of vlans or colon separated values. To make things > worse a NiciraNvp has an uri of the form lswitch: without the forward > slashes. > > To make the system more maintainable in this perspect the changes in this > patch were made. It is my intention to replace the calls to getHost by the > member call getValueFrom or the static method getValue in time. In this way > maintenance is centralized and an overview of differnces and quirks is easily > found > > > Diffs > - > > api/src/com/cloud/network/Networks.java c76c3d4 > api/test/com/cloud/network/NetworksTest.java 31114e8 > > Diff: https://reviews.apache.org/r/12849/diff/ > > > Testing > --- > > unit tests for different kind of BroadcastDomainType.values > > > Thanks, > > daan Hoogland > >
Re: Review Request 12849: added backwards compatibility code to Networks enums
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/12849/#review24925 --- api/src/com/cloud/network/Networks.java <https://reviews.apache.org/r/12849/#comment49071> Why don't we merge getValueFrom() and getValue()? public String getValue(URI uri) { if(uri.isOpaque()) { return uri.getSchemeSpecificPart(); } else { return uri.getHost(); } } - Toshiaki Hatano On Aug. 8, 2013, 11:51 a.m., daan Hoogland wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/12849/ > --- > > (Updated Aug. 8, 2013, 11:51 a.m.) > > > Review request for cloudstack, Chiradeep Vittal, Dave Cahill, Koushik Das, > and Sheng Yang. > > > Repository: cloudstack-git > > > Description > --- > > Both BroadcastDomainType and IsolationType needed some extra code for > backwards compatibility. > > All over the code calls are done to URI.getHost() to retrieve ids of > broadcastdomains. These id obviously are not hosts so this call is confusing > and requires maintenance all over the code base. Also for different types the > value returned by getHost has different meaning. vlan://1 is id 1 of course > but others might be ranges of vlans or colon separated values. To make things > worse a NiciraNvp has an uri of the form lswitch: without the forward > slashes. > > To make the system more maintainable in this perspect the changes in this > patch were made. It is my intention to replace the calls to getHost by the > member call getValueFrom or the static method getValue in time. In this way > maintenance is centralized and an overview of differnces and quirks is easily > found > > > Diffs > - > > api/src/com/cloud/network/Networks.java c76c3d4 > api/test/com/cloud/network/NetworksTest.java 31114e8 > > Diff: https://reviews.apache.org/r/12849/diff/ > > > Testing > --- > > unit tests for different kind of BroadcastDomainType.values > > > Thanks, > > daan Hoogland > >
RE: [DISCUSS] [jira] make affectedVersion field mandatory.....
+1 Thanks, -- Toshiaki -Original Message- From: Koushik Das [mailto:koushik@citrix.com] Sent: Friday, August 02, 2013 05:25 To: dev@cloudstack.apache.org Subject: RE: [DISCUSS] [jira] make affectedVersion field mandatory. +1 > -Original Message- > From: Ram Ganesh [mailto:ram.gan...@citrix.com] > Sent: Friday, August 02, 2013 4:38 PM > To: dev@cloudstack.apache.org > Subject: [DISCUSS] [jira] make affectedVersion field mandatory. > > Hi, > > While triaging bugs I noticed that many bugs had affectedVersion field > as empty. This makes it difficult to guess the version/release the > reporter was on while filing the bug. Can we make the affectedVersion > field a mandatory field instead? > > Thanks, > RamG This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio Inc. makes no warranty that this email is error or virus free. Thank you.
Re: Review Request 13004: Bug:advance zone, create public Network with vlan id specified, but the portgroup is cloud.public.untagged.0.1-vSwitch0 not cloud.public.[vlanid].0.1-vSwitch0
> On July 30, 2013, 11:13 p.m., edison su wrote: > > I looked at latest 4.2 code, toURI() only contains one line code: return > > new URI(scheme + "://" + value.toString());, thus can't apply your patch. > > Could you tell me which branch are you using? The master? > > Jijun wrote: > hi edison, > yes,it is in the master,not the branch 4.2. > > edison su wrote: > Do you who made the change on master branch? 4.2 branch doesn't have this > issue. It suppose duplicate of patch https://reviews.apache.org/r/12849/ and https://reviews.apache.org/r/12985/. If issue CLOUDSTACK-3883 is solved, CLOUDSTACK-3682 should be solved too. - Toshiaki --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/13004/#review24296 --- On July 30, 2013, 12:58 a.m., Jijun wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/13004/ > --- > > (Updated July 30, 2013, 12:58 a.m.) > > > Review request for cloudstack, daan Hoogland, edison su, and Wei Zhou. > > > Bugs: https://issues.apache.org/jira/browse/CLOUDSTACK-3883 > > > Repository: cloudstack-git > > > Description > --- > > In lastest CS 4.2 code, I create advance zone with hypervisor vmware esxi, > create public Network with vlan id 509 specified and label vSwitch0, when cs > create systemvm(cpvm,ssvm), > a new portgroup with name cloud.public.untagged.0.1-vSwitch0 was created, > not a portgroup cloud.public.509.0.1-vSwitch0 as expected. > in database table nics, the field broadcast_uri for new systemvm is vlan:509 > , and should be vlan://509 > debug the code and found it is a syntax error in Networks.java for new > instance java.net.URI. > > > Diffs > - > > api/src/com/cloud/network/Networks.java c76c3d4 > > Diff: https://reviews.apache.org/r/13004/diff/ > > > Testing > --- > > recreate the system vm or create a new advance zone , public network with vlan > > > Thanks, > > Jijun > >
Review Request 13088: CLOUDSTACK-3959: [KVM] agent setup failed when physical interface name is in pXpY format
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/13088/ --- Review request for cloudstack, edison su and Wido den Hollander. Bugs: https://issues.apache.org/jira/browse/CLOUDSTACK-3959 Repository: cloudstack-git Description --- CloudStack failed to add kvm host when 'pXpY' name format (udev named) physical interface is a member of the bridge (kvm traffic label) on the host. Root cause is in com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.matchPifFileInDirectory(String). There're no handling code for 'pXpY' format interface, so I added that. Diffs - plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java 571bcc8 Diff: https://reviews.apache.org/r/13088/diff/ Testing --- Tested locally, advanced zone with vlan isolation, kvm hypervisor (ubuntu 13.04). Thanks, Toshiaki Hatano
RE: [DISCUSS] Compatibility issue between network plugins and hypervisors
Daan, In my opinion, it's responsibility of the developer to provide information and to stop Ops from failing over cliff. Obviously Dev knows how code works (sometime not :), but Ops doesn't. So, if Dev deny to help Ops, how can Ops setup the cloud in proper way? When someone enhance hypervisor and/or network implementations, they should conduct test of their enhancement before submitting patch. Then, they may be able to notice that the checks are not up to date. Even if they didn't notice that, someone who try to use the feature should notice that. Do you think this model will not work? Thanks, -- Toshiaki -Original Message- From: Daan Hoogland [mailto:daan.hoogl...@gmail.com] Sent: Saturday, July 27, 2013 05:06 To: dev Subject: Re: [DISCUSS] Compatibility issue between network plugins and hypervisors H, isn't it the responsibility of the administrator to setup the cloud in a proper way? hypervisor and network implementations may enhance their capabilities at minor upgrades so it will not be easy to keep checks on this up to date in cloudstack. Am I missing the point here? regards, Daan On Sat, Jul 27, 2013 at 1:10 AM, Toshiaki Hatano wrote: > I agree with Murali. > > I feel NetworkGuru should know their capability and should called when > we add cluster. > NetworkGurus already provide canHandle(NetworkOffering, NetworkType, > PhysicalNetwork) method to check their capability. > So, how about overloading this method to get HypervisorType in arguments? > > Thanks, > -- > Toshiaki > > -Original Message- > From: Murali Reddy [mailto:murali.re...@citrix.com] > Sent: Thursday, July 25, 2013 22:33 > To: dev@cloudstack.apache.org > Subject: Re: [DISCUSS] Compatibility issue between network plugins and > hypervisors > > Also, should not we treat 'isolation' as Network Element capability > rather than Hypervisor. Tunnelling capability could be a Hypervisor > capability, but isolation (STT/GRE) is Network Element capability? > So,zone isolation > -> isolation provider -> supported hypervisors should be checked > -> against > add cluster IMO. > > On 26/07/13 9:24 AM, "Chiradeep Vittal" > wrote: > > >+1 (with a caveat), good idea since isolation method is supported on > >+a > >per-zone basis. > >The caveat is that sometimes it makes sense to support multiple > >isolation methods in a zone. > >For example, VPC(advanced) + basic in the same zone. > >Why would one do this? Simply because someone might start with one > >isolation method (basic) and then offer advanced (using overlays like > >VxLAN f.e). Since templates/snapshots/volumes tend to be > >zone-specific, this makes the transition easier. > >This is not unlike AWS "EC2-classic" and "VPC" in the same zone. > > > > > >On 7/26/13 3:34 AM, "Alex Huang" wrote: > > > >>+1 > >> > >>I think we should take advantage of hypervisor capabilities to look > >>for that compatibility. > >> > >>--Alex > >> > >>> -Original Message- > >>> From: Toshiaki Hatano [mailto:toshiaki.hat...@verio.net] > >>> Sent: Thursday, July 25, 2013 3:01 PM > >>> To: dev@cloudstack.apache.org > >>> Subject: [DISCUSS] Compatibility issue between network plugins and > >>> hypervisors > >>> > >>> Hi devs, > >>> > >>> > >>> > >>> CloudStack supports many hypervisors and many network isolation > >>>methods. > >>> > >>> Some isolation method doesn't (or cannot) support some > >>> hypervisors, > >>> > >>> but it looks cloudstack doesn't check compatibility between > >>>network isolation and hypervisors. > >>> > >>> > >>> > >>> Why don't we check it during addCluster, first timing cloudstack- > >>>management know isolation and hypervisor, and fail if it's > >>>incompatible? > >>> > >>> > >>> > >>> Best Regards, > >>> > >>> -- > >>> > >>> Toshiaki Hatano > >>> > >>> Verio, an NTT Communications company > >>> E-mail: toshiaki.hat...@verio.net > >>> <mailto:toshiaki.hat...@verio.net> > >>> > >>> AIM: toshiaki.hat...@verio.net <mailto:toshiaki.hat...@verio.net> > >>> > >>> > >>> > >>> > >>> > >>> This email message is intended for the use of the person to whom > &g
Re: Review Request 12849: added backwards compatibility code to Networks enums
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/12849/#review24161 --- I tested this patch locally, VLAN isolated advanced zone with 1 KVM hypervisor. Confirmed this patch resolve bug https://issues.apache.org/jira/browse/CLOUDSTACK-3682 Thanks! - Toshiaki Hatano On July 23, 2013, 10:02 a.m., daan Hoogland wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/12849/ > --- > > (Updated July 23, 2013, 10:02 a.m.) > > > Review request for cloudstack and Koushik Das. > > > Repository: cloudstack-git > > > Description > --- > > Both BroadcastDomainType and IsolationType needed some extra code for > backwards compatibility > > > Diffs > - > > api/src/com/cloud/network/Networks.java c76c3d4 > > server/src/com/cloud/network/router/VpcVirtualNetworkApplianceManagerImpl.java > 6fafa3e > > Diff: https://reviews.apache.org/r/12849/diff/ > > > Testing > --- > > > Thanks, > > daan Hoogland > >
Review Request 12985: CLOUDSTACK-3682: NPE in BridgeVifDriver causing systemvm startup failure in KVM
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/12985/ --- Review request for cloudstack, Alena Prokharchyk, Chiradeep Vittal, Murali Reddy, Hugo Trippaers, and Sheng Yang. Bugs: https://issues.apache.org/jira/browse/CLOUDSTACK-3682 Repository: cloudstack-git Description --- CloudStack-management suddenly start generating broadcastUri in 'vlan:' style instead of 'vlan://' style. That cause issue in KVM-agent VifDrivers (both Bridge and OVS), since they cannot purse different style Uri. For the compatibility to older version, CloudStack-management should generate 'vlan://' style BroadcastUri. Change the URI creation way in 2 method below to make sure they generate 'scheme://host' style BroadcastUri. - com.cloud.network.Networks.IsolationType.toUri(T) - com.cloud.network.Networks.BroadcastDomainType.toUri(T) Diffs - api/src/com/cloud/network/Networks.java c76c3d4 plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/BridgeVifDriver.java 195cf40 plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/OvsVifDriver.java 7038d7e server/src/com/cloud/network/guru/PublicNetworkGuru.java 8beb42e Diff: https://reviews.apache.org/r/12985/diff/ Testing --- Tested locally, advanced zone with a KVM hypervisor. Thanks, Toshiaki Hatano
RE: [DISCUSS] Compatibility issue between network plugins and hypervisors
I agree with Murali. I feel NetworkGuru should know their capability and should called when we add cluster. NetworkGurus already provide canHandle(NetworkOffering, NetworkType, PhysicalNetwork) method to check their capability. So, how about overloading this method to get HypervisorType in arguments? Thanks, -- Toshiaki -Original Message- From: Murali Reddy [mailto:murali.re...@citrix.com] Sent: Thursday, July 25, 2013 22:33 To: dev@cloudstack.apache.org Subject: Re: [DISCUSS] Compatibility issue between network plugins and hypervisors Also, should not we treat 'isolation' as Network Element capability rather than Hypervisor. Tunnelling capability could be a Hypervisor capability, but isolation (STT/GRE) is Network Element capability? So,zone isolation -> isolation provider -> supported hypervisors should be checked against add cluster IMO. On 26/07/13 9:24 AM, "Chiradeep Vittal" wrote: >+1 (with a caveat), good idea since isolation method is supported on a >per-zone basis. >The caveat is that sometimes it makes sense to support multiple >isolation methods in a zone. >For example, VPC(advanced) + basic in the same zone. >Why would one do this? Simply because someone might start with one >isolation method (basic) and then offer advanced (using overlays like >VxLAN f.e). Since templates/snapshots/volumes tend to be zone-specific, >this makes the transition easier. >This is not unlike AWS "EC2-classic" and "VPC" in the same zone. > > >On 7/26/13 3:34 AM, "Alex Huang" wrote: > >>+1 >> >>I think we should take advantage of hypervisor capabilities to look >>for that compatibility. >> >>--Alex >> >>> -Original Message- >>> From: Toshiaki Hatano [mailto:toshiaki.hat...@verio.net] >>> Sent: Thursday, July 25, 2013 3:01 PM >>> To: dev@cloudstack.apache.org >>> Subject: [DISCUSS] Compatibility issue between network plugins and >>> hypervisors >>> >>> Hi devs, >>> >>> >>> >>> CloudStack supports many hypervisors and many network isolation >>>methods. >>> >>> Some isolation method doesn't (or cannot) support some hypervisors, >>> >>> but it looks cloudstack doesn't check compatibility between network >>>isolation and hypervisors. >>> >>> >>> >>> Why don't we check it during addCluster, first timing cloudstack- >>>management know isolation and hypervisor, and fail if it's >>>incompatible? >>> >>> >>> >>> Best Regards, >>> >>> -- >>> >>> Toshiaki Hatano >>> >>> Verio, an NTT Communications company >>> E-mail: toshiaki.hat...@verio.net >>> <mailto:toshiaki.hat...@verio.net> >>> >>> AIM: toshiaki.hat...@verio.net <mailto:toshiaki.hat...@verio.net> >>> >>> >>> >>> >>> >>> This email message is intended for the use of the person to whom it >>>has been sent, and may contain information that is confidential or >>>legally protected. If you are not the intended recipient or have >>>received this message in error, you are not authorized to copy, >>>distribute, or otherwise use this message or its attachments. Please >>>notify the sender immediately by return e-mail and permanently >>>delete this message and any attachments. >>> Verio Inc. makes no warranty that this email is error or virus free. >>>Thank you. > > This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio Inc. makes no warranty that this email is error or virus free. Thank you.
[DISCUSS] Compatibility issue between network plugins and hypervisors
Hi devs, CloudStack supports many hypervisors and many network isolation methods. Some isolation method doesn't (or cannot) support some hypervisors, but it looks cloudstack doesn't check compatibility between network isolation and hypervisors. Why don't we check it during addCluster, first timing cloudstack-management know isolation and hypervisor, and fail if it's incompatible? Best Regards, -- Toshiaki Hatano Verio, an NTT Communications company E-mail: toshiaki.hat...@verio.net <mailto:toshiaki.hat...@verio.net> AIM: toshiaki.hat...@verio.net <mailto:toshiaki.hat...@verio.net> This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio Inc. makes no warranty that this email is error or virus free. Thank you.
Re: Review Request 12623: CLOUDSTACK-2328: Linux native VXLAN support on KVM hypervisor
> On July 19, 2013, 4:56 p.m., Chiradeep Vittal wrote: > > scripts/vm/network/vnet/modifyvxlan.sh, line 28 > > <https://reviews.apache.org/r/12623/diff/2/?file=323001#file323001line28> > > > > I think there is a need to prevent the guest vm from spoofing the > > multicast? > > Toshiaki Hatano wrote: > We don't need to. > > VM can only see Inner Frame of VXLAN packet. > VXLAN uses multicast for Outer Packet that VM cannot see or manipulate. > > Below is bridging diagram within KVM host. > > InnerFrame: VM <-> eth*|vnet* <-> brethX-Y <-> vxlanY > || (*1) > > OuterPacket: cloudbr* <-> eth* ==> > Outside of the Host > > > All frame that VM sent are encapsulated at (*1). > Almost frames are mapped to unicast here, since vxlanY interface learns > mapping between other VMs' MAC address and Hosts IP address. > Only when vxlanY interface haven't learned mapping yet or inner frame is > multicast or unicast frame, vxlanY interface uses multicast group statically > assigned at line 33 of modifyvxlan.sh. > > Multicast group is assigned statically to vxlan interface of host, so VM > cannot spoof multicast group. > > Chiradeep Vittal wrote: > Yes, I agree. > What about throttling multicasts to prevent broadcast/multicast storms > (using ebtables for example)? I think throttling is possible and good to have. But I see some difficulty in implementation. E.g. What is valid threshold, limit per VM or per vNet, how to match all BUM traffic without having false positive, etc... I'll put it in TODO list of the patch. On July 19, 2013, 4:56 p.m., Toshiaki Hatano wrote: > > I just wanted to make sure that you have tested your patch with regular > > VLANs as well. > > And, what the behavior will be when VxLAN is enabled in the zone, but only > > Xen / VMW hypervisors are there > > Also, some documentation on how the cloud operator can get this feature > > (KVM version/Which version of Centos/Ubuntu/etc), configuration of > > switches, bridges, etc would be useful. > > Toshiaki Hatano wrote: > Yes, I did same test with regular VLANs and it works fine. > > I don't have Xen nor VMWare to test with, but as far as I read the code... > Xen-agent would rise CloudRuntimeException("Unable to support this type > of network broadcast domain: " + nic.getBroadcastUri()) in > com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNetwork(Connection, > NicTO) before they actually submit VIF.create to hypervisor. > VMWare-agent would warn("Unrecognized broadcast type in VmwareResource, > type: " + nicTo.getBroadcastType().toString() + ". Use vlan info from > labeling: " + defaultVlan) in > com.cloud.hypervisor.vmware.resource.VmwareResource.getVlanInfo(NicTO, > String) and assign vlan interface with defultVlan instead of VXLAN. > > In short, Xen-agent handle unknown isolation type as error and don't > start VM. > VMWare-agent just ignore unknown isolation type and start VM with default > VLAN. > > > Yes, I will write documentation. > Is that requirement for this patch to be committed? > > Chiradeep Vittal wrote: > Generally speaking we need documentation. We have so many network plugins > committed without any documentation, that *IMHO* renders them useless. QA is > not able to include these test cases because they do not understand the > feature. Developers will break compatibility because they do not know how to > replicate / test the feature. And docs will ignore it because they have never > heard of it. > > Re: compatibility with other hypervisors, can you bring up this issue on > the dev mailing list and we can discuss a better solution there? > > Otherwise the patch is good. Re: documentation I agree. I started writing documentation in publican format. Once it's done, I'll submit it too. Re: compatibility Sure, I'll submit it. Thanks for the review! - Toshiaki --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/12623/#review23523 --- On July 23, 2013, 6:45 a.m., Toshiaki Hatano wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/12623/ > -
Re: Review Request 12623: CLOUDSTACK-2328: Linux native VXLAN support on KVM hypervisor
te VM from (C) to empty host (A) 9. Confirm it's pingable from VM to public network #) Test case 5: plug/unplug Nic 1. Add an instance from UI, create network during wizard. 2. Create additional network 3. Add NIC for network created in step 2 to the VM 4. Confirm it's pingable from VM to public network by using both side of NICs 5. Delete NIC created in step 3 6. Confirm bridge/vxlan device deleted on the host Thanks, Toshiaki Hatano
RE: 4.2 KVM agent can't communicate with 4.1 management server
I got a question from this issue. Should cloudstack allow version mismatch between management and agent? I think it's very good to have for ops. But it looks very difficult for dev, since every little change could break the interoperability. (We have to test it?) -- Toshiaki -Original Message- From: Wido den Hollander [mailto:w...@widodh.nl] Sent: Monday, July 22, 2013 07:53 To: dev@cloudstack.apache.org Subject: 4.2 KVM agent can't communicate with 4.1 management server Hi, While reviewing 12775 I upgraded my Agents from 4.1 to 4.2, but kept my management server at 4.1 When the Agent starts it sends a StartupRoutingCommand to the management server, but this has changed it seems: In 4.1 the Agent sends this JSON: Sending Startup: Seq 4-0: { Cmd , MgmtId: -1, via: 4, Ver: v1, Flags: 1, [{"StartupRoutingCommand":{ In 4.2 however the JSON data starts with: Sending Startup: Seq 1-6: { Cmd , MgmtId: -1, via: 1, Ver: v1, Flags: 1, [{"com.cloud.agent.api.StartupRoutingCommand":{ So the Agent sends the full name of the class and this confuses the Management server, it throws an Exception: Caused by: com.cloud.utils.exception.CloudRuntimeException: can't find com.cloud.agent.api.com.cloud.agent.api.StartupRoutingCommand at com.cloud.agent.transport.ArrayTypeAdaptor.deserialize(ArrayTypeAdaptor. java:79) at com.cloud.agent.transport.ArrayTypeAdaptor.deserialize(ArrayTypeAdaptor. java:37) at com.google.gson.JsonDeserializerExceptionWrapper.deserialize(JsonDeseria lizerExceptionWrapper.java:51) ... 15 more So it's searching for "com.cloud.agent.api.com.cloud.agent.api.StartupRoutingCommand" which obviously fails. I'm not sure how to fix this, since StartupRoutingCommand simply calls "RouterPrivateIpStrategy.class.getCanonicalName()" I created this issue for it: https://issues.apache.org/jira/browse/CLOUDSTACK-3714 Any suggestions on how to fix this? Wido This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio Inc. makes no warranty that this email is error or virus free. Thank you.
Re: Review Request 12623: CLOUDSTACK-2328: Linux native VXLAN support on KVM hypervisor
> On July 19, 2013, 4:56 p.m., Chiradeep Vittal wrote: > > scripts/vm/network/vnet/modifyvxlan.sh, line 28 > > <https://reviews.apache.org/r/12623/diff/2/?file=323001#file323001line28> > > > > I think there is a need to prevent the guest vm from spoofing the > > multicast? We don't need to. VM can only see Inner Frame of VXLAN packet. VXLAN uses multicast for Outer Packet that VM cannot see or manipulate. Below is bridging diagram within KVM host. InnerFrame: VM <-> eth*|vnet* <-> brethX-Y <-> vxlanY || (*1) OuterPacket: cloudbr* <-> eth* ==> Outside of the Host All frame that VM sent are encapsulated at (*1). Almost frames are mapped to unicast here, since vxlanY interface learns mapping between other VMs' MAC address and Hosts IP address. Only when vxlanY interface haven't learned mapping yet or inner frame is multicast or unicast frame, vxlanY interface uses multicast group statically assigned at line 33 of modifyvxlan.sh. Multicast group is assigned statically to vxlan interface of host, so VM cannot spoof multicast group. > On July 19, 2013, 4:56 p.m., Chiradeep Vittal wrote: > > scripts/vm/network/vnet/modifyvxlan.sh, line 202 > > <https://reviews.apache.org/r/12623/diff/2/?file=323001#file323001line202> > > > > What about vxlan with OVS, will this work? Unfortunately, it won't work with OVS. But it's not problem of my implementation. Problem is that current release of OVS doesn't fully support VXLAN protocol. Lack of multicast support is critical, because VXLAN isolation depends on multicast learning feature of VXLAN protocol. Please see "Q: How much of the VXLAN protocol does Open vSwitch currently support?" in the URL below for detail. http://git.openvswitch.org/cgi-bin/gitweb.cgi?p=openvswitch;a=blob;f=FAQ;h=98d273dd2d4311d16a3fff33051b0c3beed6e6b1;hb=d4c5b6423aa063eaf296ec8cf7d1a50197863cec VXLAN without multicast learning is very similar to GRE. That means, if we want to support VXLAN tunneling frame format with current OVS, we have to have SDN controller to setup flow rules in OVS. I think SDN controller is unnecessary component for VXLAN isolation, since VXLAN protocol is designed to work well without a hassle in SDN. In my opinion, what we have to do is wait for OVS release supporting VXLAN completely. Once OVS start supporting multicast aspect of VXLAN, we can hopefully start implementing VXLAN isolation for KVM with OVS and/or Xen. On July 19, 2013, 4:56 p.m., Toshiaki Hatano wrote: > > I just wanted to make sure that you have tested your patch with regular > > VLANs as well. > > And, what the behavior will be when VxLAN is enabled in the zone, but only > > Xen / VMW hypervisors are there > > Also, some documentation on how the cloud operator can get this feature > > (KVM version/Which version of Centos/Ubuntu/etc), configuration of > > switches, bridges, etc would be useful. Yes, I did same test with regular VLANs and it works fine. I don't have Xen nor VMWare to test with, but as far as I read the code... Xen-agent would rise CloudRuntimeException("Unable to support this type of network broadcast domain: " + nic.getBroadcastUri()) in com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNetwork(Connection, NicTO) before they actually submit VIF.create to hypervisor. VMWare-agent would warn("Unrecognized broadcast type in VmwareResource, type: " + nicTo.getBroadcastType().toString() + ". Use vlan info from labeling: " + defaultVlan) in com.cloud.hypervisor.vmware.resource.VmwareResource.getVlanInfo(NicTO, String) and assign vlan interface with defultVlan instead of VXLAN. In short, Xen-agent handle unknown isolation type as error and don't start VM. VMWare-agent just ignore unknown isolation type and start VM with default VLAN. Yes, I will write documentation. Is that requirement for this patch to be committed? - Toshiaki ------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/12623/#review23523 --- On July 17, 2013, 11:54 p.m., Toshiaki Hatano wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/12623/ > --- > > (Updated July 17, 2013, 11:54 p.m.) > > > Review request for cloudstack, Alena Prokharchyk, Chiradeep Vittal, Murali > Reddy, Hugo Trippaers, and Sheng Yang. > > > Bugs: https://issues.apache.org/jira/browse/CLOUDSTAC
Re: Review Request 12623: CLOUDSTACK-2328: Linux native VXLAN support on KVM hypervisor
grate VM from (C) to empty host (A) 9. Confirm it's pingable from VM to public network #) Test case 5: plug/unplug Nic 1. Add an instance from UI, create network during wizard. 2. Create additional network 3. Add NIC for network created in step 2 to the VM 4. Confirm it's pingable from VM to public network by using both side of NICs 5. Delete NIC created in step 3 6. Confirm bridge/vxlan device deleted on the host Thanks, Toshiaki Hatano
Re: Review Request 12623: CLOUDSTACK-2328: Linux native VXLAN support on KVM hypervisor
> On July 17, 2013, 8:07 a.m., Chiradeep Vittal wrote: > > Hi Chiradeep, Thanks for the review! I've revised patch. > On July 17, 2013, 8:07 a.m., Chiradeep Vittal wrote: > > plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/BridgeVifDriver.java, > > line 229 > > <https://reviews.apache.org/r/12623/diff/1/?file=322352#file322352line229> > > > > it seems a little pointless to determine protocol as it does not get > > passed anywhere. Fixed it. > On July 17, 2013, 8:07 a.m., Chiradeep Vittal wrote: > > plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/BridgeVifDriver.java, > > line 230 > > <https://reviews.apache.org/r/12623/diff/1/?file=322352#file322352line230> > > > > should this be case insensitive? No. Name of VXLAN interfaces are generated in line 31 of modifyvxlan.sh, and it always be lower case. > On July 17, 2013, 8:07 a.m., Chiradeep Vittal wrote: > > plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/BridgeVifDriver.java, > > line 198 > > <https://reviews.apache.org/r/12623/diff/1/?file=322352#file322352line198> > > > > is this related? What is the reason for this change? Can you put in a > > comment or a test case showing the format of the expected string and how it > > is getting processed by the script? Yes, it's related. Originally cmdout is used just to check if there're active VM using the bridge, by checking "vnet" interface exist or not. Since Script.runSimpleBashScript(String) only returns first line of command output, I put grep to make sure "vnetX" will be returned in first line if exist. For VXLAN, in addition to the reason above, BridgeVifDriver have to check the output of "ls /sys/class/net/$brName/brif" to determine which protocol (VLAN or VXLAN) used in the bridge. (Line 230-234) But grep discard output lines other than "vnet". It's inconvenient for the purpose because BridgeVifDriver have to check output lines other than "vnet" to determine protocol. "tr '\n' ' '" just replace newline character and make it one line. So, Script.runSimpleBashScript returns all interface name belonging to the bridge, separated by space character. Expected result is like "vnet10 vnet11 eth0.123 " (VLAN isolation) or "vnet6 vnet9 vxlan12345 " (VXLAN isolation). > On July 17, 2013, 8:07 a.m., Chiradeep Vittal wrote: > > plugins/network-elements/vxlan/src/com/cloud/network/guru/VxlanGuestNetworkGuru.java, > > line 87 > > <https://reviews.apache.org/r/12623/diff/1/?file=322355#file322355line87> > > > > What is the impact of this TODO? What I'd like to mean were VXLAN isolation uses same DB-tables that VLAN uses to store vNet allocation status. It's actually not a TODO, removed. > On July 17, 2013, 8:07 a.m., Chiradeep Vittal wrote: > > plugins/network-elements/vxlan/src/com/cloud/network/guru/VxlanGuestNetworkGuru.java, > > line 156 > > <https://reviews.apache.org/r/12623/diff/1/?file=322355#file322355line156> > > > > Remove unnecessary toDOs Fixed it. - Toshiaki --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/12623/#review23257 --- On July 17, 2013, 3:06 a.m., Toshiaki Hatano wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/12623/ > --- > > (Updated July 17, 2013, 3:06 a.m.) > > > Review request for cloudstack, Alena Prokharchyk, Chiradeep Vittal, Murali > Reddy, Hugo Trippaers, and Sheng Yang. > > > Bugs: https://issues.apache.org/jira/browse/CLOUDSTACK-2328 > > > Repository: cloudstack-git > > > Description > --- > > CLOUDSTACK-2328: Linux native VXLAN support on KVM hypervisor > > Initial patch for VXLAN support. > Fully functional, hopefully, for GuestNetwork - AdvancedZone. > > Patch Note: > in cloudstack-server > - Add isolation method VXLAN > - Add VxlanGuestNetworkGuru as plugin for VXLAN isolation > - Modify NetworkServiceImpl to handle extended vNet range for VXLAN isolation > - Add VXLAN isolation option in zoneWizard UI > > in cloudstack-agent (kvm) > - Add modifyvxlan.sh script that handle bridge/vxlan interface manipulation > script > -- Usage is exactly same to modifyvlan.sh > - BridgeVifDriver will call modifyvxlan.sh instead of modifyvlan.sh when > VXLAN is used for isolatio
Review Request 12623: CLOUDSTACK-2328: Linux native VXLAN support on KVM hypervisor
ic network #) Test case 5: plug/unplug Nic 1. Add an instance from UI, create network during wizard. 2. Create additional network 3. Add NIC for network created in step 2 to the VM 4. Confirm it's pingable from VM to public network by using both side of NICs 5. Delete NIC created in step 3 6. Confirm bridge/vxlan device deleted on the host Thanks, Toshiaki Hatano
Review Request 12496: CLOUDSTACK-3431: KVM: cloudstack-plugin-hypervisor-kvm with BridgeVifDriver doesn't cleanup vNet due to multiple reasons
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/12496/ --- Review request for cloudstack, Alena Prokharchyk, Chiradeep Vittal, Murali Reddy, Hugo Trippaers, and Sheng Yang. Bugs: https://issues.apache.org/jira/browse/CLOUDSTACK-3431 Repository: cloudstack-git Description --- #) Bug description When last VM which uses the guest network disappeared on the host, KVM agent should clean up it's guest network bridge and vlanIf but it doesn't. Biggest cause is missing parameter on calling modifyvlan.sh. But I found there're more errors related to this issue. 1) missing parameter on calling modifyvlan.sh in com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.cleanupVnet(Connect, String) 2) VirtualMachineName.getVnet(String) will never return vnet 3) com.cloud.agent.api.StopCommand.getVnet(String) will never return vnet 4) com.cloud.hypervisor.kvm.resource.LibvirtComputingResource doesn't call cleanupVnet(Connect, String) when they execute UnPlugCommand #) This patch changes as below. - Move vnetBridge clean up function from LibvirtComputingResource to BridgeVifDriver -- since only BridgeVifDriver have to handle this event - LibvirtComputingResource now properly call VifDriver.unplug() when it receives UnPlugCommand - Remove not working and no longer used method getVnet(String) from VirtualMachineName - Remove not working and no longer used method getVnet() from StopCommand - Remove unused constructer StopCommand(VirtualMachine, String, boolean) from StopCommand - Remove unused member vnet from StopCommand - Remove unused member _modifyVlanPath from OvsVifDriver Diffs - api/src/com/cloud/vm/VirtualMachineName.java 81838b9 core/src/com/cloud/agent/api/StopCommand.java a3ee3c9 plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/BridgeVifDriver.java 0db83cc plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java 914017c plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/OvsVifDriver.java 951badd Diff: https://reviews.apache.org/r/12496/diff/ Testing --- Tested with 2 KVM hosts and confirmed it correctly manipulate vnetBridge with start, stop, migrate, plug, and unplug event Thanks, Toshiaki Hatano
RE: Review Request 12278: CLOUDSTACK-3384: CloudStack allow VLAN range between 0-4096. Should be 0-'4095'.
My apology for sending review request in HTML format... How can I change this behavior of Review Board? (Is there setting?) -- Toshiaki From: Toshiaki Hatano [mailto:nore...@reviews.apache.org] On Behalf Of Toshiaki Hatano Sent: Friday, July 05, 2013 14:15 To: Hugo Trippaers; Sheng Yang; Chiradeep Vittal; Alena Prokharchyk; Murali Reddy Cc: cloudstack; Toshiaki Hatano Subject: Review Request 12278: CLOUDSTACK-3384: CloudStack allow VLAN range between 0-4096. Should be 0-'4095'. This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/12278/ Review request for cloudstack, Alena Prokharchyk, Chiradeep Vittal, Murali Reddy, Hugo Trippaers, and Sheng Yang. By Toshiaki Hatano. Bugs: https://issues.apache.org/jira/browse/CLOUDSTACK-3384 Repository: cloudstack-git Description CLOUDSTACK-3384: CloudStack allow VLAN range between 0-4096. Should be 0-'4095'. There're VLAN range check code in com.cloud.network.NetworkServiceImpl. But it allows VLAN range between 0-4096. VLAN ID have 12 bit field and it's between 0-4095 (0x000 - 0xFFF) . CloudStack should return error when someone try to assign VLAN ID 4096 to network. Testing Trying to create zone with Guest VLAN range 4090-4096 from WebUI. updatePhysicalNetwork returns error correctly. Diffs • server/src/com/cloud/network/NetworkServiceImpl.java (05df742) View Diff This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio Inc. makes no warranty that this email is error or virus free. Thank you.
Review Request 12278: CLOUDSTACK-3384: CloudStack allow VLAN range between 0-4096. Should be 0-'4095'.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/12278/ --- Review request for cloudstack, Alena Prokharchyk, Chiradeep Vittal, Murali Reddy, Hugo Trippaers, and Sheng Yang. Bugs: https://issues.apache.org/jira/browse/CLOUDSTACK-3384 Repository: cloudstack-git Description --- CLOUDSTACK-3384: CloudStack allow VLAN range between 0-4096. Should be 0-'4095'. There're VLAN range check code in com.cloud.network.NetworkServiceImpl. But it allows VLAN range between 0-4096. VLAN ID have 12 bit field and it's between 0-4095 (0x000 - 0xFFF) . CloudStack should return error when someone try to assign VLAN ID 4096 to network. Diffs - server/src/com/cloud/network/NetworkServiceImpl.java 05df742 Diff: https://reviews.apache.org/r/12278/diff/ Testing --- Trying to create zone with Guest VLAN range 4090-4096 from WebUI. updatePhysicalNetwork returns error correctly. Thanks, Toshiaki Hatano
RE: [PROPOSAL] [CLOUDSTACK-2328] Linux native VXLAN support on KVM hypervisor
Thanks David! I've assigned the task to myself. Sincerely, -- Toshiaki > -Original Message- > From: David Nalley [mailto:da...@gnsa.us] > Sent: Friday, May 17, 2013 6:58 PM > To: dev@cloudstack.apache.org > Subject: Re: [PROPOSAL] [CLOUDSTACK-2328] Linux native VXLAN support on > KVM hypervisor > > > > > I'd like to assign Jira ticket to me but it looks I don't have permission > to do so. > > https://issues.apache.org/jira/browse/CLOUDSTACK-2328 > > Could anyone kindly assign the ticket to me? > > > > I've granted you enough karma so you can do this and many other things in > Jira now. Feel free to assign it to yourself. > > Thanks, > > --David This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio Inc. makes no warranty that this email is error or virus free. Thank you.
[PROPOSAL] [CLOUDSTACK-2328] Linux native VXLAN support on KVM hypervisor
Hi all, I've done initial draft of FS. https://cwiki.apache.org/confluence/display/CLOUDSTACK/Feature+Proposal+-+Linux+native+VXLAN+support+on+KVM+hypervisor Please review it and provide your comment and feedback. And let me know if there're something unclear or missing, I'll update as soon as possible. I'd like to assign Jira ticket to me but it looks I don't have permission to do so. https://issues.apache.org/jira/browse/CLOUDSTACK-2328 Could anyone kindly assign the ticket to me? Thanks, -- Toshiaki > -Original Message- > From: Toshiaki Hatano [mailto:toshiaki.hat...@verio.net] > Sent: Friday, May 03, 2013 3:50 PM > To: dev@cloudstack.apache.org; us...@cloudstack.apache.org > Subject: RE: [DISCUSS] Linux native VXLAN support on KVM hypervisor > > Thank you all for your comment and approval! > > I agree with Justin's proposal. > And yesterday, Open vSwitch released v1.10.0 with VXLAN support. > http://openvswitch.org/pipermail/announce/2013-May/52.html > > > I've created Jira ticket and start writing design doc. > https://issues.apache.org/jira/browse/CLOUDSTACK-2328 > > Once I've done design doc, I'll submit it to dev list. > > Sincerely, > -- > Toshiaki > > > -Original Message- > > From: Justin Grudzien [mailto:grudz...@gmail.com] > > Sent: Thursday, May 02, 2013 5:35 PM > > To: dev@cloudstack.apache.org > > Cc: dev@cloudstack.apache.org > > Subject: Re: [DISCUSS] Linux native VXLAN support on KVM hypervisor > > > > I will +1 this. I spoke with Cisco a few weeks ago and they certainly > > see VXLANS as being the future for cloud infrastructures. In addition > > to Linux support we should look at the Cisco 1000v and open vSwitch support > a well. > > Cisco said they already have VMWare support today on the 1000v with > > KVM coming soon. > > > > Justin > > > > Sent from my iPhone > > > > On May 2, 2013, at 3:31 PM, Chip Childers > wrote: > > > > > On Wed, May 01, 2013 at 04:58:12PM -0400, Toshiaki Hatano wrote: > > >> Hi all, > > >> > > >> I’d like to add Linux native VXLAN support on KVM hypervisor. > > >> > > >> Currently, advanced zone with VLAN isolation can hold only 4k > > >> networks > > (= accounts) in a zone due to the VLAN ID limitation. > > >> 4k accounts per zone is not enough for IaaS provider like us. > > >> Furthermore, VPC will allow single account to consume multiple > networks. > > >> > > >> Linux kernel 3.7 or later supports VXLAN as part of its ordinal > > >> networking > > function. > > >> VXLAN enable Layer 2 tunneling over UDP/IP with VLAN-like > > >> encapsulation > > and allow 16M isolated networks in the domain. > > >> So, by using linux native VXLAN support, we can extend network > > >> limits > > without introducing unnecessary complexity. > > >> (But in other words, it’s not as flexible as Open vSwitch. Only > > >> thing Linux native VXLAN provides is multipoint L2 tunneling.) > > >> > > >> Any thoughts about this? > > >> > > >> > > >> P.S. > > >> > > >> I’m currently working on this as my internship project. > > >> As proof of concept, I’ve modified “modifyvlan.sh” script which is > > >> actual > > VLAN create/delete manipulation script called from cloud-agent, to > > create and to use VXLAN interface instead of VLAN interface. > > >> Modified script is tested with CloudStack 4.0.1 and 3 KVM > > >> hypervisors > > based on CentOS 6.4 + 3.8.6 kernel. > > >> And it looks working. (But I’m still testing) > > >> > > >> > > >> P.S.2. > > >> > > >> FYI: OpenStack already started process [1] to support Linux native > VXLAN. > > >> [1] https://review.openstack.org/#/c/26516/ > > >> > > >> > > >> Best Regards, > > >> -- > > >> Toshiaki Hatano > > > > > > I note that no one has replied to this thread yet, but I'll give you > > > my general +1 on the idea. > > > > > > Can some of the network-centric folks on the dev list please speak > > > up on the proposal? > > > > > > -chip > > > This email message is intended for the use of the person to whom it has > been sent, and may contain information that is confidential or legally > protected. If you are not the intended recipient or have received this > message in error, you are not authorized to copy, distribute, or otherwise > use this message or its attachments. Please notify the sender immediately > by return e-mail and permanently delete this message and any attachments. > Verio Inc. makes no warranty that this email is error or virus free. Thank > you. This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio Inc. makes no warranty that this email is error or virus free. Thank you.
RE: [DISCUSS] Linux native VXLAN support on KVM hypervisor
Thank you all for your comment and approval! I agree with Justin's proposal. And yesterday, Open vSwitch released v1.10.0 with VXLAN support. http://openvswitch.org/pipermail/announce/2013-May/52.html I've created Jira ticket and start writing design doc. https://issues.apache.org/jira/browse/CLOUDSTACK-2328 Once I've done design doc, I'll submit it to dev list. Sincerely, -- Toshiaki > -Original Message- > From: Justin Grudzien [mailto:grudz...@gmail.com] > Sent: Thursday, May 02, 2013 5:35 PM > To: dev@cloudstack.apache.org > Cc: dev@cloudstack.apache.org > Subject: Re: [DISCUSS] Linux native VXLAN support on KVM hypervisor > > I will +1 this. I spoke with Cisco a few weeks ago and they certainly see > VXLANS as being the future for cloud infrastructures. In addition to Linux > support we should look at the Cisco 1000v and open vSwitch support a well. > Cisco said they already have VMWare support today on the 1000v with KVM > coming soon. > > Justin > > Sent from my iPhone > > On May 2, 2013, at 3:31 PM, Chip Childers wrote: > > > On Wed, May 01, 2013 at 04:58:12PM -0400, Toshiaki Hatano wrote: > >> Hi all, > >> > >> I’d like to add Linux native VXLAN support on KVM hypervisor. > >> > >> Currently, advanced zone with VLAN isolation can hold only 4k networks > (= accounts) in a zone due to the VLAN ID limitation. > >> 4k accounts per zone is not enough for IaaS provider like us. > >> Furthermore, VPC will allow single account to consume multiple networks. > >> > >> Linux kernel 3.7 or later supports VXLAN as part of its ordinal networking > function. > >> VXLAN enable Layer 2 tunneling over UDP/IP with VLAN-like encapsulation > and allow 16M isolated networks in the domain. > >> So, by using linux native VXLAN support, we can extend network limits > without introducing unnecessary complexity. > >> (But in other words, it’s not as flexible as Open vSwitch. Only thing > >> Linux native VXLAN provides is multipoint L2 tunneling.) > >> > >> Any thoughts about this? > >> > >> > >> P.S. > >> > >> I’m currently working on this as my internship project. > >> As proof of concept, I’ve modified “modifyvlan.sh” script which is actual > VLAN create/delete manipulation script called from cloud-agent, to create > and to use VXLAN interface instead of VLAN interface. > >> Modified script is tested with CloudStack 4.0.1 and 3 KVM hypervisors > based on CentOS 6.4 + 3.8.6 kernel. > >> And it looks working. (But I’m still testing) > >> > >> > >> P.S.2. > >> > >> FYI: OpenStack already started process [1] to support Linux native VXLAN. > >> [1] https://review.openstack.org/#/c/26516/ > >> > >> > >> Best Regards, > >> -- > >> Toshiaki Hatano > > > > I note that no one has replied to this thread yet, but I'll give you > > my general +1 on the idea. > > > > Can some of the network-centric folks on the dev list please speak up > > on the proposal? > > > > -chip This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio Inc. makes no warranty that this email is error or virus free. Thank you.
RE: [DISCUSS] Linux native VXLAN support on KVM hypervisor
Sorry, I forgot to add users list in To. Thanks, -- Toshiaki > -Original Message- > From: Toshiaki Hatano [mailto:toshiaki.hat...@verio.net] > Sent: Wednesday, May 01, 2013 2:58 PM > To: dev@cloudstack.apache.org > Subject: [DISCUSS] Linux native VXLAN support on KVM hypervisor > > Hi all, > > I’d like to add Linux native VXLAN support on KVM hypervisor. > > Currently, advanced zone with VLAN isolation can hold only 4k networks (= > accounts) in a zone due to the VLAN ID limitation. > 4k accounts per zone is not enough for IaaS provider like us. > Furthermore, VPC will allow single account to consume multiple networks. > > Linux kernel 3.7 or later supports VXLAN as part of its ordinal networking > function. > VXLAN enable Layer 2 tunneling over UDP/IP with VLAN-like encapsulation > and allow 16M isolated networks in the domain. > So, by using linux native VXLAN support, we can extend network limits without > introducing unnecessary complexity. > (But in other words, it’s not as flexible as Open vSwitch. Only thing Linux > native VXLAN provides is multipoint L2 tunneling.) > > Any thoughts about this? > > > P.S. > > I’m currently working on this as my internship project. > As proof of concept, I’ve modified “modifyvlan.sh” script which is actual > VLAN create/delete manipulation script called from cloud-agent, to create > and to use VXLAN interface instead of VLAN interface. > Modified script is tested with CloudStack 4.0.1 and 3 KVM hypervisors based > on CentOS 6.4 + 3.8.6 kernel. > And it looks working. (But I’m still testing) > > > P.S.2. > > FYI: OpenStack already started process [1] to support Linux native VXLAN. > [1] https://review.openstack.org/#/c/26516/ > > > Best Regards, > -- > Toshiaki Hatano > Verio, an NTT Communications company > E-mail: toshiaki.hat...@verio.net > AIM: toshiaki.hat...@verio.net > Phone: (801)437-7482 Office > (801)960-6410 Cellular > > > This email message is intended for the use of the person to whom it has > been sent, and may contain information that is confidential or legally > protected. If you are not the intended recipient or have received this > message in error, you are not authorized to copy, distribute, or otherwise > use this message or its attachments. Please notify the sender immediately > by return e-mail and permanently delete this message and any attachments. > Verio Inc. makes no warranty that this email is error or virus free. Thank > you. This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio Inc. makes no warranty that this email is error or virus free. Thank you.
[DISCUSS] Linux native VXLAN support on KVM hypervisor
Hi all, I’d like to add Linux native VXLAN support on KVM hypervisor. Currently, advanced zone with VLAN isolation can hold only 4k networks (= accounts) in a zone due to the VLAN ID limitation. 4k accounts per zone is not enough for IaaS provider like us. Furthermore, VPC will allow single account to consume multiple networks. Linux kernel 3.7 or later supports VXLAN as part of its ordinal networking function. VXLAN enable Layer 2 tunneling over UDP/IP with VLAN-like encapsulation and allow 16M isolated networks in the domain. So, by using linux native VXLAN support, we can extend network limits without introducing unnecessary complexity. (But in other words, it’s not as flexible as Open vSwitch. Only thing Linux native VXLAN provides is multipoint L2 tunneling.) Any thoughts about this? P.S. I’m currently working on this as my internship project. As proof of concept, I’ve modified “modifyvlan.sh” script which is actual VLAN create/delete manipulation script called from cloud-agent, to create and to use VXLAN interface instead of VLAN interface. Modified script is tested with CloudStack 4.0.1 and 3 KVM hypervisors based on CentOS 6.4 + 3.8.6 kernel. And it looks working. (But I’m still testing) P.S.2. FYI: OpenStack already started process [1] to support Linux native VXLAN. [1] https://review.openstack.org/#/c/26516/ Best Regards, -- Toshiaki Hatano Verio, an NTT Communications company E-mail: toshiaki.hat...@verio.net AIM: toshiaki.hat...@verio.net Phone: (801)437-7482 Office (801)960-6410 Cellular This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio Inc. makes no warranty that this email is error or virus free. Thank you.