Hello Palash, Thanks for the update, I'm glad you were able to resolve it. So the underlay interface for the overlay network is a 802.1q tagged interface then. Yes, the frames just won't end up going uplink with MTU mismatch.
Thanks, Jayanth On Mon, Jan 15, 2024 at 5:11 PM Palash Biswas <palash...@gmail.com> wrote: > Hi Jayanth, > > Thanks for the help! We resolved the issue by setting MTU at bond interface > instead of physical interface. > > Thanks again for the help, lifesaver. > > Regards, > Palash > > > On Mon, 15 Jan 2024 at 12:21 PM, Jayanth Reddy <jayanthreddy5...@gmail.com > > > wrote: > > > Hi, > > I just remembered about my testing setup with the MTU 1550 set. The > virtual > > interfaces of VMs and VRs have 1500 and I don't see any issues with the > > file transfers which make use of TCP over the Public Networks. Please > > analyze your uplinks. > > > > Thanks, > > Jayanth > > > > On Mon, Jan 15, 2024 at 11:58 AM Jayanth Reddy < > jayanthreddy5...@gmail.com > > > > > wrote: > > > > > Hello Palash, > > > Thanks for the response. The VR should have one NIC on your VXLAN > overlay > > > (guest) network and one on the public network. Please login to the > > > hypervisors to ensure the vNICs of your VM, VTEP IF, VR, bridge are on > an > > > expected MTU of exactly 1500 bytes. Also for the Public network, your > > VR's > > > NIC should be having the same MTU as your physical interface (or > bridge). > > > Please also ensure you have IGMP snooping turned on, on your switches > > > (because it is better to have); and as a quick test, can you try moving > > > your VR to the same host where your VM resides and test if that works? > If > > > you still observe loss with VMs and VRs being in the same host, please > > > inspect your Public Network on the uplinks for issues. > > > > > > *T**ip: *We also use VXLAN with multicast in our production but we make > > > use of Jumbo Frames on Hypervisors (MTU: 9000). CloudStack creates our > > > TEPs, vNICs, bridges with MTU 8950 and VMs, VRs are configured to use > > 1500 > > > MTU by default inside the guest OS. Your issue may not exactly be with > > MTU > > > but something else. > > > > > > Regards, > > > Jayanth > > > ------------------------------ > > > *From:* Palash Biswas <palash...@gmail.com> > > > *Sent:* Monday, January 15, 2024 10:53:38 AM > > > *To:* users@cloudstack.apache.org <users@cloudstack.apache.org> > > > *Subject:* Re: Disconnections to VMs from Public Network and VXLAN > > > > > > Hi Jayanth, > > > > > > The transfers work between VMs residing in different hypervisors when > > using > > > Guest network. Hypervisor physical interface is set at MTU 1550 > > > > > > The transfers failed when attempting to connect using Public network > > which > > > is using VLAN and different physical network from the Guest network. > > > Besides file transfer using port 22, port 3389 (Windows RDP) also > failed. > > > > > > Regards, > > > Palash > > > > > > > > > On Mon, 15 Jan 2024 at 9:08 AM, Jayanth Reddy < > > jayanthreddy5...@gmail.com> > > > wrote: > > > > > > > Hello Palash, > > > > > > > > Please confirm if the file transfers work between VMs residing in > > > > different hypervisors. I hope you've already considered the overhead > of > > > 50 > > > > bytes for VXLAN and have already set atleast 1550 as MTU at your > > physdev > > > > interface used for VTEP communication among hypervisors. > > > > > > > > Thanks, > > > > Jayanth Reddy > > > > > > > > ________________________________ > > > > From: Palash Biswas <palash...@gmail.com> > > > > Sent: Monday, January 15, 2024 8:54:02 AM > > > > To: users@cloudstack.apache.org <users@cloudstack.apache.org> > > > > Subject: Disconnections to VMs from Public Network and VXLAN > > > > > > > > Hi Community, > > > > > > > > We have set up Cloudstack using VXLAN (and Multicast). > > > > > > > > VXLAN = Guest Network VLAN = Management and Public Network (Using > > > different > > > > Physical Network) VLAN = Storage Network (Using Linstor SDS) > > > > > > > > Our VMs can ping each other. But the moment we have a file transfer > to > > > the > > > > VMs from the public internet, the connections keeps disconnecting. > > > > > > > > We've been spending few days on this issue. Any ideas where we could > > look > > > > at? > > > > > > > > Regards, > > > > Palash > > > > > > > > > > > > > > > > >
