I've been warned away from using the quad cards and speaking from experience I myself have used quad ethernet cards with x86 mikrotiks in the past with little success. All sorts of funky issues.
I understand clearly we can use vlans and such in ESXI but I'm talking the pure amount of traffic I'll need will be beyond the demands the bus on the motherboard will be able to handle. I guess I really need to look at the specs. I know PCI Express V6 can do over 100Gbps. But I'll prob use a couple off lease Dells in its own routing cluster. [ https://www.wavedirect.net/ | ] [ https://www.facebook.com/ruralhighspeed ] [ https://www.instagram.com/wave.direct/ ] [ https://www.linkedin.com/company/wavedirect-telecommunication/ ] [ https://twitter.com/wavedirect1 ] [ https://www.youtube.com/user/WaveDirect ] STEVEN KENNEY DIRECTOR OF GLOBAL CONNECTIVITY & CONTINUITY A: 158 Erie St. N | Leamington ON E: st...@wavedirect.org | P: 519-737-9283 W: www.wavedirect.net From: "Josh Baird" <joshba...@gmail.com> To: "af" <af@af.afmug.com> Sent: Monday, December 21, 2020 4:03:21 PM Subject: Re: [AFMUG] Mikrotik 1072 Frustrations The x710 has 4 SFP+ ports. You don't necessarily need dedicated physical interfaces for each router either. Our CHR boxes have a single x710 and several instances of CHR "sharing" the physical interfaces. We use ESXi with a virtual switch trunking VLANs to the CHR instances. On Mon, Dec 21, 2020 at 3:43 PM Steven Kenney < [ mailto:st...@wavedirect.org | st...@wavedirect.org ] > wrote: Right those are 2x though for a total of 4 ports. I need more than that, especially if I want to run more than one router on these. Might even consider to see if there is any support for 40Gbps cards or whether a server's bus can handle those speeds with a card. I can aggregate different devices on a switch instead. [ https://www.wavedirect.net/ | ] [ https://www.facebook.com/ruralhighspeed ] [ https://www.instagram.com/wave.direct/ ] [ https://www.linkedin.com/company/wavedirect-telecommunication/ ] [ https://twitter.com/wavedirect1 ] [ https://www.youtube.com/user/WaveDirect ] STEVEN KENNEY DIRECTOR OF GLOBAL CONNECTIVITY & CONTINUITY A: 158 Erie St. N | Leamington ON E: [ mailto:st...@wavedirect.org | st...@wavedirect.org ] | P: 519-737-9283 W: [ http://www.wavedirect.net/ | www.wavedirect.net ] From: "Josh Baird" < [ mailto:joshba...@gmail.com | joshba...@gmail.com ] > To: "af" < [ mailto:af@af.afmug.com | af@af.afmug.com ] > Sent: Monday, December 21, 2020 3:34:23 PM Subject: Re: [AFMUG] Mikrotik 1072 Frustrations 2x Intel x710. On Mon, Dec 21, 2020 at 2:22 PM Steven Kenney < [ mailto:st...@wavedirect.org | st...@wavedirect.org ] > wrote: BQ_BEGIN I've considered this too as a temporary measure because I can get some off lease well maintained Dell servers for peanuts. Problem is finding the right cards when you need 8 sfp+ ports. [ https://www.wavedirect.net/ | ] [ https://www.facebook.com/ruralhighspeed ] [ https://www.instagram.com/wave.direct/ ] [ https://www.linkedin.com/company/wavedirect-telecommunication/ ] [ https://twitter.com/wavedirect1 ] [ https://www.youtube.com/user/WaveDirect ] STEVEN KENNEY DIRECTOR OF GLOBAL CONNECTIVITY & CONTINUITY A: 158 Erie St. N | Leamington ON E: [ mailto:st...@wavedirect.org | st...@wavedirect.org ] | P: 519-737-9283 W: [ http://www.wavedirect.net/ | www.wavedirect.net ] From: "TJ Trout" < [ mailto:t...@voltbb.com | t...@voltbb.com ] > To: "af" < [ mailto:af@af.afmug.com | af@af.afmug.com ] > Sent: Monday, December 21, 2020 12:41:53 PM Subject: Re: [AFMUG] Mikrotik 1072 Frustrations We just bought 5x brand new 1072s and they are rebooting by watchdog. Upgraded ros and firmware, Disabled watchdog and then they hard freeze and require a reboot. We are going chr route. 1072 is junk. On Mon, Dec 21, 2020, 9:34 AM Dennis Burgess < [ mailto:dmburg...@linktechs.net | dmburg...@linktechs.net ] > wrote: BQ_BEGIN Hate to say it, but worked. Keep in mind that these units that we have had this issue had been running for 3+ years, so, it was out of warranty anyways. Dennis Burgess, Mikrotik Certified Trainer MTCNA, MTCRE, MTCWE, MTCTCE, MTCINE, MTCSE, HE IPv6 Sage, Cambium ePMP Certified Author of "Learn RouterOS- Second Edition” Link Technologies, Inc -- Mikrotik & WISP Support Services Office : 314-735-0270 Website: [ http://www.linktechs.net/ | http://www.linktechs.net ] Create Wireless Coverage’s with [ http://www.towercoverage.com/ | www.towercoverage.com ] From: AF < [ mailto:af-boun...@af.afmug.com | af-boun...@af.afmug.com ] > On Behalf Of Josh Baird Sent: Monday, December 21, 2020 9:45 AM To: AnimalFarm Microwave Users Group < [ mailto:af@af.afmug.com | af@af.afmug.com ] > Subject: Re: [AFMUG] Mikrotik 1072 Frustrations Haha - "Get a new one" .. only with MT. On Mon, Dec 21, 2020 at 8:22 AM Dennis Burgess < [ mailto:dmburg...@linktechs.net | dmburg...@linktechs.net ] > wrote: BQ_BEGIN Get a new one. That’s what we did, replaced it and the issue went away. Its like the 1072 has some kind of bug, or maybe a leaky cap, that eventually causes them to do this. No matter of fiddling fixes it, we swapped out the unit and have not have the same issue since. We have 1072s running 15gig inbound without issues. So, while I do attest that this is a MT (rather routeros) issue, we don’t have any data to know where it is unless the manufacture can tell us. So far they are tight lipped. Dennis Burgess, Mikrotik Certified Trainer MTCNA, MTCRE, MTCWE, MTCTCE, MTCINE, MTCSE, HE IPv6 Sage, Cambium ePMP Certified Author of "Learn RouterOS- Second Edition” Link Technologies, Inc -- Mikrotik & WISP Support Services Office : 314-735-0270 Website: [ http://www.linktechs.net/ | http://www.linktechs.net ] Create Wireless Coverage’s with [ http://www.towercoverage.com/ | www.towercoverage.com ] From: AF < [ mailto:af-boun...@af.afmug.com | af-boun...@af.afmug.com ] > On Behalf Of Steven Kenney Sent: Monday, December 21, 2020 12:56 AM To: af < [ mailto:af@af.afmug.com | af@af.afmug.com ] > Subject: Re: [AFMUG] Mikrotik 1072 Frustrations Their support is behaving the same way exactly. They told me to take the watchdog off and allow the internet to be frozen (downtime for all my customers) whenever it happens at any random point in the day/night while I rush to log into it with a serial cable. The absurdity of it. Luckily I told him from day one that I have a CONSOLE SERVER and that I log all the serial output of all my switches and routers. But even though I told them that a couple times they still requested I plug in a serial cable and wait for it to freeze. So I'm like .. hellooo there is a serial cable plugged in at all times! I told him there is no output. It just reboots. I even humored him and turned the watchdog off and we had a reboot on the weekend while the cable was plugged in and the console was logged in. I even had logging set to echo for a lot of things and nothing.. just a hard freeze. There is no BGP on this one and the processor is not high leading up to the crash at all. Updated 5 times already and its just a placebo. They need to get their shit together. [ https://imsva91-ctp.trendmicro.com/wis/clicktime/v1/query?url=https%3a%2f%2fwww.wavedirect.net&umid=D7E8655E-B6F3-F705-88EA-2A08303811DF&auth=079c058f437b7c6303d36c6513e5e8848d0c5ac4-ed4acb70d3fb71b72aeea4066f6012bfa1e2e66e ] [ https://www.facebook.com/ruralhighspeed ] [ https://www.instagram.com/wave.direct/ ] [ https://www.linkedin.com/company/wavedirect-telecommunication/ | ] [ https://twitter.com/wavedirect1 ] [ https://www.youtube.com/user/WaveDirect ] STEVEN KENNEY DIRECTOR OF GLOBAL CONNECTIVITY & CONTINUITY A: 158 Erie St. N | Leamington ON E: [ mailto:st...@wavedirect.org | st...@wavedirect.org ] | P: 519-737-9283 W: [ https://imsva91-ctp.trendmicro.com/wis/clicktime/v1/query?url=www.wavedirect.net&umid=D7E8655E-B6F3-F705-88EA-2A08303811DF&auth=079c058f437b7c6303d36c6513e5e8848d0c5ac4-a20d49e3e6a4f958e127c88b05686fb87748d1dd | https://imsva91-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=www.wavedirect.net&umid=D7E8655E-B6F3-F705-88EA-2A08303811DF&auth=079c058f437b7c6303d36c6513e5e8848d0c5ac4-a20d49e3e6a4f958e127c88b05686fb87748d1dd ] From: "Colin Stanners" < [ mailto:cstann...@gmail.com | cstann...@gmail.com ] > To: "af" < [ mailto:af@af.afmug.com | af@af.afmug.com ] > Sent: Monday, December 21, 2020 12:59:09 AM Subject: Re: [AFMUG] Mikrotik 1072 Frustrations This last year, I've seen a MikroTik CCR1072 switch from long being rock-solid to now having occasional random reboots (from watchdog) or 100% CPU usage, which strangles the BGP process. In the latter case, tools->profile would show the firewall taking 100% of CPU, even after temporarily disabling all firewall filter and NAT rules and connection tracking. Not fun. MT tech support did not seem super helpful or interested, mostly recommending to disable watchdog (unacceptable on a production router) or to upgrade firmware (without specifying the suspected cause of the problem or nature of the fix). Tried 1 update, that didn't seem to help, have now tried another... On Sun, Dec 20, 2020, 11:38 PM Steven Kenney < [ mailto:st...@wavedirect.org | st...@wavedirect.org ] > wrote: BQ_BEGIN MIkrotik has been rock solid for me for years. Until this year and the 1072's. Random reboots set off by watchdog timer on all of my 1072's. Some more than others. Threads in the forum all discuss the same problem exactly. Its a connection tracking issue.. however I need connection tracking on one particular router. I've adjusted everything I could. Firmware and board firmware all up to date etc. Happens randomly with low levels of traffic, high levels of traffic, sometimes a couple times a day, sometimes weeks. No DDOS evidence at all from upstream routers. Configs checked and rechecked by third party experts. I graph everything about the Mikrotik and there are no clues or anything abnormal happening before the crash. Plenty of memory, disk space, CPU etc. Replaces all the trannies, power cables and such. Not running BGP only OSPF on the one that is giving me the most trouble. Even have a serial console cable plugged into them to my opengear and set it to log pretty much everything to console including the kernel and nothing. A hard freeze. Then there is Mikrotik support... I've never needed their support before until now. So I put a ticket in and the shitty attitude I'm getting from them seems like they KNOW there is something wrong with the hardware and they are intentionally not being helpful. It is pretty clear to see with all the people reporting this issue that there IS an issue. If this is any indication of how things are going to go with Mikrotik on the newer hardware going forware I think its time to jump to an enterprise level system. Juniper most likely. Shame because they are just about keeping up with the demands with their hardware. Getting closer to 100Gbps etc and ROS7 ... but at their current pace I think we've outgrew them. All the threads discussing this issue has been absolutely quiet when it comes to Mikrotik jumping in to mention or try to help troubleshoot. I think they know they had bad hardware out there and do not want to honor warranties. I've heard rumors of bad batches of 1072's. Anyone else encounter this? [ https://imsva91-ctp.trendmicro.com/wis/clicktime/v1/query?url=https%3a%2f%2fwww.wavedirect.net&umid=D7E8655E-B6F3-F705-88EA-2A08303811DF&auth=079c058f437b7c6303d36c6513e5e8848d0c5ac4-ed4acb70d3fb71b72aeea4066f6012bfa1e2e66e ] [ https://www.facebook.com/ruralhighspeed ] [ https://www.instagram.com/wave.direct/ ] [ https://www.linkedin.com/company/wavedirect-telecommunication/ | ] [ https://twitter.com/wavedirect1 ] [ https://www.youtube.com/user/WaveDirect ] STEVEN KENNEY DIRECTOR OF GLOBAL CONNECTIVITY & CONTINUITY A: 158 Erie St. N | Leamington ON E: [ mailto:st...@wavedirect.org | st...@wavedirect.org ] | P: 519-737-9283 W: [ https://imsva91-ctp.trendmicro.com/wis/clicktime/v1/query?url=http%3a%2f%2fwww.wavedirect.net&umid=D7E8655E-B6F3-F705-88EA-2A08303811DF&auth=079c058f437b7c6303d36c6513e5e8848d0c5ac4-3ca9c8d0ac0d6566b9daf27839af27539533a38d | www.wavedirect.net ] -- AF mailing list [ mailto:AF@af.afmug.com | AF@af.afmug.com ] [ http://af.afmug.com/mailman/listinfo/af_af.afmug.com | http://af.afmug.com/mailman/listinfo/af_af.afmug.com ] -- AF mailing list [ mailto:AF@af.afmug.com | AF@af.afmug.com ] [ http://af.afmug.com/mailman/listinfo/af_af.afmug.com | http://af.afmug.com/mailman/listinfo/af_af.afmug.com ] -- AF mailing list [ mailto:AF@af.afmug.com | AF@af.afmug.com ] [ http://af.afmug.com/mailman/listinfo/af_af.afmug.com | http://af.afmug.com/mailman/listinfo/af_af.afmug.com ] BQ_END -- AF mailing list [ mailto:AF@af.afmug.com | AF@af.afmug.com ] [ http://af.afmug.com/mailman/listinfo/af_af.afmug.com | http://af.afmug.com/mailman/listinfo/af_af.afmug.com ] BQ_END -- AF mailing list [ mailto:AF@af.afmug.com | AF@af.afmug.com ] [ http://af.afmug.com/mailman/listinfo/af_af.afmug.com | http://af.afmug.com/mailman/listinfo/af_af.afmug.com ] -- AF mailing list [ mailto:AF@af.afmug.com | AF@af.afmug.com ] [ http://af.afmug.com/mailman/listinfo/af_af.afmug.com | http://af.afmug.com/mailman/listinfo/af_af.afmug.com ] BQ_END -- AF mailing list [ mailto:AF@af.afmug.com | AF@af.afmug.com ] [ http://af.afmug.com/mailman/listinfo/af_af.afmug.com | http://af.afmug.com/mailman/listinfo/af_af.afmug.com ] -- AF mailing list [ mailto:AF@af.afmug.com | AF@af.afmug.com ] [ http://af.afmug.com/mailman/listinfo/af_af.afmug.com | http://af.afmug.com/mailman/listinfo/af_af.afmug.com ] BQ_END -- AF mailing list AF@af.afmug.com http://af.afmug.com/mailman/listinfo/af_af.afmug.com
-- AF mailing list AF@af.afmug.com http://af.afmug.com/mailman/listinfo/af_af.afmug.com
