(sorry I should have also mentioned one other thing below)
On Wed, Jan 1, 2025 at 3:30 PM Christopher Morrow
<[email protected]> wrote:
>
>
>
> On Tue, Dec 24, 2024 at 8:26 AM Mike Hammett <[email protected]> wrote:
>>
>> In the articles I've read and videos I've watched, they have mentioned
>> varying amounts of reduced power. I didn't commit them to memory because
>> that wasn't the part I was interested in at the moment.
>>
>
> I'd think that, especially as data rates climb, the power consumption is
> going to really get important fast.
> When a single device requires ~50kw to run ... I think you'll want to make
> sure you have space/power to deal with that :(
>
> I'm not sure that distributed fabric plans make that problem better? (maybe
> it's all the same problem in the end because the fabric interconnect is
> going to be distance limited/etc too)
>
One thing to really think about here is:
"What is the traffic pattern you expect to see?"
I mean here:
* "I have jewels in the center of my network, and everyone comes
through the edge to the jewel(s), and then back out"
- "just have a ton of pizza boxes, all traffic is edge-to-core
and I'm not spending optics/etc in the local edge area bouncing
traffic around"
* "I have no jewels, I'm providing any-to-any connectivity, folk
might just bounce traffic through me and right back out the same
location to someone else"
- "oops, now I need to spray traffic in the local pop/metro and
do not need as much core-facing capacity out of that local area"
anyway, interesting conversation :)
>>
>> Management of the things is a big thing I've been concerned about going into
>> more modern systems. So often there's hand waiving regarding the
>> orchestration piece of non-traditional systems. From what I've seen (and I
>> would love to be wrong), you either build it in-house (not a small lift) or
>> you buy something that ends up taking away all of the cost advantages that
>> path had.
>>
>
> You almost certainly get into (pretty quickly) something that smells a bunch
> like:
> "here's my pile of ansible recipes for...."
> (choice of ansible here for example only, s/ansible/<whatever>/ of course
> to whatever you feel like)
>
> That's maybe fine if that's your jam? I think it's hard to reason/plan/build
> without some automation plan 'now',
> and it looks like a ton of folk start without that then try to retrofit once:
> "omg this is very large now... ugh" happens.
> (1-10 devices? sure fine do it by hand, 5-><bunches more> you really ought
> to have had an automation plan at ~5 ... my opinion clearly)
>
>>
>> Failure domain stuff is part of what I'm trying to learn more about, which
>> goes back to more about the fundamentals of how the fabric works.
>>
>
> yea... This part(reasoning about failure domains) I assume is also a tad hard.
> A scenario is:
> "I built this 200tb fabric, I interconnect to the outside with ~100T max
> and internally with ~100T"
> now that ~100T breaks and (ideally!) everything on the outside re-routes
> around to a different front-door... oops are you prepared for an extra ~100T
> arriving?
> How do you deal with parts (fabric parts) failing in part? "oops only 50T of
> my 100T can get through here and ... I also am still telling my external
> neighbors all's good"
>
> Really that failure-domain problem is tightly linked to the 'manage a ton of
> things' problem too.. at least for containing damage in a quick manner.
