We just leveled up HARDCORE - Fibre Networking Adventure

[jakkuh_t]

Over the years we've upgraded our networking many times, but we've always neglected one upgrade - fibre optics. Today that changes.

 

 

Buy 25G Fibre Transceivers:
From FS.com (Dell): https://rb.gy/obqekd
From FS.com (Mellanox): https://rb.gy/jb2upm

 

Buy Direct Attach 100G Cabling:
From FS.com: https://rb.gy/vym8b5

 

Buy Fibre Optic Cabling:
From Infinite Cables: https://lmg.gg/fibre

 

Check out Fibre Testing Equipment:
From Fluke: https://lmg.gg/flukefibre 

 

[SignatureSigner]

Fiber Optics.... Sigh 

 

[Prodigy_Smit]

Ayy I actually watched a WAN show for once and this was there. Very cool.

 

PS: Your ingest stations are now 10 times faster than my home network rather than 4x faster.

[NZLaurence]

Pro Tech Tip for anyone doing this in their own office, Use single mode fibre.

I'm pretty sure in the next 12 months we will see the next video where they are trying to run 100gbit to workstations and have to re-run all this cable  

[samcool55]

Maybe off-topic but, since when does he have braces?

It's the first time I see them.

Nothing bad or whatever. Just surprising.

[Lurick]

Just now, NZLaurence said:

Pro Tech Tip for anyone doing this in their own office, Use single mode fibre.

I'm pretty sure in the next 12 months we will see the next video where they are trying to run 100gbit to workstations and have to re-run all this cable  

Nah, multimode can do 400Gbit over 100m or so on OM5 and shorter distance on OM4 but I'm doubtful on it's ability to do 800Gb+ on current OM4 or OM5 fiber. OM3 can do 100Gbit just fine for 100m and while I agree single mode is more expensive up front the reason is to not have to rip and replace since singlemode has theoretically limitless bandwidth compared to OM5 for multimode which is a much worse endeavor when you're talking about tens of thousands of cables

[sub68]

another thing I can't afford

[NZLaurence]

9 minutes ago, Lurick said:

Nah, multimode can do 400Gbit over 100m or so on OM5 and shorter distance on OM4 but I'm doubtful on it's ability to do 800Gb+ on current OM4 or OM5 fiber. OM3 can do 100Gbit just fine for 100m and while I agree single mode is more expensive up front the reason is to not have to rip and replace since singlemode has theoretically limitless bandwidth compared to OM5 for multimode which is a much worse endeavor when you're talking about tens of thousands of cables

Yes and No. 100Gbit over muiltmode requires more then 2 strands (normally MTP-12 connections). 100Gbit over singlemode can use DWDM to run over the same 2 strands.

 

If they decided tomorrow to put 100gbit to one of the workstations, they would need to redo the cable. If they had single mode cable, they could just swap the SFP modules. The price difference between single and muiltmode is minimal (Sometimes its even cheaper).

 

Also their description on Muiltmode vs Single mode is a little misleading. Muiltimode does take multiple paths down the cable, but it doesn't do multiple wavelengths. Meaning only one signal can go in one direction on a single cable.

Singlemode allows you to put multiple distant wavelengths, each carrying different data, in both directions down the same cable.

 

 

[Lurick]

Just now, NZLaurence said:

Yes and No. 100Gbit over muiltmode requires more then 2 strands (normally MTP-12 connections). 100Gbit over single move can use DWDM to run over the same 2 strands.

 

If they decided tomorrow to put 100gbit to one of the workstations, they would need to redo the cable. If they had single mode cable, they could just swap the SFP modules. The price difference between single and muiltmode is minimal (Sometimes its even cheaper).

 

Also their description on Muiltmode vs Single mode is a little misleading. Muiltimode does take multiple paths down the cable, but it doesn't do multiple wavelengths. Meaning only one signal can go in one direction on a single cable.

Singlemode allows you to put multiple distant wavelengths, each carrying different data, in both directions down the same cable.

 

 

Ah, not quite. There are 40/100Gbit BiDi optics for 100G

https://www.cisco.com/c/en/us/products/collateral/interfaces-modules/transceiver-modules/at-a-glance-c45-740242.pdf

[NZLaurence]

2 minutes ago, Lurick said:

Ah, not quite. There are 40/100Gbit BiDi optics for 100G

https://www.cisco.com/c/en/us/products/collateral/interfaces-modules/transceiver-modules/at-a-glance-c45-740242.pdf

I have heard people working on this technology a few years ago but had thought it turned into vaperware and have never actually seen one in the wild. Do you know if they are they actually orderable? They don't appear in any of the supported matrixes. 

[Lurick]

1 minute ago, NZLaurence said:

I have heard people working on this technology a few years ago but had thought it turned into vaperware and have never actually seen one in the wild. Do you know if they are they actually orderable? They don't appear in any of the supported matrixes. 

Yup, I've got a few dozen of them right now

 

I know the Cisco PID is QSFP-40/100-SRBD, I'll see if FS.com has any.

[Levent]

Nice Finnegan Speed and Marine shirt Jake. 

[Velcade]

Bold of you to assume I don't have fiber at home.  

[Dujith]

1 minute ago, NZLaurence said:

If they decided tomorrow to put 100gbit to one of the workstations, they would need to redo the cable. If they had single mode cable, they could just swap the SFP modules. The price difference between single and muiltmode is minimal (Sometimes its even cheaper).

I dont know where they got the price difference from? but we work mainly with singlemode (big ass halls with the average distance being 80m per hal, and 10 of them next to each other.) Singlemode cable is cheaper and modules are getting close to each other price wise.

And unlike what was said in this video splicing fiber is pretty much painless these days with good equiptment. I can be as fast as someone doing cat6-7. 

8 minutes ago, Lurick said:

Ah, not quite. There are 40/100Gbit BiDi optics for 100G

https://www.cisco.com/c/en/us/products/collateral/interfaces-modules/transceiver-modules/at-a-glance-c45-740242.pdf

 

Yes, but distances drop quickly with higher speeds. Tho irl any small office will be fine. 

[Lurick]

Just now, Dujith said:

I dont know where they got the price difference from? but we work mainly with singlemode (big ass halls with the average distance being 80m per hal, and 10 of them next to each other.) Singlemode cable is cheaper and modules are getting close to each other price wise.

And unlike what was said in this video splicing fiber is pretty much painless these days with good equiptment. I can be as fast as someone doing cat6-7. 

 

Yes, but distances drop quickly with higher speeds. Tho irl any small office will be fine. 

Yah, they are limited to 100m tops over OM4 with those and I know the 400G ones are still up in the air right now but will probably be like 30m

[-Old-Tom-]

13 minutes ago, Lurick said:

Nah, multimode can do 400Gbit over 100m or so on OM5 and shorter distance on OM4 but I'm doubtful on it's ability to do 800Gb+ on current OM4 or OM5 fiber. OM3 can do 100Gbit just fine for 100m and while I agree single mode is more expensive up front the reason is to not have to rip and replace since singlemode has theoretically limitless bandwidth compared to OM5 for multimode which is a much worse endeavor when you're talking about tens of thousands of cables

Yeh the cost difference in price in singlemode v multimode isn't as mush as Linus makes out in the video. the biggest cots difference is in the splicing. we basically only use single mode fibre with 10 km modules at work and the cost is pretty much the same.

the main reason there is such a price difference in the enterprise space is just the vendors maintaining their profit margins. i had a quote from HPE recently for a 10 gB, 10 km module for $2k when I could get a module of the same quality for tens of dollars. 

[Lurick]

1 minute ago, -Old-Tom- said:

Yeh the cost difference in price in singlemode v multimode isn't as mush as Linus makes out in the video. the biggest cots difference is in the splicing. we basically only use single mode fibre with 10 km modules at work and the cost is pretty much the same.

the main reason there is such a price difference in the enterprise space is just the vendors maintaining their profit margins. i had a quote from HPE recently for a 10 gB, 10 km module for $2k when I could get a module of the same quality for tens of dollars. 

That and the ZR optics for like 40 to 120km are stupid expensive too

[NZLaurence]

6 minutes ago, Lurick said:

Yah, they are limited to 100m tops over OM4 with those and I know the 400G ones are still up in the air right now but will probably be like 30m

I bet they are just pretending that the OM4 is singlemode. I have seen people use singlemode SFP's on OM4 fiber for sub 50m before.

[Velcade]

Who's in charge of site safety? Oy Vey

 

[Tristerin]

I got to mess with fiber for the first time last week.  Did a troubleshoot for the corporate job, we dont have IT on site - thats me lol, and found we had a bad SFP port on our network switches.  Switch port, fixed.  I look like a genius to the plebs (see also, non tech literate), its hilarious.

[Nystemy]

All the fiber stuffed into the wall box there in the beginning is honestly a bit unsettling...

Considering that fiber tends to not like being bent at sharp angles, generally recommended to not have a bend radius sharper than about 2 inches. In wall boxes like these, I dubt that the bend radius even reaches 1 inch. Not saying that it "won't work", but it could be a source to potential problems.

I would have personally used a whisker box instead. Or a blank plate with a hole drilled in it for the protective hose. _{(And obviously do slack handling up in the drop ceiling, though it will probably fall down over time unless one has a cable restraint.)}

[mynameisjuan]

12 minutes ago, Nystemy said:

All the fiber stuffed into the wall box there in the beginning is honestly a bit unsettling...

Considering that fiber tends to not like being bent at sharp angles, generally recommended to not have a bend radius sharper than about 2 inches. In wall boxes like these, I dubt that the bend radius even reaches 1 inch. Not saying that it "won't work", but it could be a source to potential problems.

I would have personally used a whisker box instead. Or a blank plate with a hole drilled in it for the protective hose. _{(And obviously do slack handling up in the drop ceiling, though it will probably fall down over time unless one has a cable restraint.)}

That is just best practice. SM can easily be wrapped around a pencil without issues. Its done in the field by techs when working with live fiber in the many instances of lack of labeling, documentation or poorly managed. Every two wraps tend to give -2db of loss so you can monitor ports on the blade for loss without disrupting customer traffic and common.

 

MM is about the same. It does not damage the fiber, just introducing reflection. Bends, not arcs, are more of a concern

[Nystemy]

9 minutes ago, mynameisjuan said:

That is just best practice. SM can easily be wrapped around a pencil without issues. Its done in the field by techs when working with live fiber in the many instances of lack of labeling, documentation or poorly managed. Every two wraps tend to give -2db of loss so you can monitor ports on the blade for loss without disrupting customer traffic and common.

 

MM is about the same. It does not damage the fiber, just introducing reflection. Bends, not arcs, are more of a concern

Wrapping a fiber around a pencil is something that "can" be done, but that doesn't mean that it is good for the fiber.
Long term it could cause issues. One of the main ones would be cracking of the plastic layer around the fiber and could lead to signal loss. Or delamination and creasing of it on the inside of the bend.

Best practice standards are a thing for a reason, and if one can avoid tight bends, then it is preferable.
In this situation I see little reason for just jamming the excess fiber into a tight box, when the drop ceiling has plenty of room for handling the slack.

[mynameisjuan]

6 minutes ago, Nystemy said:

Wrapping a fiber around a pencil is something that "can" be done, but that doesn't mean that it is good for the fiber.
Long term it could cause issues. One of the main ones would be cracking of the plastic layer around the fiber and could lead to signal loss. Or delamination and creasing of it on the inside of the bend.

Best practice standards are a thing for a reason, and if one can avoid tight bends, then it is preferable.
In this situation I see little reason for just jamming the excess fiber into a tight box, when the drop ceiling has plenty of room for handling the slack.

I don't think you realize how malleable fiber is and its casing. It can stretch and retract up to a point but that is smaller than the dia of a pencil. Stretching is how fiber is made. Again arcs are not the problem, bends are and wrapping it around a dia equivalent to a pencil does not introduce a bend. If you are worried about the plastic cracking, unless you are dealing with un-shielded fiber that has been baking in the sun for some time, not going to happen without some serious pulling or fast kinks. Goes with the former above.

 

Best practices are not always based on hard limits but also are there because others have gotten tired of people bending, kinking and pulling causing damage to the fiber or excessive loss and introduced a practice well beyond when it will become a problem.

 

I have plenty of fiber splicing, tracing and troubleshooting under my belt and have ran into runs bent to such an extreme where there is loss but straightening its out and tossing it on an OTDR or the scope in the splicer with little to no damage. Fiber is a lot more robust that you are making it out to be, Linus piling it in the box, while bothers me from a sheer cable management view, is not the worst I have seen and is of no concern

[NZLaurence]

@Lurick and @mynameisjuan,  Just out of curiosity, what fiber type are you guys recommending for new installs now days (both in the Office and in the DC)?

 

I have been recommending singlemode for all new installs everywhere for years now.