Jump to content

Bittenfleax

Member
  • Posts

    1,992
  • Joined

  • Last visited

Reputation Activity

  1. Like
    Bittenfleax got a reaction from devtank in Mini ITX and ATX screw holes   
    @BelgiumCast what for? 

     
    It's cool, you can do it (look at pic above)
  2. Funny
    Bittenfleax reacted to hardtofindinthefuture in One computer kills entire network   
    Thow it out the window
     
    Problem solved
  3. Informative
    Bittenfleax reacted to FaultyWarrior in The Dark Arts of eBay Networking Gear   
    ***This post is work-in-progress, isn't 100% complete, and is surely littered with spelling and grammatical errors.  I'll be editing it as I put it together.  Feel free to contribute any info you see fit!***
     
    I was originally planning this as a page on my website, but after Linus's video on the cheap 10Gb gear last night (here if you've not seen it), I figured I might as well just post it to the LTT forums instead, since I'm sure there are some other network aficionados who'd benefit from it.  I'm also working on a video series on all of this for those who prefer that method of content consumption.  My channel is here, and the playlist of this stuff is here.
     
    As the title suggests, this thread is to diverge into the (apparent) dark arts of second-hand, but still very usable high-end networking gear that can be had for very little on eBay, along my trials and tribulations as I move my network from an aging CAT5e infrastructure to multi-mode fibre & CAT 7.  It's broken down by item type to make it a bit easier to navigate. {note - if people have an interest in me expanding this to other formally very expensive, but now worthless enterprise hardware like older SAN gear, let me know...I've got plenty to go off there!}

    Setup & Config:
    My network has 5 nodes - my daily driver Mac Pro workstation, my Sun Microsystem Ultra 40 M2 hardware development workstation, my PowerMac G5 "Quad" SAN filer; my Intel OEM-built backup server; and my Dell PowerEdge 2850 pfSense router.  Eventually my parent's Mac Mini, along with my XServe G5 NAS will gain NICs and be added on here as well. (they'll both just have to limp along on gigabit for now)
     
    I have the 4 copper gigabit ports of the 10Gb switch in an LACP group and connected to the first 4 ports on a 48-port Netgear copper gigabit switch, so I have plenty of gigabit ports to work with as well.
     
    All devices had Jumbo Frames enabled and set to 9000.  No other settings were tweaked since I don't fully understand what commands do what just yet (mostly CLI based with piss-poor documentation)
    If you're looking for a really in-dept look at my network layout and all of the components, see my YT channel for videos...far too much stuff to try and stuff into a post.

    Below are my first real-world testing with my hardware.  Do note that all of this testing is done using RAM drives, since outside of my 56-drive SAN pool; nothing else I own could keep up with 10 Gigabit.  If you plan to use SoftPerfect's tool that Linus recommended, you'll have to grab the older version off MajorGeeks since they changed the licensing to paid on the latest version (lame!)
     
    So - here are the results:
    NFS transfers from my Mac Pro workstation to my Sun Ultra 40 M2 workstation (Booted into CentOS 7) was a near-perfect pegging of the connection.
    AFP transfers from my Mac Pro workstation to my SAN also pegged the connection
     
    Now for the downer.  When trying SMB shares, I saw the same problems Linus saw.  From my Mac Pro to my Windows 10-based Backup Server, the transfter sputtered out at around 300MB/s.  Booting the same server into a live Linux distro and using NFS pegged the connection, so again proving that SMB is just not designed by default to handle this kind of bandwith.  While this isn't a big deal for me, since my backup server is the only Windows box on the network (Because BackBlaze nor DropBox have Linux clients!), it'll just mean that if I were to ever have to restore data from this machine that it wouldn't go nearly as fast as it could.
     
    As for Internet performance, this setup obviously didn't make a difference.  My mediocre 150/20 connection isn't going to magically get faster by upgrading my network.  I put that here because this seems to be something people are getting mixed up with Linus's videos.  LAN speed and WAN speed are VERY different things.  The exception is for those of us lucky enough to live in an area where either FTTH or MetroE is available.  Upgrading to 10Gb internally could make a difference, although the case for that is VERY limited.  Unless you have Comcast's "Gigabit Pro" 2Gb/s residential MetroE service (FWIW, I'll have it by the end of the year, so I'll be able to give feedback on that soon!), or you're able to shell out thousands of dollars a month for an enterprise MetroE connection from any of the major carriers, anything faster than gigabit won't help your internet - even Google Fiber tops out at 1Gb.  On a quick side-note - based on my readings, for those with Gigabit Pro, you actually get a 10Gb link - the Juniper access router they provide throttles you to 2Gb/s; so in the future they'll be able to provision higher speeds.
     
    Now for the actual hardware I'm running:
     
    Let's start with the switch, since it's really the first step.  While you *CAN* forgo this and do what Linus did initially - daisy-chaining machines to each-other with dual-port NICs (see video here); with switches this cheap, there isn't much of a point in this anymore.
     
    My experience is with the Quanta LB6M switch that Linus showed in his video yesterday.  I bought one for my day job about 6 months ago.  We toyed with it for a few hours, then shelved it since we needed to wait until we got our 2017 budget to make the building infrastructure changes to support it (bleh).  After my initial experiences with the one at work, when it came time to start revamping my home network, I picked up the same switch for myself about 2 weeks ago.  The switch itself is pretty awesome.  It's got TONS of options, however the documentation is abominable.  There's a few random forum threads and the manual for a different model which uses the same CLI, although it's very much an "enter command and hope it works" kind of thing.  Once you get past this and start figuring things out, it's very good...perhaps not on the level of Cisco or Juniper, but a pretty damn close second IMO.
     
    Now for the elephant in the room - the interconnect part of this.
     
    Everyone seems to give SFP+ stuff a bad rap, saying it's hard to work with and expensive.  It's a mixed bag IMO.  What ever you do, avoid direct-attached copper cables at all costs.  THOSE are expensive and a bitch to work with due to their physical thickness and large connectors.  For old hands at this kind of stuff, it's only challenger for "impossible to work with" wiring that can match direct-attach copper cabling is 10BASE5 thicknet.  The alternative (and preferred medium) is fibre optics.  On the NIC side, it's WAY cheaper than 10 Gigabit RJ45 stuff, and while not as easy to pull through a building and/or terminate as CAT6A or CAT7;  there is viable workaround to that issue.  I bought pre-made "patch cords" of roughly the correct length, and then used LC keystone jacks on both sides.  While this could result in situations where you have a random bundle of extra cable you have to hide somewhere, if you can't swing the cost to buy a fibre termination kit or to call in a professional to do it (in which case you can probably afford to just skip this thread entirely and get RJ45 stuff); it's a very viable solution for a home environment.
     
    To tag off that, pre-made fiber cables are dirt-cheap - about $1.30 (USD) per meter, give or take a few cents.  For the dozen 3 meter (a hair shy of 10 feet) cables I bought, the total came to ~$46.  For reference, MonoPrice charges ~$1.50 per meter for pre-made CAT6A cables, so price-wise you're not doing too bad.

    SFP+ optical transceivers are equally cheap - $10-$20 (USD) on average, including shipping.  Mine are a mix of Myricom and JDSU units.  All of them are short-reach 850nm units, since even my in-wall runs are at-most 30 meters end-to-end. (SR is rated for 300 meters).

    Now the NICs.  Mine are older Myricom units.  Specifically, 10G-PCIE-8B-S.  They're readily available for $60-$100 on eBay.  I chose these for not only their cost, but for their EXCELLENT driver support.  They've got drivers available for pretty much any OS from the last decade, and if they don't have what you need, you can just grab the source code package and compile the drivers yourself.  (This will come in handy when trying to get the card in my Sun workstation working under Solaris.)
     
    I haven't tried the cards under Windows XP, however I've found presentation slides that suggest it works.  Once I get XP loaded on the Sun workstation I'll give it a shot and post my results; although outside of fringe cases like mine where I use XP for its compatibility when doing hardware development and debugging, I don't think anyone else will be realisticlly still running it at this point; especially in an instance where they'll want a 10 Gigabit connection to said machine.
  4. Funny
    Bittenfleax reacted to TubsAlwaysWins in Ubiquiti Unifi Pro Access Point   
    Get the injector as close to the modem as you can. Just make sure the cable isn't plugged in to it if you need to cut the other end. (Got some sparks. It was cool)
  5. Like
    Bittenfleax reacted to tlink in Quantum computing is real, and D-wave just open-sourced it   
    read the posting guidelines before posting. you also can't open source quantum computing, quantum computing is a concept not a product. they open sourced their programming software for their line of quantum computers.
  6. Like
    Bittenfleax reacted to leadeater in Best way to set up subnet (best practices)   
    Have a look at the Ubnt ES-16-XG, it's the most basic 10Gb switch you'll find with that many ports at such a low new cost.
    https://www.ubnt.com/edgemax/edgeswitch-16-xg/
     
    If you want anything better buy used, used may even be better,
  7. Funny
    Bittenfleax reacted to Falconevo in Dead thread   
    P.S just replace the WG software with pfSense on the Firebox device and tell your networks team to deal with it
  8. Agree
    Bittenfleax reacted to vorticalbox in my computer is running slow lately any reasons why this may be happening?   
    Just because something is not good doesn't mean you have to reinstall windows also, if you're running windows 10 you can reset it while keeping all your files. Backing up your files is something you should do anyway not just for troubleshooting. 
     
    @donggun222 use task manager to see what process is taking up the disk could very will just be windows updates.
  9. Informative
    Bittenfleax got a reaction from donggun222 in my computer is running slow lately any reasons why this may be happening?   
    Have a look in task manager at the memory and CPU usage. 
     
    Also look in resmon.exe (find it in the performance tab of task manager) at the disk usage (on Windows 7 - disk usage should just be in task manager on Windows 8-10). See if it is abnormally high.
     
    Also check if any virus scans are running in the background. They slow down systems a lot.
  10. Informative
    Bittenfleax got a reaction from darkwizard06 in PFSense per IP Bandwidth Allocation   
    First of all, you need to create an Alias for each user on DHCP. This will ensure that you always have an "identifier" name for that IP/Machine.
     
    The page looks like this and the location can be seen at the top.

     
    Next, you have to create a limiter. Well, you have to create two. One for download and one for upload. Like so... Make sure that the Enable limiter and its children is checked.

     
    Then create a firewall rule, with the protocol as any, the Source as your alias and then at the bottom with advanced settings, where it says In/Out pipe, put in your limiters. Like so...


  11. Like
    Bittenfleax reacted to darkwizard06 in PFSense per IP Bandwidth Allocation   
    Thank you very much for this. I really appreciate it. I will follow your suggested steps in configuring my box.
  12. Funny
    Bittenfleax reacted to porina in UK Mass Surveillance Law has just passed the Parliament's Approval   
    TunnelBear on standby...
  13. Agree
    Bittenfleax reacted to suicidalfranco in UK Mass Surveillance Law has just passed the Parliament's Approval   
    the UK didn't need the EU to screw them, they already do it very well on their own 
  14. Funny
    Bittenfleax reacted to HarryNyquist in UK Mass Surveillance Law has just passed the Parliament's Approval   
    high risk alert: user is collaborating with ISIS through encrypted channels
  15. Like
    Bittenfleax reacted to leadeater in weird but cool cooling solution for rack servers   
    This is actually something I am planning on doing. Forget the aircon/chiller as part of the loop as that won't work, not well enough to be of any benefit and would cost WAY too much.
     
    There are two ways this is done in the industry now:
    Rack water cooling Direct to chip (much like what we do on our desktops) Rack water cooling circulates water through a mostly sealed rack where the heat from the servers is exchanged in to the rack itself and the water then moves the heat away to a passive cooling unit/tower outside the room.
     
    Direct to chip is a full purpose built design and uses drip-less quick disconnects on the servers to plum them in to the cooling loop, more efficient but more complicated and costly and much higher risk.
     
    http://www.asetek.com/data-center/oem-data-center-coolers/rackcdu-d2c/
     
    There are other factors to be aware of too. You have to be careful of things like hard drives as enterprise disks run very hot and if you reduce the airflow in the server you can induce rapid failure of these, and potentially other components of the server. It is generally a bad idea to convert an air cool server to water cooled as rack servers are very highly optimized and engineered, messing with that can turn out very badly.
     
    What I personally plan on doing is Direct to Chip using custom built servers with custom cases and at the top of the rack will be a 4U/5U rackmount chassis that will contain all the pumps and radiators. The rack will be sealed at the back and split sealed front to back in a way that air must flow through the servers to get to the back and air can only exit the top of the rack (cannot pass back to the front). I'll then duct the air directly out of the room. This is only a 1 or 2 rack solution but that is all I'll ever personally need.
  16. Funny
    Bittenfleax reacted to rn8686 in New Ukrainian Sound-Homing Rocket Launcher Appears to Use Raspberry Pi   
    How to make a missile only from electronics; Raspberry Pi + Samsung Note 7 
  17. Funny
    Bittenfleax got a reaction from Danielh90 in where to keep my server   
    Buy her some ear protection  
  18. Agree
    Bittenfleax got a reaction from Thermite in pfsense cache storage size   
    Depends on the amount of users and the diversity of sites they visit. I have a 120GB SSD in mine and has barley use any. See the image of my usage:

  19. Like
    Bittenfleax reacted to Edelrat in Use pfSense as Adblock   
    No worries  I am trying to do more for fun and experience than use aswell..
    If you have success, or get to know anything new about it, let me know!
  20. Like
    Bittenfleax reacted to foobar42 in Use pfSense as Adblock   
    Using IP-based firewall rules to block ad servers is not very efficient, since there are already many hosts lists out there to block ads. I'd recommend using https://github.com/StevenBlack/hosts
    You block them by running a caching DNS-resolver on your pfsense box, which by itself will speed up your browsing experience, and use that blocklist as an additional hosts file. Make sure to enable DNSSEC while you are at it, and make sure you upstream your DNS requests to a fast DNS-Server which is not from your ISP and provides DNSSEC and doesn't do NXDOMAIN hijacking. 
    You'll also need to make sure that you are announcing the local DNS-Server via DHCP to the clients and that your firewall policies allow your local clients to reach it.
  21. Agree
    Bittenfleax got a reaction from BiscuitMassacre in Router's DNS not working   
    Log onto the routers (normally 192.168.1.1 and I guess 192.168.1.2 for the second but I imagine it is just an access point so you should only have to edit one) and look at the DNS settings. Set it to 8.8.8.8 which is Googles DNS severs.
  22. Like
    Bittenfleax reacted to Edelrat in Use pfSense as Adblock   
    I will give it a try later, I don't have time right now
    It's the same for me, I am the only one in my network..
     
    I will report back later!
  23. Agree
    Bittenfleax reacted to VerticalDiscussions in successor to the Hubble Telescope is ready - the James Webb Space Telescope   
    I love space technology so much. The exquisite and advanced composite materials, design of the aircraft, construction process and the stability and reliability of these floatcrafts is so freaking amazing and mindblowing. Then again, so is our universe ^^.
  24. Funny
    Bittenfleax reacted to ThinkWithPortals in Nvidia adds telemetry to Geforce experience its latest drivers (Update)   
    *looks to right through case window at RX 480*
    *looks at AMD Crimson icon in taskbar*
    *smiles*
  25. Agree
    Bittenfleax reacted to SansVarnic in Valve makes game dev advertised with in game screenshots   
    Finally... I am so tired passing on games because I could not see what it actually looked like. In game shots are soo much more appealing and helps me decide on the purchase.  
×