Jump to content

Blade Server

BladeServerBanner.png

Networking update is here!

This will be a long post with alot of images, beware.

Because of my issues with my current networking gear (ASUS N66u), I decided to upgrade to some more professional stuff:

all-boxes.jpg

Ubiquiti gear!

netgear-box.jpg

ap-box.jpg

ap-in-box.jpg

edgerouter-box.jpg

edgerouter-top.jpg

edgerouter-front.jpg

all-unboxed.jpg

Something that kind of surprised me, but that I quite liked is that both products had a QA test/sign off:

ap-qa.jpg

edgerouter-qa.jpg

I got the EdgeRouter Lite and the Ubiquiti AP-LR. And to connect them all, I got a 4 (5 including the "uplink" port) port Netgear ProSafe switch (I got it for a good price and it comes with a lifetime warranty).

Setting up the EdgeRouter Lite:

Keep in mind, this is my playing around with it while learning how to set it up. I also didn’t realise that there was a firmware upgrade until I finished, so there may be some inaccuracies.

I started by plugging into the wall, and then connecting it directly to my desktop via Ethernet. Then, I went into my windows settings and set my NIC to have an IP address in the 192.168.1.1/24 range (192.168.1.2 to be exact). Then I connected to 192.168.1.1 entered the default password and agreed to their ToS. I Then went to the “Users” tab and created another admin account for myself, logged out, then logged back in with that account. I will deleted the default account later on.

edge-router-lite-dashboard.png

Now it’s time to configure the interfaces.

Going back to the Dashboard, I clicked on “Actions” and then “Configure” for eth1

eth1-configure.png

I tried setting it to the static address of 192.168.1.1/24, but I got some errors (because eth0 is also 192.168.1.1/24). Yeah, I tried it a few times.

eth1-configured.png

eth1-errors.png

I spent a little bit of time trying to figure out how to do this, then realised that I should just give it a 192.168.2.1/24 address, and fix it later. So that’s what I did. Success!

eth1-configure-success.png

I then went to the “Services” tab (top right), and clicked “Add DHCP Server”.

dhcp-configure.png

I decided to fill it out with the following:

dhcp-configured.png

I then went to “Actions” then “Details” and am greeted with this nice looking menu showing me how many available IPs I can hand out, and how many ones I have already.

dhcp-details.png

I then disconnected from eth0, and connected myself to eth1. I also enabled DHCP on my computer so that it would grab an address from the router.

Eth0 is going to be my wan port, and my ISP doesn’t handout static address (even if they would they wouldn’t be 192.168.1.0/24 address either ;)), so I need to change that port to get it’s address through DHCP. I clicked on “Actions” and then “Config”, and change it to the following:

eth0-configured.png

Now, I kind of did something weird. I wanted my network to be a 192.168.1.0/24 network, so I went and configured eth2 as its own network (as you cannot set multiple ports to the same network), and added a DHCP server to it too. This is my backup port that I can connect to if anything ever goes wrong with my other network, and it is used to setup the rest of the ports (without worrying about changing something which would require a reset).

eth2-configured.png

I followed the same process as I did to setup the other DHCP server, but I did my range from 192.168.10.2-192.168.10.10.

configure-dhcp-details.png

After setting up its DHCP server, I disconnected my computer from 192.168.2.1, and connected to 192.168.10.1

eth3-connected.png

Now, I’m going to configure eth0 to use DHCP so that is will work with my ISP.

eth0-configured.png

Now, I’m going to go back and partially undo some of my work. I’m going to reassign eth1 to 192.168.1.1, and make another DHCP server for it.

lan-dhcp-server.png

Now everything is setup to how I would like it!

interface-config-finished.png

Next, I need to setup up a masquerade for interface eth0. You go to the “Security” tab, then “NAT”, and then “Add Source NAT Rule”. It looks like this:

wan-masquerade.png

Next I go to the “Firewall Policies” tab, two ruleset - WAN_IN and WAN_LOCAL, WAN_IN to accept, and set WAN_LOCAL to drop, both to log.

wan_local.png

Then I click “Actions” and “Edit Ruleset”, then “Add New Rule”. The first rule is for establish sessions, it looks like this.

established-sessions.png

Then you click on “Advanced” and click “Established” and “Related”. Click save.

established-sessions-advanced.png

After that, I’m adding another rule. For this one the action is set to drop, and the rest are default. Hit save.

Now, go to the “Interfaces” tab, and change the Interface to “eth0” and the Direction to “in”.

ruleset-interface.png

Now save the ruleset.

Add the same rules to WAN_LOCAL, but change the interface direction to “local”.

After doing that I used the Port Forwarding wizard to setup all my required ports.

Finally, I just need to do the basic configuration in this menu, filling in the time, the name of the device, and the domain name.

basic-settings.png

And that’s it! That’s all (took me about 4 hours, including troubleshooting and reading) it takes to setup my router.

After that I hooked the router to my modem, then the eth1 port to my gigabit switch, and my uniFi AP to the switch. Sadly, I didn't have enough proper Ethernet cables, so I am having some speed issues, and I think that there’s an issue with the switch. I've ordered some more Ethernet cables and hopefully that will fix any speed issues, if not I will be returning the switch and getting another one.

I’m not going to walk you through setting up the uniFi Aps, as it’s quite straight forward due to the setup wizard, but the basics is

Download the uniFi controller software from ubnt.com

Plug devices (computer and AP) into the same switch

Run the controller software

Follow the wizard

I also have a mini update relating to server hardware to come :)

15" MBP TB

AMD 5800X | Gigabyte Aorus Master | EVGA 2060 KO Ultra | Define 7 || Blade Server: Intel 3570k | GD65 | Corsair C70 | 13TB

Link to comment
Share on other sites

Link to post
Share on other sites

Thank God for the Google servers 8.8.8.8 and 8.8.4.4. I have them set up as well.

 

Love the depth you go into on your build log. Will have to set aside some time and re-read everything.

I do not feel obliged to believe that the same God who has endowed us with sense, reason and intellect has intended us to forgo their use, and by some other means to give us knowledge which we can attain by them. - Galileo Galilei
Build Logs: Tophat (in progress), DNAF | Useful Links: How To: Choosing Your Storage Devices and Configuration, Case Study: RAID Tolerance to Failure, Reducing Single Points of Failure in Redundant Storage , Why Choose an SSD?, ZFS From A to Z (Eric1024), Advanced RAID: Survival Rates, Flashing LSI RAID Cards (alpenwasser), SAN and Storage Networking

Link to comment
Share on other sites

Link to post
Share on other sites

Thank God for the Google servers 8.8.8.8 and 8.8.4.4. I have them set up as well.

Love the depth you go into on your build log. Will have to set aside some time and re-read everything.

They're so easy to remember too, one of the reasons why I use them.

Thanks man! :)

15" MBP TB

AMD 5800X | Gigabyte Aorus Master | EVGA 2060 KO Ultra | Define 7 || Blade Server: Intel 3570k | GD65 | Corsair C70 | 13TB

Link to comment
Share on other sites

Link to post
Share on other sites

Woahh Nice Network Update !!! Ubiquiti gear is amazing ;)

Yeah, I quite like it so far. The range on the AP-LR is quite great to say the least. When I'm on the other side of my house (approx 70 feet, through two brick walls and 4 internal walls (plaster and lath)) I get full bars on my phone (about 65-75dBm signal strength according to WiFi Analytics)).

I just need to solve the speed issue (which I think is a mixture of the old Cat 5 cables that are being used, and the netgear switch (which I guess I'll be RMAing once I verify that it's the problem).

15" MBP TB

AMD 5800X | Gigabyte Aorus Master | EVGA 2060 KO Ultra | Define 7 || Blade Server: Intel 3570k | GD65 | Corsair C70 | 13TB

Link to comment
Share on other sites

Link to post
Share on other sites

Yeah, I quite like it so far. The range on the AP-LR is quite great to say the least. When I'm on the other side of my house (approx 70 feet, through two brick walls and 4 internal walls (plaster and lath)) I get full bars on my phone (about 65-75dBm signal strength according to WiFi Analytics)).

I just need to solve the speed issue (which I think is a mixture of the old Cat 5 cables that are being used, and the netgear switch (which I guess I'll be RMAing once I verify that it's the problem).

Hope you solve ! What I did was to buy 100 m cat 6 and replace all the old wires with the exact size for my needs. 

Link to comment
Share on other sites

Link to post
Share on other sites

Hope you solve ! What I did was to buy 100 m cat 6 and replace all the old wires with the exact size for my needs.

I want to do that in the future, instead I spent $7 on cat6a cables for now.

(cat6a because they were only abit more than their 6 and 5e counterparts)

15" MBP TB

AMD 5800X | Gigabyte Aorus Master | EVGA 2060 KO Ultra | Define 7 || Blade Server: Intel 3570k | GD65 | Corsair C70 | 13TB

Link to comment
Share on other sites

Link to post
Share on other sites

BladeServerBanner.png

I've done abit of trouble shooting today, mainly around my network/samba issues, and I've gotten some peculiar results.

I tested my network running iperf, between a Crunchbang VM on my desktop and my server, and got a very weird discrepancy in speed.

Here is testing with TCP:

iperf-tcp.png

And here is with UDP:

iperf-udp.png

Quite the big discrepancy.

The other thing that was weird, is that after running the UDP iperf test, my router disrupted my connection (still connected to the network, but it wouldn't route).

Now, I tried running it over ethernet (plugged my laptop into the router) and I got this:

iperf-tcp-wired.png

 

While doing the tests, not matter what (wired or wifi), the CPU utilization never went above 6%, so it's not a bottleneck there. RAM usage is also in the low 500MBs.

So the speed issue seems to be occurring somewhere in the wifi-router-ethernet vicinity, sadly, I don't have a secondary router that I can use to test my theory, so I may just have to give it a shot and get new hardware.

 

More updates to come :)

Crunchbang <3

I make Rainmeter things and other art :D

Link to comment
Share on other sites

Link to post
Share on other sites

BladeServerBanner.png

Mini hardware update!

Along with my new networking gear, I also ordered these:

nas-drive-overview.jpg

3TB Seagate NAS drives. Newegg had them for a steal, being about $5 cheaper/GB then their 4TB equivalent, and I realised that I would eventually be getting more drives, so I might as well get them now.

nas-drive-right-logo.jpg

nas-drive-left-logo.jpg

I'm still trying to figure out exactly what I'm going to do with these drives. I'm think going to do something simple for the time being by creating a plain VDEV (essentially, JBOD) and then changing it in the future when I get/need more drives.

That update will come soon :)

15" MBP TB

AMD 5800X | Gigabyte Aorus Master | EVGA 2060 KO Ultra | Define 7 || Blade Server: Intel 3570k | GD65 | Corsair C70 | 13TB

Link to comment
Share on other sites

Link to post
Share on other sites

WOW just wow! What camera are you using? Those photos look stunning!

AMD FX 8320@ Stock - Asus M5A99X Evo R2.0 - Kingston HyperX 8GB 1600Mhz - Corsair Carbide 200R - Powercolor Radeon HD 7950 PCS+OC@970Mhz core 1400Mhz memory - Corsair CS650W - Samsung 840 EVO 250GB 
LG 22EA53VQ 21.5" - CM Storm Xornet - CM Storm Quickfire TK - Creative Inspire T3130 2.1

Link to comment
Share on other sites

Link to post
Share on other sites

I'm still trying to figure out exactly what I'm going to do with these drives. I'm think going to do something simple for the time being by creating a plain VDEV (essentially, JBOD) and then changing it in the future when I get/need more drives.

That update will come soon :)

Who knew there was such a thing as HDD porn? :D

Side note: AFAIK, you can't really do JBOD with ZFS in a single VDEV.

You could create a separate storage pool for each disk (each storage

pool consisting of a single VDEV, which in turn would consist of a single

HDD), but honestly, in that case you might as well just go with a

normal filesystem such as ext4, JFS or XFS (depending on your preferences).

Not that I'm trying to dissuade you from ZFS, but just FYI. ;)

BUILD LOGS: HELIOS - Latest Update: 2015-SEP-06 ::: ZEUS - BOTW 2013-JUN-28 ::: APOLLO - Complete: 2014-MAY-10
OTHER STUFF: Cable Lacing Tutorial ::: What Is ZFS? ::: mincss Primer ::: LSI RAID Card Flashing Tutorial
FORUM INFO: Community Standards ::: The Moderating Team ::: 10TB+ Storage Showoff Topic

Link to comment
Share on other sites

Link to post
Share on other sites

Who knew there was such a thing as HDD porn? :D

Side note: AFAIK, you can't really do JBOD with ZFS in a single VDEV.

You could create a separate storage pool for each disk (each storage

pool consisting of a single VDEV, which in turn would consist of a single

HDD), but honestly, in that case you might as well just go with a

normal filesystem such as ext4, JFS or XFS (depending on your preferences).

Not that I'm trying to dissuade you from ZFS, but just FYI. ;)

I realised after reading more that I made a mistake and it's actually RAID 0, not JBOD. So I now have a RAID 0 array.

15" MBP TB

AMD 5800X | Gigabyte Aorus Master | EVGA 2060 KO Ultra | Define 7 || Blade Server: Intel 3570k | GD65 | Corsair C70 | 13TB

Link to comment
Share on other sites

Link to post
Share on other sites

Damnit man, you take beautiful pictures :o

Thanks man! :D

15" MBP TB

AMD 5800X | Gigabyte Aorus Master | EVGA 2060 KO Ultra | Define 7 || Blade Server: Intel 3570k | GD65 | Corsair C70 | 13TB

Link to comment
Share on other sites

Link to post
Share on other sites

WOW just wow! What camera are you using? Those photos look stunning!

Thanks! :D

All my photos are taken with a Nikon D7100, those particular ones were taken using a Nikon 50mm 1.8G, though occasionally I use a tele (I don't think I have yet for this build).

P.S. Triple post = on cell phone ;)

15" MBP TB

AMD 5800X | Gigabyte Aorus Master | EVGA 2060 KO Ultra | Define 7 || Blade Server: Intel 3570k | GD65 | Corsair C70 | 13TB

Link to comment
Share on other sites

Link to post
Share on other sites

I realised after reading more that I made a mistake and it's actually RAID 0, not JBOD. So I now have a RAID 0 array.

raid 0 more speed negative redundancy?
Link to comment
Share on other sites

Link to post
Share on other sites

raid 0 more speed negative redundancy?

Maybe :P

Thinking maybe I'll set it as mirrored instead, then when I go do my other raid array I can remove the one drive and add the rest to the raidz(whatever I pick), then set the other 3TB as a hot spare or maybe use it for something else.

15" MBP TB

AMD 5800X | Gigabyte Aorus Master | EVGA 2060 KO Ultra | Define 7 || Blade Server: Intel 3570k | GD65 | Corsair C70 | 13TB

Link to comment
Share on other sites

Link to post
Share on other sites

Link to comment
Share on other sites

Link to post
Share on other sites

in a server?

Look at it again, I hit submit by accident. >.<

15" MBP TB

AMD 5800X | Gigabyte Aorus Master | EVGA 2060 KO Ultra | Define 7 || Blade Server: Intel 3570k | GD65 | Corsair C70 | 13TB

Link to comment
Share on other sites

Link to post
Share on other sites

Look at it again, I hit submit by accident. >.<

ahhhhhhhhhh

Link to comment
Share on other sites

Link to post
Share on other sites

Maybe :P

Thinking maybe I'll set it as mirrored instead, then when I go do my other raid array I can remove the one drive and add the rest to the raidz(whatever I pick), then set the other 3TB as a hot spare or maybe use it for something else.

I wouldn't really bother with hot spares TBH. They make sense when you

need maximum uptime and can't necessarily get to the machine physically

within a reasonable timeframe (say, when it's buried somewhere in a data

center), but for home use I think it makes sense only in the very rarest

of circumstances. At least that seemed to be the consensus last I checked.

If you don't have a use for that drive for now, I'd say use it as an

external backup drive.

Whatever you do though, I'd definitely advise not to stay with the RAID0

setup. ;) It's a common error though for newcomers to ZFS. ZFS is a bit

unusual in that it will always do a RAID0 stripe across its VDEVs (leaving

out caches and spares), and redundancy is managed per VDEV, not for the

entire pool at once. Top priority is always making sure the VDEVs stay

healthy.

BUILD LOGS: HELIOS - Latest Update: 2015-SEP-06 ::: ZEUS - BOTW 2013-JUN-28 ::: APOLLO - Complete: 2014-MAY-10
OTHER STUFF: Cable Lacing Tutorial ::: What Is ZFS? ::: mincss Primer ::: LSI RAID Card Flashing Tutorial
FORUM INFO: Community Standards ::: The Moderating Team ::: 10TB+ Storage Showoff Topic

Link to comment
Share on other sites

Link to post
Share on other sites

I wouldn't really bother with hot spares TBH. They make sense when you

need maximum uptime and can't necessarily get to the machine physically

within a reasonable timeframe (say, when it's buried somewhere in a data

center), but for home use I think it makes sense only in the very rarest

of circumstances. At least that seemed to be the consensus last I checked.

If you don't have a use for that drive for now, I'd say use it as an

external backup drive.

Whatever you do though, I'd definitely advise not to stay with the RAID0

setup. ;) It's a common error though for newcomers to ZFS. ZFS is a bit

unusual in that it will always do a RAID0 stripe across its VDEVs (leaving

out caches and spares), and redundancy is managed per VDEV, not for the

entire pool at once. Top priority is always making sure the VDEVs stay

healthy.

I think that I'll just use it for general storage or backup of important data (does 3 copies of data in the same building mean safer? ;):P)

Yeah, I have an update about that which I'll be posting soon ;)

15" MBP TB

AMD 5800X | Gigabyte Aorus Master | EVGA 2060 KO Ultra | Define 7 || Blade Server: Intel 3570k | GD65 | Corsair C70 | 13TB

Link to comment
Share on other sites

Link to post
Share on other sites

  • 2 weeks later...

BladeServerBanner.png

Installing and setting up ZFS

I started out by installing ZFS by using the following command:

sudo wget http://archive.zfsonlinux.org/debian/pool/main/z/zfsonlinux/zfsonlinux_2~wheezy_all.deb
To download the repository lists for it, and then I ran

 

sudo dpkg -i zfsonlinux_2~wheezy_all.deb
Essentially, this added the location of the actual program, and now I can install it using apt-get, like this:

 

sudo apt-get install debian-zfs
It only used 100MB of disk space on my computer, though installing it takes awhile. And there you go, you've installed ZFS.

Creating a Zpool

After installing it, I created a simple mirrored (RAID1) on my two new drives using the command:

 

sudo zpool create -f -m /storage -o ashift=12 storage mirror <first disks ID> <second disks ID>
to get the disk IDs run the following command:

 

ls -la /dev/disk/by-id/*
Or, better yet run

 

 cd /dev/disk/by-id
So that you can use the autocomplete (pay attention to make sure you get the right disks)

-f is the force flag.

-m is the mound flag, and will moung the VDEV to /storage after creating it.

 

-o ashift=12
Sets the sector size to 4096 Bytes (which is the standard for Advanced Format drives)

That's it! ZFS is now installed and a RAID1 pool has been created.

After creating it I then ran:

 

zfs set dedup=off storagezfs set compression=lzjb storage
To turn deduplication off and enabled lzjb compression

Finally I set the mount location of it to "/storage" with the command:

 

zfs set mountpoint/storage storage
I then copied everything over from /hdd1 (my 1TB Blue) over to /storage with the following code:

 

cd /hdd1cp -avr * /storage
It took a few minutes to transfer all 250GB over, though, after compression, it was closer to 200GB.

I then went and edited my Samba share (using Webmin), and pointed it to /storage.

I also setup a Cron job that runs every week (by saving it to the /etc/cron.weekly/ directory) to scrub the pool:

#!/bin/bash# set PATHPATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin# set pool nameZFS_POOL="storage"# start scrubzpool scrub "$ZFS_POOL"
And that's it, that's what I've done to create my pool.

By running:

 

sudo zpool status
and

sudo zpool iostat
I get these respective images showing their respective information.

zfs-storage-status.PNG

zfs-storage-iostat.PNG

There may be some errors in this, but I thought I would post it first (seeing as I'm quite abit farther behind with posts that I would like), so I'll try to correct them ASAP, and add some photos.

15" MBP TB

AMD 5800X | Gigabyte Aorus Master | EVGA 2060 KO Ultra | Define 7 || Blade Server: Intel 3570k | GD65 | Corsair C70 | 13TB

Link to comment
Share on other sites

Link to post
Share on other sites

  • 1 month later...

BladeServerBanner.png

Time for another network update!

This time, starring:

logo.jpg

My new HP V1910-24!!

box-outside.jpg

switch-overview.jpg

macro-24.jpg

macro-larger.jpg

The 1910 came a plain brown box and contained a few nice accessories like a console cable (too bad I don't have anything compatible, but this should work with my ERL iirc), grounding cable, and a extremely long power cable (I'm talking about 10 feet).

box-inside.jpg

I started by plugging the switch into eth1 on my ERL and on the first port on the 1910. I then went to my router's page to find out what IP address the switch was at so I could access the web UI.

I was greeted by a nice little log-in screen that also required me to input a captcha, pretty cool to try and help prevent a brute force attack, but I think will become abit of an annoyance in the future. After logging in (with the default credentials being "admin" and no password) I'm greeted by this nice page:

/url'>

I started by clicking on the "Wizard" tab on the left side, which brought me to this window:

wizard-main.png

Clicking through I got to a window asking for a system name, location, and a contact address.

I filled out all the information and then applied it.

Then I realise that I should probably check for a firmware update, so I went to HP's website and downloaded the latest firmware. I then went (in the Web UI of the switch) to "Device" and "Device Maintenance". I selected the file, then clicked the "If a file with the same name already exists, overwrite it without any prompt" and "Reboot after the upgrade is finished".

firmware-upgrade.png

I've read that this can take awhile, so in the meantime I had a snack.

When I returned, I realised that I probably should have used the Wizard after upgrading, as the upgrade would erase it, so I had to re-do my wizard setup.

I then went to the "System Time" tab (also under "Device") and plugged in the information so that the siwtch would get the right date and time, I then had to re-log in.

After getting the time correct, I went to the "Users" tab and created myself a user account, aswell as deleting the default account.

I then went to the "Network" tab and then the "Service" tab to disable HTTP and enable HTTPS:

service.png

And that's as indepth as I've gone so far. I'm planning on setting up QoS on it to prioritise VoIP and a few other things, but I need to dig more into the details to figure out how to do that.

15" MBP TB

AMD 5800X | Gigabyte Aorus Master | EVGA 2060 KO Ultra | Define 7 || Blade Server: Intel 3570k | GD65 | Corsair C70 | 13TB

Link to comment
Share on other sites

Link to post
Share on other sites

  • 1 month later...

BladeServerBanner.png

Mini software update here:

Setting up OwnCloud has been made pretty easy.

I started by running these commands to add the OwnCloud depository so that I can easily install it.

sudo echo 'deb http://download.opensuse.org/repositories/isv:/ownCloud:/community/Debian_7.0/ /' >> /etc/apt/sources.list.d/owncloud.list sudo wget http://download.opensuse.org/repositories/isv:ownCloud:community/Debian_7.0/Release.keysudo apt-key add - < Release.key sudo apt-get updatesudo apt-get install owncloud
From there, I visited <server ip>/owncloud, and followed the installation wizard. Seeing as I'm not going to be hosting thousands upon thousands of files, I decided to go with the SQLite database.

That's it! It's installed, and usable, except, it has none of my files in it!.

To add my files (synced from my desktop), I ran:

cd /var/www/owncloud
To be able to enter the data directory, you have to be root, so:

sudo sucd datacd <user name>ln -s /path/to/my/files "Desktop Documents"
And that's all of it. I'm done, I can now access any of my documents from anywhere! :)

15" MBP TB

AMD 5800X | Gigabyte Aorus Master | EVGA 2060 KO Ultra | Define 7 || Blade Server: Intel 3570k | GD65 | Corsair C70 | 13TB

Link to comment
Share on other sites

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×