Mirroring windows 2012 server

[giozera]

Hello everyone!

so title pretty much says what is this post about and I have searched and couldn't find satisfying info for my project.

I got Dell poweredge 720 with 2 MD1200 DAS in Raid 10 total of 90TB space, but at this point used 52TB so far. I'm adding about 20TB data in about every 6-8 month. I have host OS and one trough Hyper-V (Win 2012 Server both) both OS's located on PCIe SSD and mirroring trough windows on the second PCIe SSD. (I know it's not a good Idea, but had no choice when I was building it) so I have to clone entire server with both host and virtual OS and all the data on the second server and in case of hardware failure on one I can manually fire up and forward load on second Mirror server. well I have done similar project before by backing up virtual drives of hyper-v and firing up on second server, but not in such a big scale of DATA. I have few Ideas and I'd like to hear your opinions and experiences about this project.

Ok mirroring OS: probably I have to create second virtual OS and move what's running on Host OS to it and sync virtual hard drives to mirror server with robocopy or something like GoodSync.

Mirroring Data drive: again using robocopy or something like GoodSync.

Syncing data trough: option a: Network trough 4 1Gigabit network ports that are in link aggregation and in full duplex mode. ( don't like this one since I think it will create lot of traffic in already busy network environment of 50 computers)

Option b: adding another 4 port network adapter (dedicated) and moving data in between trough crossover cable connection.

I don't know if I missed any details needed, I think I cover everything.

I would be very grateful to read your feedback and experiences about this project. thanks

[leadeater]

Have a look in to DFS replication, will keep everything in sync real time. I know it is likely too late now but you should have the shares presented through a DFS namespace so you could seamless change the backing real share without disruption and not having to update mappings etc. I know this may not specifically apply to what you are asking but it is a good practice to get in to doing, DFS really is your friend.

 

On to the topic, I find your question a little unclear. You are asking for advice on migrating between two physical servers that are also virtual hosts or just wanting to replicate between two physical servers that are also virtual hosts?

 

If your end goal is to have a high availability file service using two servers without shared disk then DFS-R is one of the best options you have available right now. In future coming with Server 2016 is Storage Spaces Direct which combines local server storage in to a single storage pool, but this does require 4 nodes minimum. Once the initial DFS replication is finished the ongoing network usage is very low since it can utilize remote differential compression (RDC), you can also throttle network bandwidth that replication traffic is allow to use.

 

Let me know if this is all completely off and not what you were asking for.

 

 

[pat-e]

The problem I see is the DAS (Direct Attached Storage). It would be much better if the storage would be FC so you can share the storage with 2 servers and create a Windows Cluster for File Services.

 

For Hyper-V: If you would use shared storage, then you could use Hyper-V HA and do a Live-Migration or in case of Hardware server loss, a storage motion.

 

With a DAS the best you could do is attach that storage to a smaller reliable machine, put some linux or bsd on it (like FreeNAS) and present the storage as iSCSI to both Hardware servers. With that you got the shared storage and you can use Cluster services for File-Clustering and VM-clustering.

 

The question is: do you want HA (High Availability) or just a simple "one machine is broken, replace it and restore the OS and software installed on that machine".

 

For HA, DAS is not the best idea as storage. But because you have it, you need a very reliable hardware box that can present the storage as iSCSI or FC to your Dell servers.

[leadeater]

2 hours ago, pat-e said:

The problem I see is the DAS (Direct Attached Storage). It would be much better if the storage would be FC so you can share the storage with 2 servers and create a Windows Cluster for File Services.

 

For Hyper-V: If you would use shared storage, then you could use Hyper-V HA and do a Live-Migration or in case of Hardware server loss, a storage motion.

 

With a DAS the best you could do is attach that storage to a smaller reliable machine, put some linux or bsd on it (like FreeNAS) and present the storage as iSCSI to both Hardware servers. With that you got the shared storage and you can use Cluster services for File-Clustering and VM-clustering.

 

The question is: do you want HA (High Availability) or just a simple "one machine is broken, replace it and restore the OS and software installed on that machine".

 

For HA, DAS is not the best idea as storage. But because you have it, you need a very reliable hardware box that can present the storage as iSCSI or FC to your Dell servers.

I agree with @pat-e on this one. If you haven't purchased any extra storage and the plan is to do so then consider an infrastructure design change and spend a bit more on storage and get something that is a bit smarter than just DAS.

 

In my old job we had about a thousand clients using shared SAS storage to a 2 or 4 node ESXi cluster. Hardware used was IBM DS3500/V3700 as the storage and IBM x3650 for the virtual hosts. You can do the same setup using Dell, or any other brand for that matter.

 

It is better to spend a bit more money and use shared storage so you can get the true benefits of virtualization. You will still need to get a robust backup process, mirroring isn't considered a backup either as it is part of the same failure domain/chain. Although likely more than what you will need due to deduplication the current 90TB with Veeam Backup Free Edition will do the job very nicely and if it proves to work well and you need more features you can upgrade to the paid version. 

[pat-e]

I was reading about VMware VSAN... so the use of DAS storage is not completely wrong. But still: needs need the additional Host for running the software defined SAN (or VMware call it VSAN).

[giozera]

Hello everyone!

thanks for your advice's.

the reason I don't want to use shared storage for both servers is because I have irreplaceable medical images that needs bulletproof or close to that data backup, so if something happens to that shared NAS for both servers it's game over, restoring all that images from RAW data that is stored somewhere else will take 4-6 months. so that's why I figured to have 2 separate servers with it's dedicated storage, also I think it's faster to access data from DAS than FC NAS, isn't it? since that data is constantly being accessed 24/7 and added more all the time by my opinion DAS makes more sense. pls correct me if I'm wrong.

I'm not interested with HA, I need more like manual or some automation in case of failure to redirect traffic to second server.

and I forgot to mention it's not file server I have that separately, this is more like web server with patient medical images for Dr's and medical personnel to access.

as of OS I need to mirror somehow both host and virtual OS since both running critical services and apps, so in shortest time could be launched from second server, and for that having both OS's virtualized and having images synced seems to me very practical and efficient?

again those are my ideas from my not so big experience in this matter, thanks

 

[leadeater]

2 hours ago, giozera said:

Hello everyone!

thanks for your advice's.

the reason I don't want to use shared storage for both servers is because I have irreplaceable medical images that needs bulletproof or close to that data backup, so if something happens to that shared NAS for both servers it's game over, restoring all that images from RAW data that is stored somewhere else will take 4-6 months. so that's why I figured to have 2 separate servers with it's dedicated storage, also I think it's faster to access data from DAS than FC NAS, isn't it? since that data is constantly being accessed 24/7 and added more all the time by my opinion DAS makes more sense. pls correct me if I'm wrong.

I'm not interested with HA, I need more like manual or some automation in case of failure to redirect traffic to second server.

and I forgot to mention it's not file server I have that separately, this is more like web server with patient medical images for Dr's and medical personnel to access.

as of OS I need to mirror somehow both host and virtual OS since both running critical services and apps, so in shortest time could be launched from second server, and for that having both OS's virtualized and having images synced seems to me very practical and efficient?

again those are my ideas from my not so big experience in this matter, thanks

 

 

Whether it is DAS, SAS or FC they will all be the same speed. FC is extremely low latency and has very good bandwidth, you can also combine this with MPIO (multipath) for very high throughput.

 

The replication methods you mentioned in your original post are very basic and this will not give true protection. The type of replication you are proposing will bring across any accidental deletions, corruption etc. You will potentially gain higher up time and fast recovery but not by much. Being as you mentioned this is medical data I would say this isn't adequate protection.

 

One of the problems I suspect you have is that you have extremely important data, lots of it, high need for resiliency but a proportionally small budget for these requirements?

 

You still have the option of buying smarter storage that can be connected to a single server node by either SAS or FC that supports replication, snapshots and clones. With two of these storage systems you can do hardware snapshots for recovery points and then use snapshot mirroring to the secondary storage system which is connected to it's own server node. If the primary storage or server fails for what ever reason all you need to do is promote (Clone) one of the mirrored snapshots to a live storage LUN and then start up the VM, up and running very quickly.

 

Also note with two of these storage systems there is nothing stopping you from connecting both server nodes to both of them and configuring a Hyper-V cluster. You still have independent storage, which is what you want, but if just the server node fails the VM will auto start on the other server. You will also be able to more easily do maintenance on the servers since you can move the VM live between the servers so they can be patched or hardware replaced/upgraded.

 

It is very important to not consider replication as backups either. Synchronous replication or very basic Asynchronous replication is only what it says it is, an exact replica of the primary data set as is. If the primary breaks then the replica breaks with it (not talking hardware). Asynchronous snapshot replication as I talked about is a huge step up and does provide protection and versioning but even this shouldn't truly be considered a backup, issues on the primary can still effect the secondary.

 

The backup product I mentioned in my previous post, Veeam, also has a feature called 'Instant VM Recovery'. This starts the VM instantly on the backup storage and then you can use Live Migration/vMotion to move the VM while running and in use back to the primary storage. This is only an option but a feature worth noting, it may not be viable for you.

 

Where I work we use Netapp storage (multiple C-Mode FAS8060 & multiple C-Mode FAS3220) which have all these features and we use them. I will mention Netapp simply because that is what I use but other competing brands have these features too. The Netapp E-Series sounds like it would be a good fit for what you are looking for if any of the above sounds appealing.

 

Finally and probably the best advice would be to invite a pre-sales engineer in from Dell, HP, Netapp etc and talk them through your requirements. They will come in at not cost and give you a very good idea of the options you have and give you suggestions on potential configurations. Just remember they are sales people so they are interested in making the most money they can, keep them realistic in terms of cost.

 

P.S. I specialize in storage, virtualization and backup infrastructure if you couldn't tell .

 

Edit: Also restoring 90TB of data won't take 4-6 months, we replicate around 20TB-80TB across the country every night to our disaster recovery data center. The amount depends on how much data changes each day per site. We also do hourly snapshot replication to the disaster recovery site.

[pat-e]

4 hours ago, leadeater said:

P.S. I specialize in storage, virtualization and backup infrastructure if you couldn't tell .

@leadeater, no, haven't noticed 

[giozera]

On 3/2/2016 at 2:15 AM, leadeater said:

 

oh woow that's lot of info to absorb... I have to read again to understand :-) thanks

[giozera]

@leadeater Hi

I'm not really limited with budget I can have 10-25K spent if needed but I want to make sure I'm spending it right and it's going to do what our business needs.

so I'm taking consultations form few different company's (one of them are Dell and HP) abut my project so let's see what they are going to suggest and now I have some idea what I need to hear from them thanks to you.

btw something of the topic my server acted strange I had to restart few times and it took more than hr to restart each time when DAS was connected but when disconnected boots up in a few min. I suspected that DAS disks become foreign during reboot or shutdown, when I checked with diskpart before booting to OS it was foreign, so long boot time could be explained because it was importing foreign disks, but why would they become foreign? any ideas? thanks

[leadeater]

12 minutes ago, giozera said:

@leadeater Hi

I'm not really limited with budget I can have 10-25K spent if needed but I want to make sure I'm spending it right and it's going to do what our business needs.

so I'm taking consultations form few different company's (one of them are Dell and HP) abut my project so let's see what they are going to suggest and now I have some idea what I need to hear from them thanks to you.

btw something of the topic my server acted strange I had to restart few times and it took more than hr to restart each time when DAS was connected but when disconnected boots up in a few min. I suspected that DAS disks become foreign during reboot or shutdown, when I checked with diskpart before booting to OS it was foreign, so long boot time could be explained because it was importing foreign disks, but why would they become foreign? any ideas? thanks

 

Glad I could be of some help, would be interested to hear what you end up going with.

 

That issue sounds very strange, disks in an array shouldn't go foreign on a reboot. If possible download the full logs of the RAID card (unless HBA?) and see if you can spot anything out of the norm. I would suggest firmware updates etc but unless you have a good backup I'd wait till you have your mirror server up and running in case something goes wrong. Also try logging a support ticket with the hardware vendor if possible.

 

Edit: Would also help to know how the storage is configured: Hardware RAID, HBA and Storage Spaces, HBA and software RAID etc?

[giozera]

I sow log of the controller and nothing strange battery charge events and something like that, just green light no warning or errors, even dell tech support remotely troubleshooted other day and nothing. I'm using Dell Perk H800 hardware raid 10. I did updated bios and firmware of controller too. that worries me a lot. could be controllers fault?

 

[leadeater]

1 minute ago, giozera said:

I sow log of the controller and nothing strange battery charge events and something like that, just green light no warning or errors, even dell tech support remotely troubleshooted other day and nothing. I'm using Dell Perk H800 hardware raid 10. I did updated bios and firmware of controller too. that worries me a lot. could be controllers fault?

 

Could be controller, a disk in the array or a bug between the firmware of the controller and the server bios/firmware. Usually there would be some kind of evidence of any of those in the logs of the RAID card or iDRAC/iLO events.

 

If you know 100% for certain the previous firmware of the H800 did not have the long boot/foreign disk at boot issue then I'd roll back to that. Personally I'd persist with the support case with Dell, can't find the cause of an issue isn't good enough to close a case.

[giozera]

I guess I have to call them again. thanks