MartinIAm

I have a gigabit switch (24 port) and a 10 gigabit switch (24 port) and I’m thinking to do a “high” speed storage system. I have a slower hdd system for storing the files (videos plus pictures) then I have a cache server (all sas ssd with ram cache and possibly optane cache down the line). Those will be communicating with each other, then I have two compute servers that will be also communicating with those two storage servers. Is dual 10gb fine for my storage servers or should I get a 40gb switch. If I did get one they only cost around $100 eBay for a 36 port I think (last time I checked). EDIT: I will also be adding more compute servers down the line.

And maybe also lowering the quality of the video if they aren't directly watching it would help.

What is the speed of your internet? I would say (for me) it uses around 64 kilobytes per second and doesn't use that much of my downstream pipe. Also, what streaming service are you using for music? If it's without video then it shouldn't use too much, but if it's also streaming video then that could be the problem.

Haha, not manually. There is a disk caching service built into the database program I use and probably also ram cache. Also my idea of this is lets say somebody makes an account and uploads an avatar that is 2mb big and never logs on again and nobody else sees that avatar ever again for another year. In a big project this will happen a lot and it's just better to store it on something that isn't as costly per gb as ssds. Let's say 100,000 people are viewing the same page and more keep loading in and loading the page and reloading it, that would be best on ram, then 5,000 people are viewing this other person's page over 5 hours this would either work on ssd or on ram, if 100 people view one person's account in 10 hours it would be "ok" to put it on the ssd for 24 hours, if 1 person views one person's account in half a year then I would store it on the ssd for 24 hours and once a day has by then it puts it back into the main storage. Quite simple.

i agree with you on this one. I think it's time that hdds are getting outdated and that people should use ssds more, which is something good to keep in mind. But I was thinking to get a 2u server put half on ssd and the other half on hdd and when a user loads an avatar that avatar gets cached for like 24 hours and then gets moved back into the hard drives. Maybe also have a backup server also for the every avatar that is only SATA hdds possible even 5400rpm. I also already have a 10gb switch and a 10gb sfp+ network card is pretty cheap.

This looks pretty good and yea for the price it seems really good, but for just one drive I would do raid 1 and get another nvme drive with that. This should work for most people, but for me buying an enterprise server would be better because it has simply better reliability, even for storing something as simple as avatars. Probably for me it would work best with 15k or 10k drives with 600gb or more of storage per drive and put them in a 16 drive chassis, split them up into 2x raid 5 groups cloning each other to have better reliability. Also go with maybe 128gb of ram or more per server for cache because they are hard drives. I like to host everything myself and buying used older servers is just a better deal in the end for me because I already have an infrastructure built, and virtual private servers just blow my whole idea of not using cloud services. Using electricity for me isn't a problem as electricity is relatively cheap where I live (about 6 cents per kw) and I also have solar panels and a generator which means I can stay off the grid for a while. Also, my internet kinda sucks right now (250mbps down 15mbps up), but I'm planning to get a fiber line within the next year so that isn't a concern for me. In the end, as I said I will be just putting a bunch of 10k or 15k (or maybe even 7.2k rpm cause it's cheaper per gb) 2.5 inch sas hdds with 600gb or more per drive in 2x raid 5 arrays with 2 different raid controllers (clone) for maximum redundancy in a 16 drive chassis (maybe Dell r720xd) and use 128gb ecc of memory or more for cache per server and run like that. Currently, I'll start with one server and then expand in the future to more and use load balancing for avatars.

Hello, I am working on a project and I want to make a storage server, that will hold the avatar (png, jpeg, etc.) of the users that signup. Each avatar has a max of 2mb size limit. Should I use 2.5 hard drives, if so what speed (7.2k, 10k, 15k(sas btw)), or should I use cheaper enterprise sas ssds for this because they can access any data at almost anytime? Also this will most likely be in raid 5 or 6 (If raid 5 they will be split into groups of drives) so I think that gives a read speed advantage. What should I choose?

I'm just gonna say, the faster the read speed of the drive, and the faster the cpu the faster Windows 10 will load. If you have a SATA drive with 800mb/s read speed the M.2 ssd with 3500mb/s will load Windows 10 faster. Cheap SATA ssds cannot compare to high-end consumer NVMe ssds in loading and boot times. They are not "spitting lies" because it is true, and the limitations of your speeds could be because you don't have your system correctly setup, or don't leave enough pcie bandwidth for the ssd. I use a WD black NVMe drive and I see my computer load 15-25 seconds faster than the previous SATA ssd I owned (Samsung 840 evo), and loading times are significant. SATA gen. 3 is simply slow and M.2 ssds were not designed because there was no cables (at least not the main point), but to have better speeds than SATA.

This depends on the type of NVME M.2 ssd you are using, if it's incredibly cheap it's gonna perform around the same as a SATA ssd. If your using a very high quality one, you can see speeds of 3gb or more per second. Boot times are dependent on a lot of variables and not just your ssd, like your cpu and motherboard, and you can improve this by turning on fast boot or ultra fast boot if you are using Windows. Loading times is also faster in larger sized files. There are a lot of variables that go into your experience of an NVME M.2 drive. If you have multiple virtual machines on your computer with many iops, then these nvme M.2 ssds are a big life saver. Regular ssds use SATA (I hope that is what your talking about) and SATA has a speed limit of around 800megabytes per second, whereas M.2 Gen4 speeds go up to 4.95GB/S, so no it is not just about the "smoke and mirrors". I mean, in the end it all depends in your situation. If you just browse the internet you might not see a big difference, but if you are doing something like 8k video scrubbing, then it really matters to have a fast ssd. Also, if your loading big games like GTA V then you will see a performance boost of load times. Sometimes the extra money is worth it, but if you don't really care about loading times of bigger games and don't want to spend money on NVMe drives, then don't because for some people they will only see the performance boost in some rare situations.

Just to confirm, do you use Mongo in config/storage.yml?

I would recommend to see if all fans are still spinning, if you want you can check the bios for this, but you can't check it through your bios for your psu so make sure that fan is also spinning.

What graphics card are you using?

If this thing was as loud as a gunshot then it's probably something serious because even a psu that has gone bad has made that kind of noise.

Yes, I will be running this inside of a server room with other loud equipment so that shouldn't be a problem for me. EDIT: I have bought and received the switch and yes, it is loud. At boot is is very loud, but even after it calms down it is my personal loudest switch I've ever owned, BUT I am fine with this as I'm not concerned about noise level and it was a good value.

Yea, I'm not planning to do anything too advanced with it anyway other than to use it's switching capacity and simple layer 3 functionalities.

You can dual boot it and it will be kinda hard and will work as long as you have an Intel cpu. If you already have macos on the computer you can install bootcamp, but if you have Windows 10 already installed there are lots of useful guides out there that are not too hard to follow and you can do it pretty easily, BUT you must have either one, have a mac and create a bootable macos usb installer OR buy an older version of macos directly from Apple like OS X Lion for $20 and follow a guide with that and once you have fully installed OS X Lion then you can upgrade to something newer like High Sierra or Mojave.

It shouldn't matter because your pcie cards are using x1 and if you plug it into a x16 it will still only using x1 because it has that many pins. It shouldn't make a difference to your main x16 slot though.

How can I let the my user upload their picture, then with jcrop the user can crop their image to the designated ratio that I have provided (i.e 4:3). I am using active storage for this currently. I am also using Ruby version 2.6.3 and rails 6.0.2.1.

Would the Arista DCS-7124SX-F be a good switch for 10 gigabit? I found it on eBay for $250 and I think for 24 ports of sfp+ is really good. Is $250 for a 10 gigabit, 24 port, layer 2/3 switch a good bang for it's buck? Planning to use it for my dual server setup for fast data transfer across them and multiple other computers on the network. What do you guys think?

AMD will perform better and it will be a better bang for your buck. Also the with the x570 board you’ll have pcie gen 4 if you want to add fast storage later on.

I have an app where it displays the user's avatar but instead of cropping it, it stretches the image to fit inside the 400px by 400px image. How can I fix this so it doesn't stretch it but instead centers the image to use a 1920x1080 image that is cropped to 400x400

Oh! I think I understand now. So I can use mongodb as storage for active storage, and the use something like NFS to have the same replicated project across the multiple servers.

I didn't know you could use mongodb for storage.

I don’t run it off the cloud as I am self hosted so kubernetes and ansible isn’t a viable option for me but docker does sound like an option. You also didn’t explain how you manage your storage across the multiples virtual machines.

I see. I was gonna use gsuite’s relay to deal with the blacklist, but the IP address changing was a problem that I kept in mind. I use a dynamic ip currently so it most likely won’t work anyway. And as schizznick and beerskyin’s said they mostly likely block incoming ports which also would not make it an option. Renting a vm for me is a option for me, but I tried doing this so I can get as much storage as I please at my own cost instead of paying someone like amazon per gb of storage. I think I might stay with gsuite for now until I switch to a business account or get a static ip and ask for incoming ports to be opened for me.