Apollod

We've haven't made a call on how the files will be managed yet, we've got a few months before the grant money comes in. I know local caching can really improve the workflow on the compute server, but occasionally we deal with up to 10 files that are 8-14 gigabytes by themselves. I'm not sure how that would work on the cache. In that capacity I wonder what the performance impacts would be for direct calling. We don't use any job queues currently, its never been something we've even considered to my knowledge. I don't know much about that area. We can typically run several workloads at once with a couple notable exceptions that will easily take up our entire capacity. We currently run Ubuntu on our two servers and we work through SSH terminal to run our jobs and such. What's the overhead and benefits of the job queue? Would it prevent us from running things simultaneously?

Having all the files in one place would be amazing, I may have to make some budget magic happen though! For caching, is there any typical rule for how large the cache should be? Would it be an issue if multiple users were using the server at once?

Would doing disks this way basically all act as one machine? Could it be worth getting a separate NAS and processing server? (Possible expansions are down the road)

I must’ve missed the HP ones. Dell just doesn’t seem like we can get everything we want for the right budget. Yea the 30k sounds like a lot, but after two CPUs you’re already up to 10k

So I fairly recently started working in a research lab that works with microbial genomes. Basically it’s a ton of data processing. My boss and I talked about one of my hobbies being pc building, and he asked me if I would be interested in leading the search for a new server. With the rough budget of 30,000 USD, I pretty much agreed just for the adventure. Only thing is, I’m very new to the server space. Here’s the key specs he’s after- At least 80 threads (more would be better) At least 1TB RAM (once again, more is good) Somewhere above 150TB of storage (In a RAID array, ~200TB ideally) We already run two other servers with less storage that are 24c/48t and 40c/80t Do you wonderful people have any recommendations on where to look to either get the right parts or matching servers to the specs? Dell Enterprise and HPE just aren’t cutting it, especially since I know Epyc Rome exists. I’ve thought about either doing it all in one like the existing servers or setting up two separate processing and storage servers. What do you all think?