Proxmox multi-cluster use

[igorrocks]

 

Has anyone used Proxmox VE in multi cluster environments? I am hard set on using OpenStack, cloud engineers advocate for Proxmox VE multiple cluster environment setup. Justifying it with deployment time. They suggest to rollout 250 servers in 50 nodes clusters:

1. Ceph and CephFS

2. Compute

3. GPU

4. PBS

5. PMG

Network fabric is 25 Gbps with  OpenSENSE server router 

GPUs use Nvidia 400 Gbps network cards and equipment. I am sceptical, because I remember that Proxmox VE had 32 nodes communication limit. Is it really 50 nodes limit now? Or engineering is taking network backbone into account? They say that it’s going to be 4 times more powerful that any Edge DCs available in Canada + we are not vendor lock. 

[Electronics Wizardy]

This really calls for hiring someone who is experienced with clusters and your use case. This is beyond what a forum should really be helping with.

 

There shoudln't be a limited for cluster size in Proxmox, so I don't think that's a issue here.

 

I'd be tempted to rent some cloud nodes to run tests on so see how well different setups would work.

[igorrocks]

53 minutes ago, Electronics Wizardy said:

This really calls for hiring someone who is experienced with clusters and your use case. This is beyond what a forum should really be helping with.

 

There shoudln't be a limited for cluster size in Proxmox, so I don't think that's a issue here.

 

I'd be tempted to rent some cloud nodes to run tests on so see how well different setups would work.

My reason for asking is because LMG team use Proxmox VE in production environment. Maybe they cab advise on this. Plus Linus can actually do a test for everyone. 

[igorrocks]

57 minutes ago, Electronics Wizardy said:

This really calls for hiring someone who is experienced with clusters and your use case. This is beyond what a forum should really be helping with.

 

There shoudln't be a limited for cluster size in Proxmox, so I don't think that's a issue here.

 

I'd be tempted to rent some cloud nodes to run tests on so see how well different setups would work.

Once we rollout our environment you more than welcome to test it. I can PM you website, if you wish.

[LIGISTX]

1 hour ago, igorrocks said:

My reason for asking is because LMG team use Proxmox VE in production environment. Maybe they cab advise on this. Plus Linus can actually do a test for everyone. 

I don't think LMG is going to roll out a 250 server deployment to test for the "fans", or the community. Thats a hell of an infrastructure deployment... 

 

I agree with @Electronics Wizardy, this is something you need to pay a consultant to help with. The cost of the consultant will be a fraction of the price of a 250 server deployment, so its probably worth getting it right from the start, and having a resource to call when things don't work as intended.

[StDragon]

Lots of orgs jumping from VMWare over to Proxmox or Nutanix. Either way, be sure you have a proper backup solution as part of the project scope.

Veeam supports both, but read the documentation to ensure proper integration with Proxmox. Beyond that, yes, pay a consultant for enterprise level consultation and deployment.