Cluster computing for some reason or other

[H0R53]

So I just bought 6 HP Elites with an i5 650. No HDDs and they have 2GB of single channel RAM (single 2gb stick).

 

I want to cluster them.

 

How do?

 

p.s. the one closest to the camera is an i5 3470, 8GB RAM and a 500GB HDD. Gonna put a 1050tiLP in.

[Sir Asvald]

You could use them as VMware servers. Cluster them.  Get more ram

 

 

[porina]

If you want multiple PCs to work on the same task at the same time, good luck. Well, you're gonna need to really want to do it as a learning exercise, as I can't think of any "normal" applications that could suit this. In the past I had looked at beowulf clusters. Then you need to have or write software that can allow the machines to talk to each other while working on the same task. Imagine multi-thread, but a whole lot more complicated and lower performing, in that scaling will be even more difficult. I'm thinking linpack might be a good test starting point to get going as that's what they use to rank supercomputers.

 

There's the not-quite cluster of having them independently run a small part of a bigger job, which is distributed computing. This is much easier, but unless you have a personal goal behind it, it doesn't have the same potential value.

 

For the performance, it is possible an old dual xeon server would have given similar ball park to all these combined.

[H0R53]

18 minutes ago, CUDA_Cores said:

sell them and buy a single PC. It would be faster, consume less power and be less complicated to set up. 

 

computer clustering only works for highly specialized workloads, so forget about running google chrome across 7 different machines as it just wouldn't work. The reason you see the enterprise market doing cluster computing is because they have highly specialized needs that can only be satisfied with multiple PCs and very special software. trust me mate, it is not worth your time (I have tried it). You will spend days trying to figure it out and rip almost al your hair out over this. Just sell off those old PCs and buy a single very fast one. 

 

Of course, there are exceptions. What do you plan to actually run on this cluster?

 

16 minutes ago, Abdul201588 said:

You could use them as VMware servers. Cluster them.  Get more ram

 

 

 

9 minutes ago, porina said:

If you want multiple PCs to work on the same task at the same time, good luck. Well, you're gonna need to really want to do it as a learning exercise, as I can't think of any "normal" applications that could suit this. In the past I had looked at beowulf clusters. Then you need to have or write software that can allow the machines to talk to each other while working on the same task. Imagine multi-thread, but a whole lot more complicated and lower performing, in that scaling will be even more difficult. I'm thinking linpack might be a good test starting point to get going as that's what they use to rank supercomputers.

 

There's the not-quite cluster of having them independently run a small part of a bigger job, which is distributed computing. This is much easier, but unless you have a personal goal behind it, it doesn't have the same potential value.

 

For the performance, it is possible an old dual xeon server would have given similar ball park to all these combined.

 

 

What about CPU tasks? I was thinking about mining but that's pointless since Ethereum is already dead and mining is stupid and pointless anyway. Don't want to be a hypocrite!

 

I know RPi's can be clustered.

 

In all honesty I was gonna buy some bulk RAM and HDDs, slap them in, and sell them for ~$150 each, maybe $200 with a GT 720 or similar GPU.

[Mdgtman91]

You could make a SAN by clustering the storage. No real benefit other than how cool it would be to do and learn. If you are interested, look into using something with centOS like gluster or something. Sometimes its just fun to make ridiculous things.

 

Oh and I think someone mentions vmware, look into Xenserver. You can cluster that and then have a virtual playground for poops and giggles.

[H0R53]

2 hours ago, Mdgtman91 said:

You could make a SAN by clustering the storage. No real benefit other than how cool it would be to do and learn. If you are interested, look into using something with centOS like gluster or something. Sometimes its just fun to make ridiculous things.

 

Oh and I think someone mentions vmware, look into Xenserver. You can cluster that and then have a virtual playground for poops and giggles.

Hmmmmmmmmmmmmmmmmmmmmmm

[hiperbolt]

You shoud look into Plan9 clusters https://9p.io/plan9/

Wouldn't have much practicle use unless you want to do a very specific task.

[Jarsky]

If you're talking about parallel compute, this isn't really done at the OS level - it's done by specialised applications.

If you're talking about failover clustering, then again your configuration depends on what you're running - Windows Server has a failover cluster manager and many apps are clustered within this to provide uptime.

If you're talking about resource sharing/load balancing, then something like VMware allows you to build datacentres. You do need a full vcenter license to get access to these features though - but keep in mind, this isnt parallel processing for a single VM - VM's are vmotion'd (migrated) between hosts based on policies and resource availability.

[System Error Message]

power use not so much, you can have the PCs sleep and wake them up for tasks. You can cluster them for some tasks though, depends what you want.

The most 2 common are storage and compilations (yes code compilation can be intensive, hence distcc exists). Some OS applications support clustering and fileover.

[H0R53]

7 hours ago, Jarsky said:

If you're talking about parallel compute, this isn't really done at the OS level - it's done by specialised applications.

If you're talking about failover clustering, then again your configuration depends on what you're running - Windows Server has a failover cluster manager and many apps are clustered within this to provide uptime.

If you're talking about resource sharing/load balancing, then something like VMware allows you to build datacentres. You do need a full vcenter license to get access to these features though - but keep in mind, this isnt parallel processing for a single VM - VM's are vmotion'd (migrated) between hosts based on policies and resource availability.

Tell me more about VMWare clustering with datacenters.

[Oxydoreduction]

51 minutes ago, H0R53 said:

Tell me more about VMWare clustering with datacenters.

There's a guy who did that and explained some of his hardware use:

 

http://www.cheesyboofs.co.uk/esxi-vsan-microserver-homelab

 

Bear in mind that VMWare vcenter's price is... not far from $9.000

[CFstorm]

On 7/11/2017 at 4:28 PM, H0R53 said:

So I just bought 6 HP Elites with an i5 650. No HDDs and they have 2GB of single channel RAM (single 2gb stick).

 

I want to cluster them.

 

How do?

 

p.s. the one closest to the camera is an i5 3470, 8GB RAM and a 500GB HDD. Gonna put a 1050tiLP in.

You could make a storage cluster. Try glusterfs: https://www.gluster.org/

[Kered124]

+1 vote for making a gluster cluster- Good learning project. 

[H0R53]

On 7/12/2017 at 4:50 PM, Oxydoreduction said:

There's a guy who did that and explained some of his hardware use:

 

http://www.cheesyboofs.co.uk/esxi-vsan-microserver-homelab

 

Bear in mind that VMWare vcenter's price is... not far from $9.000

Is that nine dollars or nine thousand with a period instead of a comma?

[BuckGup]

Use rocks cluster 

[System Error Message]

its over $9000

many many things can be clustered and there are useful daily things that benefit from it.

I'll list a few here that you can try.

File storage (various mentioned)

openMP (C++ network API)

distcc (compiler)

Single image OS (like pelican OS) that makes the cluster appear as 1 machine but intolerant to changes.

AI learning

virtualisation

firewall with IDS (your router will communicate with various machines sending traffic to them for analysis, if suspicious it will tell the router to block) - only works with configurable routers

Cloud stuff (steam files, local file syncs, etc)

 

You can use these for various things. If you are a programmer you could create your own clustered, multi threaded physics simulation for example, better start learning to code.

 

 

[H0R53]

1 hour ago, System Error Message said:

Single image OS (like pelican OS) that makes the cluster appear as 1 machine but intolerant to changes.

 

Tell me more please

1 hour ago, System Error Message said:

firewall with IDS (your router will communicate with various machines sending traffic to them for analysis, if suspicious it will tell the router to block) - only works with configurable routers

 

This is also very intriguing.

[System Error Message]

Just now, H0R53 said:

Tell me more please

This is also very intriguing.

just google pelican OS, i dont think its still being worked on but worth a try. Its a linux OS and as i said is intolerant to change so you just cant turn off a PC in the cluster.

 

suricata is an example of an IDS that can be used with a configurable router. To use multiple PCs, just have your router send each traffic to a different PC in some form. For example you could set each connection to be sent to each PC in series.

[H0R53]

5 minutes ago, System Error Message said:

as i said is intolerant to change

What exactly does this mean?

[System Error Message]

Just now, H0R53 said:

What exactly does this mean?

meaning you cant add to or remove from the cluster while the OS is running.

[H0R53]

Just now, System Error Message said:

meaning you cant add to or remove from the cluster while the OS is running.

Oh I don't have a problem with that