Jump to content

Cluster Computing Tutorial (Request)

Little Bear
By Little Bear
in Servers, NAS, and Home Lab
 Share
Go to solution Solved by Little Bear,
8 minutes ago, Electronics Wizardy said:

Ha Ha me again

 

The thing with clusters is that the software the master runs is normally very depend't on exactly what you want to do on the nodes and why your making a cluster. Normally your running a cluster for 1 of 2 main reasons, or a combo of both. 

 

1. HA(high availability. This is when you have 2 or more servers running the same thing at once, and only one is used at a time, with the other nodes seeing if the main one is running, and if it fails, they will take the same ip and config, and can switch in about 100msec, so you won't notice any downtime. This is mainly used for things where uptime is very imporant, like the stock exchange. The easy way to do this is heartbeart on linux or drbd.

 

2. High performance. This is when you split a task into smaller ones to have work on little chunks. This is used for things like rendering, scientific computations, VM's. THe controller depends on the exact task, but something like a beowolf cluster can be used to shove work out to a ton of slaves with a variable value, and have them work on that chuck. 

 

 

If you want to do any of this you first need a task to cluster.

Thanks for the response, but I've already set it up.

Posted

Does anyone know how to setup a cluster computer?

Motivation is where, and what you make of it.

 

“It is relatively unusual that a physical scientist is truly an atheist. Why is this true? Some point to the anthropic constraints, the remarkable fine tuning of the universe. For example, Freeman Dyson, a Princeton faculty member, has said, ‘Nature has been kinder to us that we had any right to expect.'”  —Albert Einstein

Link to comment
https://linustechtips.com/topic/709411-cluster-computing-tutorial-request/
Share on other sites
Link to post
Share on other sites
Posted
  • Author
3 minutes ago, Electronics Wizardy said:

Clusters are very dependt on what you want to do?

 

What do you want to compute?

Boinc, and things like that.

Motivation is where, and what you make of it.

 

“It is relatively unusual that a physical scientist is truly an atheist. Why is this true? Some point to the anthropic constraints, the remarkable fine tuning of the universe. For example, Freeman Dyson, a Princeton faculty member, has said, ‘Nature has been kinder to us that we had any right to expect.'”  —Albert Einstein

Link to post
Share on other sites
Posted
  • Author
Just now, Electronics Wizardy said:

boinc already has a manager for clustering. You just install it on all of the systems, and run it. BOINC is managed by a external server.

I didn't mean the software setup, more so the hardware setup.

Motivation is where, and what you make of it.

 

“It is relatively unusual that a physical scientist is truly an atheist. Why is this true? Some point to the anthropic constraints, the remarkable fine tuning of the universe. For example, Freeman Dyson, a Princeton faculty member, has said, ‘Nature has been kinder to us that we had any right to expect.'”  —Albert Einstein

Link to post
Share on other sites
Posted
1 minute ago, Little Bear said:

I didn't mean the software setup, more so the hardware setup.

You can run boinc on almost anything these days.

 

What specifically are you trying to do? Normally running BOINC isn't worth it.

Link to post
Share on other sites
Posted
  • Author
1 minute ago, Electronics Wizardy said:

You can run boinc on almost anything these days.

 

What specifically are you trying to do? Normally running BOINC isn't worth it.

I know. I'm trying to run boinc on a cluster computer. Its for a self-assigned project, I don't care if its not worth it.

Motivation is where, and what you make of it.

 

“It is relatively unusual that a physical scientist is truly an atheist. Why is this true? Some point to the anthropic constraints, the remarkable fine tuning of the universe. For example, Freeman Dyson, a Princeton faculty member, has said, ‘Nature has been kinder to us that we had any right to expect.'”  —Albert Einstein

Link to post
Share on other sites
Posted
Just now, Little Bear said:

I know. I'm trying to run boinc on a cluster computer. Its for a self-assigned project, I don't care if its not worth it.

A cluster is just a lot of computers. You can use any system together. Just get some systems, install boinc, and boom cluster.

Link to post
Share on other sites
Posted
  • Author
1 minute ago, Electronics Wizardy said:

A cluster is just a lot of computers. You can use any system together. Just get some systems, install boinc, and boom cluster.

Please stop answering.

Motivation is where, and what you make of it.

 

“It is relatively unusual that a physical scientist is truly an atheist. Why is this true? Some point to the anthropic constraints, the remarkable fine tuning of the universe. For example, Freeman Dyson, a Princeton faculty member, has said, ‘Nature has been kinder to us that we had any right to expect.'”  —Albert Einstein

Link to post
Share on other sites
Posted
  • Author
1 minute ago, Electronics Wizardy said:

No.

 

What do you want to do?

Do you have a superiority complex?

Motivation is where, and what you make of it.

 

“It is relatively unusual that a physical scientist is truly an atheist. Why is this true? Some point to the anthropic constraints, the remarkable fine tuning of the universe. For example, Freeman Dyson, a Princeton faculty member, has said, ‘Nature has been kinder to us that we had any right to expect.'”  —Albert Einstein

Link to post
Share on other sites
Posted

In most cases, you aren't explicitly connecting a bunch of computers together except with your standard network connection.

 

Essentially you're just installing software on each machine that lets you distribute your work load. Folding and boinc are examples of this but generally it is used for a specific task such as running simulations.

 

That's why @Electronics Wizardy was specifically asking you what you wanted to do as it depends entirely on the software used.

[Out-of-date] Want to learn how to make your own custom Windows 10 image?

 

Desktop: AMD R9 3900X | ASUS ROG Strix X570-F | Radeon RX 5700 XT | EVGA GTX 1080 SC | 32GB Trident Z Neo 3600MHz | 1TB 970 EVO | 256GB 840 EVO | 960GB Corsair Force LE | EVGA G2 850W | Phanteks P400S

Laptop: Intel M-5Y10c | Intel HD Graphics | 8GB RAM | 250GB Micron SSD | Asus UX305FA

Server 01: Intel Xeon D 1541 | ASRock Rack D1541D4I-2L2T | 32GB Hynix ECC DDR4 | 4x8TB Western Digital HDDs | 32TB Raw 16TB Usable

Server 02: Intel i7 7700K | Gigabye Z170N Gaming5 | 16GB Trident Z 3200MHz

Link to post
Share on other sites
Posted
  • Author
22 minutes ago, DeadEyePsycho said:

In most cases, you aren't explicitly connecting a bunch of computers together except with your standard network connection.

 

Essentially you're just installing software on each machine that lets you distribute your work load. Folding and boinc are examples of this but generally it is used for a specific task such as running simulations.

 

That's why @Electronics Wizardy was specifically asking you what you wanted to do as it depends entirely on the software used.

Don't misunderstand; I was agreeing with them, but I wanted an actual explanation. Also, I'd like to link the computers together and have 1 master computer, and the rest slaves. 

Motivation is where, and what you make of it.

 

“It is relatively unusual that a physical scientist is truly an atheist. Why is this true? Some point to the anthropic constraints, the remarkable fine tuning of the universe. For example, Freeman Dyson, a Princeton faculty member, has said, ‘Nature has been kinder to us that we had any right to expect.'”  —Albert Einstein

Link to post
Share on other sites
Posted
On 12/20/2016 at 0:45 AM, Little Bear said:

Don't misunderstand; I was agreeing with them, but I wanted an actual explanation. Also, I'd like to link the computers together and have 1 master computer, and the rest slaves. 

Ha Ha me again

 

The thing with clusters is that the software the master runs is normally very depend't on exactly what you want to do on the nodes and why your making a cluster. Normally your running a cluster for 1 of 2 main reasons, or a combo of both. 

 

1. HA(high availability. This is when you have 2 or more servers running the same thing at once, and only one is used at a time, with the other nodes seeing if the main one is running, and if it fails, they will take the same ip and config, and can switch in about 100msec, so you won't notice any downtime. This is mainly used for things where uptime is very imporant, like the stock exchange. The easy way to do this is heartbeart on linux or drbd.

 

2. High performance. This is when you split a task into smaller ones to have work on little chunks. This is used for things like rendering, scientific computations, VM's. THe controller depends on the exact task, but something like a beowolf cluster can be used to shove work out to a ton of slaves with a variable value, and have them work on that chuck. 

 

 

If you want to do any of this you first need a task to cluster.

Link to post
Share on other sites
Posted
  • Author
  • Solution
8 minutes ago, Electronics Wizardy said:

Ha Ha me again

 

The thing with clusters is that the software the master runs is normally very depend't on exactly what you want to do on the nodes and why your making a cluster. Normally your running a cluster for 1 of 2 main reasons, or a combo of both. 

 

1. HA(high availability. This is when you have 2 or more servers running the same thing at once, and only one is used at a time, with the other nodes seeing if the main one is running, and if it fails, they will take the same ip and config, and can switch in about 100msec, so you won't notice any downtime. This is mainly used for things where uptime is very imporant, like the stock exchange. The easy way to do this is heartbeart on linux or drbd.

 

2. High performance. This is when you split a task into smaller ones to have work on little chunks. This is used for things like rendering, scientific computations, VM's. THe controller depends on the exact task, but something like a beowolf cluster can be used to shove work out to a ton of slaves with a variable value, and have them work on that chuck. 

 

 

If you want to do any of this you first need a task to cluster.

Thanks for the response, but I've already set it up.

Motivation is where, and what you make of it.

 

“It is relatively unusual that a physical scientist is truly an atheist. Why is this true? Some point to the anthropic constraints, the remarkable fine tuning of the universe. For example, Freeman Dyson, a Princeton faculty member, has said, ‘Nature has been kinder to us that we had any right to expect.'”  —Albert Einstein

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing Servers, NAS, and Home Lab

×