Parallel Computing and Computer Clusters/Overview

Overview of the subject matter

As mentioned in the Introduction, both parallel computing and computing clusters have a heavy overlap in their subject matter.

Parallel Computing

Parallel computing is an ambiguous term covering two distinct areas of computing: designing single machines with many processors (hardware parallel computing) and designing software to split complex scenarios into manageable chunks (software parallel computing).

Parallel Computer

A parallel computer is one which has been designed and houses multiple MPUs each of which can work independently of its peers. Many modern PCs are capable of housing two CPUs and an individual CPU can house multiple cores each of which can work independently of the others - both are examples of a computer which can perform multiple tasks in parallel. Note that a single-CPU, single-core machine running a multi-tasking operating system (such as MS Windows or one of the various flavours of Unix/Linux) is not running in parallel but is instead running many small chunks of tasks serially: the software is parallelised but the hardware is not.

Parallel Software

When designing software to work on a large data set it is often possible to split the data set into multiple smaller subsets without affecting the goal of the work as a whole. For example, to retrieve all of the static web pages on an individual server, it is possible to retrieve each one individually without affecting the end result and the chunks of the data set can therefore be performed in parallel. Once the data has been successfully determined to be split then the software can be designed in a way to allow individual CPUs to work on the smaller chunks & thus many CPUs can work in parallel on many different chunks.

Computer Clusters

Computer clusters are a network made up of multiple individual machines to work towards or provide a common theme or aim. For example, a simple search engine could be comprised of a pair of machines: one machine to retrieve and index the data and one machine to provide a front end for people to access the indexes. In this manner, the cluster is formed of multiple different tasks, but are seen to be a single entity by those utilising the cluster.

Many clusters are set up to work towards the same common goal, working on similar data sets in similar manners. In this form of scenario the cluster is computing in parallel and thus the divide between Parallel Computing and Computer Clusters becomes unclear.