Cluster-Handbook/Network

Network

The nodes in a cluster need to be connected to share information. For this purpose, each node becomes a master (head) or a server node. Once a configuration was decided, the network configurations for DHCP and DNS must be set.

DHCP/DNS

DHCP - Dynamic Host Configuration Protocol - allows the assignment of network configurations on the client by a server. The advantage is that no further manual configurations on the client are needed. When building large interconnected systems that have hundreds of clients, any manual configuration will quickly become bothersome. However, the server has to be set on every client and it's important that this assignment is distinct. DNS - Domain Name System - resolves host or domain names to IP addresses. This allows more readable und understandable connection since it associates various information with domain names. Requirement for all this is of course a shared physical network.

Master/Server

For the master, two files must be configured. One is located in /etc/network/interfaces :

Listing 6.1 Configuration for master

auto lo

iface lo inet loopback
auto eth0
iface eth0 inet dhcp #Externe Addresse fuer Master

auto eth1
iface eth1 inet static #IP-Addresse fuer das interne Netz

   address 10.0.x.250
   netmask 255.255.255.0

address shows the local network.The first three numbers separated by points are the network prefix. How big the network prefix is defined by the network mask, shown in the line below. Both specifications define ultimately what IP addresses on the local network and which are recognized by the router on other networks. All with 255 masked parts of the IP address form the network prefix. All devices that want to be included in the local network must have the same network prefix. In our example it starts with 10.0.. Subsequent x stands for the number of the local network, if more exist. All units which can be classified to the local network 1, must have the network prefix 10.0.1. The fourth entry, which is masked with a 0, describes the number of the device on the local network between 0 and 255. A convenient number for the server is 250, since it is relatively large and thus well distinguishable from the clients (unless there are more than 250 clients to register). Of course, it could have been any other permissible number.

The second file that has to be configured for the server is located in /etc/hosts

Listing 6.2 Configuration of the hosts for master

127.0.0.1    localhost

#Mapping IP addresses to host names. Worker and Master / Clients and Server

10.0.1.1        worker1
10.0.1.2        worker2
10.0.1.3        worker3
10.0.1.4        worker4

10.0.1.250      master

::1     ip6-localhost ip6-loopback

fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix

ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

At this point the name resolutions are entered. Unlike the example above, for the x a 1 was chosen. It is important not to forget to enter the master.

The existing line 127.0.1.1 ... must be removed - for master and worker.

Worker/Client

The affected files have to be modified for the worker and clients. Usually the DHCP server takes care of that. However, it can cause major problems if the external network can not be reached or permissions are missing. In the worst case you have to enter the static IP entries manually.

/etc/network/interfaces :

Listing 6.3 Configuration for workers

auto lo

iface lo inet loopback

#IP Addresse worker. Nameserver IP Addresse -&gt; Master

auto eth0

iface eth0 inet static

   address 10.0.1.1

   netmask 255.255.255.0
   dns-nameservers 10.0.1.250 #[,10.0.x.weitere_server ]

The names and IP addresses must of course match the records of the master.

/etc/hosts :

Listing 6.4 Configuration of the hosts for worker

 
127.0.0.1    localhost

::1     ip6-localhost ip6-loopback

fe00::0 ip6-localnet

ff00::0 ip6-mcastprefix

ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

To check if everything went well, you can check if the machines can ping each other:

$ ping master

NFS

It is not only necessary to facilitate communication between clients and servers, but also to give access to shared data. This can be configured through the NFS - Network File System. The data will not be transmitted if required. A read action is possible as if the data in its own memory (of course, with other access times).

Master/Server

First the NFS-Kernel-Server-Package must be installed:

$ sudo apt-get install nfs-kernel-server In /etc/ the following file exports has to be configured:

/home 10.0.1.0/24(rw,no_subtree_check,no_root_squash) In this example, three options are set:

rw gives the network read and write permissions
no_subtree_check ensures that the root belonging files are released; increases the transmission speed, since not every subdirectory is checked when a user requests a file (useful if the entire file system is unlocked)
no_root_squash gives the root-User writing permissions (otherwise the root will be mapped to nobody-User to ensure safety)

There must be no spaces in front and as well in the brackets.

Worker/Client

First the NFS-Common-Package must be installed:

$ sudo apt-get install nfs-common In /etc/ the following file fstab has to be configured as shown:

master:/home /home nfs rw,auto,proto=tcp,intr,nfsvers=3 0 0 For meanings of individual options refer to man fstab.

Should the clients do not have internet access because the internal network doesn't provide it, you have to activate routing on the master/server, as only it has an connection to an external network. See section 6.3

After installation a status update is needed:

$ sudo exportfs -ra , where

-r : Export all directories. This option synchonizes /var/lib/nfs/xtab with /etc/exports. Entries in /var/lib/nfs/xtab, that have been removed from /etc/exports. In addition, all entries from the kernel tables are deleted that are no longer valid.
-a : (Un-)Export all directories (that are listed in exports).

The NFS server should be restarted:

$ sudo /etc/init.d/nfs-kernel-server restart

Routing

Master

The routing determines the entire path of a stream of messages through the network. Forwarding describes the decision-making process of a single network node, over which it forwards a message to his neighbors </ref> Our goal is to provide the nodes with access to the internet via the server node, which are only accessible on the local network (thus on the clients packages can be downloaded).

The following lines need to inserted into /etc/sysctl.conf to active IP-Forwarding:

net.ipv4.ip_forward=1 In addition, NAT must be activated on the external interface eth0. Therefore add the following line into /etc/rc.local (via exit 0).

Worker/Client

Here, the master must be set up as a gateway:

gateway 10.0.x.250 The hardware should be restarted. At default settings, the /home directory will be mounted, otherwise you can do it manually:

$ sudo mount /home Note that the nodes should have only access to the internet in the configuration phase. In normal operation, this would be a security risk.