Setting up the Master Node

The Box cluster machine does not have a CD-ROM drive. One option is to attach a CD/DVD drive to the SATA slot. The other option is to boot this node through netboot (PXE). To do this, we should use another machine. In our case, we use the first option - viz. adding a CD-ROM drive.

After this, we setup a) a DHCP server, b) NFS server, c) PXE boot server and d) web server.

We need to do some network setup for IP masquerading and configuration of the Fully Automated Install using Kickstart as well.

DHCP server

A DHCP server is needed to dynamically assign IP addresses to the slave-nodes during the PXE boot process. We install it using

apt-get install dhcp3-server

We use the following configuration in /etc/dhcp/dhcpd.conf



ddns-update-style none;
default-lease-time 600;
max-lease-time 7200;
authoritative;
ddns-update-style none;
subnet 192.168.1.0 netmask 255.255.255.0 {
     default-lease-time 86400;
     range 192.168.1.2 192.168.1.20;
     max-lease-time 604800;
     option routers 192.168.1.1;
     option subnet-mask 255.255.255.0;
     option domain-name "local";
     option domain-name-servers 163.143.1.100;
     option nis-domain "hpc";
     option broadcast-address 192.168.1.255;
     allow booting;
     allow bootp;
     if (substring (option vendor-class-identifier, 0, 20)
                     = "PXEClient:Arch:00002") {
             # ia64
             filename "elilo.efi";
             next-server 192.168.1.1;
     } elsif ((substring (option vendor-class-identifier, 0, 9)
                     = "PXEClient") or
             (substring (option vendor-class-identifier, 0, 9)
                     = "Etherboot")) {
             # i386 and x86_64
             filename "pxelinux.0";
             next-server 192.168.1.1;
     } else {
             filename "/install/sbin/kickstart.cgi";
             next-server 192.168.1.1;
     }

    host hpcs02 {
        hardware ethernet 00:15:17:31:11:00;
        option host-name "hpcs02";
        fixed-address 192.168.1.2;
    }
    host hpcs03 {
        hardware ethernet 00:15:17:31:0D:8C;
        option host-name "hpcs03";
        fixed-address 192.168.1.3;
    }
# and so on .........
}

We should edit /etc/network/interfaces and add the following



auto eth1
iface eth1 inet static
     address 192.168.1.1
     netmask 255.255.255.0
     network 192.168.1.0
     broadcast 192.168.1.255
     gateway 163.143.166.104
    dns-nameservers 163.143.1.100

Now restart the network and the dhcp server

service networking restart
service isc-dhcp-server restart

More network configuration

Edit /etc/hosts to include all the nodes. For example



127.0.0.1	localhost
192.168.1.1  hpcs01.u-aizu.ac.jp hpcs01
192.168.1.2  hpcs02.u-aizu.ac.jp hpcs02
192.168.1.3  hpcs03.u-aizu.ac.jp hpcs03
192.168.1.4  hpcs04.u-aizu.ac.jp hpcs04
192.168.1.5  hpcs05.u-aizu.ac.jp hpcs05
192.168.1.6  hpcs06.u-aizu.ac.jp hpcs06
192.168.1.7  hpcs07.u-aizu.ac.jp hpcs07

Also create a text file in /etc/machines with the names of all the slave nodes for example. We will use this for scripting common tasks across the nodes.



hpcs01
hpcs02
...etc

PXE boot server

Install tftpd-hpa: apt-get install tfptd-hpa

Edit /etc/default/tfptd-hpa to include: #Defaults for tftpd-hpa RUN_DAEMON=”yes” OPTIONS=”-l -s /var/lib/tftpboot”

Now download the netboot file for Ubuntu amd64 from http://cdimage.ubuntu.com/netboot/

For example cd /var/lib/tftpboot wget http://archive.ubuntu.com/ubuntu/dists/precise/main/installer-amd64/current/images/netboot/netboot.tar.gz

     tar xvzf netboot.tar.gz

Restart the service: service tftpd-hpa restart

Web server

apt-get install apache2

NFS server

Install the packages: apt-get nfs-common nfs-kernel-server

Edit /etc/exports to export /home and /var/lib/tftpboot:

/home     192.168.1.0/24(rw,no_root_squash,sync,no_subtree_check)
/var/lib/tftpboot 192.168.1.0/24(rw,no_root_squash,sync,no_subtree_check)

Export these files with exportfs -av

IP Masquerading

First edit /etc/sysctl.conf and uncomment the following line net.ipv4.ip_forward=1

Follow this by sudo sysctl -p

For IP masquerading we use sudo iptables -t nat -A POSTROUTING -s 192.168.1.0/16 -p eth0 -j MASQUERADE

iptables-save > /etc/network/iptables

For permanent IP masquerading, append the following to /etc/network/interfaces as follows

auto eth1
iface eth1 inet static
address 192.168.0.1
netmask 255.255.255.0
network 192.168.0.0
broadcast 192.168.0.255
up iptables-restore < /etc/network/iptables

Fully Automated Install Using Kickstart

We use the following Kickstart file which is placed at /var/www/ubuntu/ks.cfg



text
#Generated by Kickstart Configurator
#platform=x86

#System language
lang en_US
#Language modules to install
langsupport en_US
#System keyboard
keyboard us
#System mouse
mouse
#System timezone
timezone --utc Asia/Tokyo
#Root password
rootpw --disabled
#Initial user
user terasaka --fullname "Haruo Terasaka" --iscrypted --password cryptedPass
#Reboot after installation
reboot
#Install OS instead of upgrade
install
#Use Web installation
#url --url http://jp.archive.ubuntu.com/ubuntu
# see if we can use apt-cacher
url --url http://192.168.1.1:3142/ubuntu
#System bootloader configuration
bootloader --location=mbr 
#Clear the Master Boot Record
zerombr yes
#Partition clearing information
clearpart --all --initlabel
part / --fstype ext4 --size 1 --grow
part swap --recommended
#System authorization infomation
auth  --useshadow  --enablemd5
#Network information
network --bootproto=dhcp --device=eth0
#Firewall configuration
firewall --disabled
#Do not configure the X Window System
skipx
%packages
ruby1.9.1
ssh
nfs-common


To let the install process to use the kickstart file, we do the following: Delete the original contents of the /var/lib/tftpboot/pxelinux.cfg/default
file and replace with the following


default myinstall
prompt 0
TIMEOUT 0
label myinstall
   SAY Now booting from PXELINUX ...
        kernel ubuntu-installer/amd64/linux
        append ks=http://192.168.1.1/ubuntu/ks.cfg vga=788 initrd=ubuntu-installer/amd64/initrd.gz -- quiet
~                          

Caching ubuntu packages

To cache packages used by the installer we use apt-cacher. First install it apt-get install apt-cacher

Then edit /etc/apt-cacher/apt-cacher.conf and add a path_map entry: path_map = ubuntu jp.archive.ubuntu.com/ubuntu

The Client Nodes

Boot the client nodes and press F12 when the BIOS screen appears. After this the PXE will start and do the installation on its own without any manual key presses.