Installing FreeBSD 8 using PXE and NFS mount

There are a few other blogs about this very topic, however, none I could find that would work perfectly for me. So I decided to gather some notes from the recent install of our development environment, a couple of SunFire X2200's.

The two blogs that really helped me and contained really well written information are:
Jeremy Chadwick and from High5!

If these instructions aren't working 100%, its worth going through these two blogs as well to get some extra insight.

Hardware wise you will need:

1. A Server, the machine with FreeBSD already installed. You can use other OS's as well, but then some commands and config files might differ.
2. A Client, The machine that you want to install FreeBSD onto.
3. Cross over cable to link the two, or else both located on the same subnet
4. Monitor on the client for your first install, it handy so see error messages

In order, here is what we will be going through together:

1. Set Server NIC to static IP address
2. Install and configure DHCP server
3. Configure TFTP
4. Configure NFS
5. Copy FreeBSD CD to NFS
6. Configure loader.conf of PXEBOOT
7. Workaround for a bug in mfs_root
8. Boot PXE
9. Installation
10. Problems and solutions

1. Set server NIC to static IP Address

First, we need to get the name of the network card. Type in the following command, and you will get similar output.

# ifconfig
msk0: flags=8863 mtu 1500
      ether 00:17:f2:50:4a:d4
      media: autoselect () status: inactive 
      supported media: autoselect
lo0:  flags=8049 mtu 16384
      inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
      inet 127.0.0.1 netmask 0xff000000
      inet6 ::1 prefixlen 128

Now open /etc/rc.conf and add the following lines to set IP Address and subnet mask

ifconfig_msk0="inet 192.168.1.1 netmask 255.255.255.0"

Notice, now ifconfig_msk0 matches the name of the card. This is standard convention, you can read the doc's on it later!

2. Install and configure DHCP Server

Using Ports on FreeBSD, its quite easy to install a DHCP server
You will need to look up the documentation, if you don't already have ports installed on your machine. To install -

pkg_add -r isc-dhcp31-server

Now we should create /usr/local/etc/dhcpd.conf with the following config.

allow booting;
allow bootp;
default-lease-time 600;
max-lease-time 7200;
authoritative;
ddns-update-style ad-hoc;
log-facility local7;
option domain-name "home.lan";
option domain-name-servers 192.168.1.1;
server-name "192.168.1.1";
server-identifier 192.168.1.1;
next-server 192.168.1.1;
option routers 192.168.1.1;
option root-path "192.168.1.1:/usr/local/freebsd8";
filename "freebsd8/boot/pxeboot";

subnet 192.168.1.0 netmask 255.255.255.0 {
  range 192.168.1.10 192.168.1.11;
}

The IP address is the static IP address of the servers network card which we set in the last step.
filename "freebsd8/boot/pxeboot" - This is the file that PXE will serve when requested

option root-path - This tells pxeboot where the root filesystem will be mounted from

3. Configure TFTP

In /etc/inetd.conf, find the line similar to

  tftp dgram udp wait root /usr/libexec/tftpd tftpd -l -s /usr/local

Uncomment it, and change it so -s flag points to /usr/local

Add the following line to /etc/rc.conf

inetd_enable="YES"

Now you can start the service, which will also start tftpd

 # /etc/rc.d/inetd start

4. Configure NFS

Add the following to /etc/rc.conf

rpcbind_enable="yes"
mountd_enable="yes"
nfs_server_enable="yes"

You will need to add an NFS export to /etc/exports

/usr/local/freebsd8 -alldirs -maproot=root 192.168.1.10

The ip address here, is the ipaddress of the client machine, so that it will have access to the nfs mount.
-maproot=root is required so that the client machine can log in as root if required.

Now lets make the exported directory so we can put the FreeBSD files there

 # mkdir /usr/local/freebsd8
 # chmod 755 /usr/local/freebsd8

Now we can start all the above services

/etc/rc.d/rpcbind start
/etc/rc.d/mountd start
/etc/rc.d/nfsd start

To see check if the mount is available,

# showmount-e
Exports list on localhost:
/usr/local/freebsd8 192.168.1.0

5. Copy FreeBSD 8 contents to the NFS mount

There are two ways of doing this,
i) copy from CD, if you have one
ii) download the iso and extract it

i) Copy from CD

 # mount -t cd9660 /dev/acd0c /mnt
 # cp -pR /mnt/* /usr/local/freebsd8

ii) Download ISO
Make sure your getting the correct architecture ISO.

 # fetch ftp://ftp.freebsd.org/pub/FreeBSD/ISO-IMAGES-amd64/8.0/8.0-RELEASE-amd64-disc1.iso
 # tar -C /usr/local/freebsd8 -pxvf 8.0-RELEASE-amd64-disc1.iso

6. Configure loader.conf

/usr/local/freebsd7/boot/loader.conf needs to be updated to the client will know where to mount root from

 # echo 'vfs.root.mountfrom="ufs:/dev/md0"' >> /usr/local/freebsd7/boot/loader.conf

This will tell the client machine where to mount the image from. A number of blogs suggest that you should use 'md0c' rather than 'md0', the 'c' is a convention to start at the first block of the mount. When I did this, I had troubles getting the sysinstall programming running on the client. When I removed it, it worked without any problems.

7. Workaround for bug in mfs_root

There is a bug in the mfs_root loader, where if it is gzipped, then it will make the clients reboot. Luckily its well documentated on the internet, so I avoided that problem. Here is the quick fix:

 # cd /usr/local/freebsd8/boot
 # gzip -d mfsroot.gz

8. Installation

Time to turn on the client!
Make sure that it boots up under PXE boot, you may have to configure the bios to do it.
Once PXE boot is started, it should find the DHCP server in a few seconds, if not, double check your settings

After PXE boot, it will load all the necessary boot files and land you with the Welcome to FreeBSD screen

Select 1. for normal boot, and install

Run through all the installation steps as normal, until you select the source to install BSD from. Here you will need to choose NFS.

If it asks you to configure NIC's, select the one that is currently plugged into the DHCP server, and configure that. Let it auto configure on DHCP, if it doesn't find the server, you've chosen the wrong NIC

Select NFS mount
Give the address 192.168.1.1:/usr/local/freebsd8
Install should be automatic from here on.

Well done!

Some head scratching errors:

Problem: I kept getting this error:
ROOT MOUNT ERROR:
If you have invalid mount options, reboot, and first try the following from the loader prompt:

set vfs.root.mountfrom.options=rw

and then remove invalid mount options from /etc/fstab.

Solution: This was because within /usr/local/freebsd7/boot/loader.conf
I had set vfs.root.mountfrom="ufs:/dev/md0c"
Solved by changing it to "ufs:/dev/md0"

Problem: After loading loader.conf on the client machine, the spinning '\' would hang.
This was because I had originally rebuilt pxeboot to accept a higher com port speed.
For some reason, if the com port is set up in any way, the client will hand here.
Solution: Dont configure the com port to accept the higher speed of 115200, just use the pxeboot that comes out of the box.

Problem: When picking NFS as the installation media, the client would hang when trying to mount the NFS drive.
Solution: Quite easy actually, make sure you pick the correct network card to mount from! After picking the card, the client will try nad configure it. Make sure to configure it by DHCP. If it doesn't find your DHCP server that you found on boot, then you've picked the wrong network card from the list. The list will only be present if there is more that one NIC on your machine.