Introduction to PXE/iPXE Network Boot Featuring FreeBSD & Ubuntu Server

Windows7ge · December 10, 2021

For the uninitiated, what is PXE/iPXE Network Boot?

PXE/iPXE or Preboot eXecution Environment is a feature included with many makes and model of network adapter used in the boot process of a host from a network resource. Unlike traditional methods of booting a computer where-in you boot from a HDD/SSD, CD, or USB device PXE/iPXE enables your computer to reach out to a server on the network hosting the appropriate services to boot from. This can be a server hosting services such as TFTP, iSCSI, NFS, or even FCoE.

NOTE: At this time this tutorial only supports Legacy Boot. UEFI Boot has not successfully been tested and therefore has been excluded for the time being.

Index

Spoiler

1. Hardware Requirements

1.1 - Server Side Requirements

1.2 - Network Requirements

1.3 - Client Side Requirements

2. Preparing Installation Media

2.1 - On Windows

2.2 - On Linux

3. Prepping the Server Operating System

3.1 - Installing the Operating System

3.2 - Initial System Configuration

3.2.1 - Installing the Package Manager

3.2.2 - Installing & Configuring Sudo

3.2.3 - Installing Nano (optional)

3.2.4 - Adding Additional Network Interfaces (Important!)

3.3 - Installing & Configuring Services

3.3.1 - Installing & Configuring DHCP (Dynamic Host Configuration Protocol)

3.3.1.1 - Installation

3.3.1.2 - Configuration

3.3.2 - Installing & Configuring TFTP (Trivial File Transfer Protocol)

3.3.2.1 - Installation

3.3.2.2 - Configuration

3.3.2.3 - Creating Custom iPXE Image Files

3.3.3 - Configuring iSCSI (Internet Small Computer Systems Interface)

4. Installing & Modifying the Client Operating System

4.1 - Installation Setup

4.2 - Modifying Installation

4.2.1 - Editing Modules

4.2.2 - Editing GRUB

4.2.3 - Editing Initiator Name

1. Hardware Requirements

Spoiler

1.1 - Server Side Requirements

Spoiler

A dedicated system with at least:

A quad-core processor (Xeon recommended)
8GB of RAM (ECC recommended)
Enough SATA ports to host either individual disks for each client or a RAID array. (SSD's recommended)

OR

A hypervisor server with:

Enough spare cores for a FreeBSD VM
Enough spare memory for a FreeBSD VM
Enough spare storage for however many clients you plan to configure

IN EITHER SCENARIO

For the best results & performance you will want to use at least a 10Gig capable network adapter. Any of the four mentioned below would be appropriate and will work:

Mellanox ConnectX-2 MNPA19-XTR
Mellanox ConnectX-3 CX311A (Recommended)
Intel X520-T1 or Intel X520-T2 (Highly Recommend)
Intel X540-T1 or Intel X540-T2 (Recommend)

1.2 - Network Requirements

Spoiler

A suitable network switch with enough ports at the right speed. Possible candidates include:
- The UniFi ES‑16‑XG
- The MikroTik CRS305-1G-4S+IN
- The Netgear XS708T
The cables for whichever network adapters you chose.
- DAC (Direct Attach Copper) cables.
- OM3 or OM4 fiber patch cables with LC connectors.
- Transceivers like the Cisco SFP-10G-SR.
- Cat6/Cat6a patch cables with RJ-45 connectors.
NO ROUTER. This network will not need a stand-alone router. If you connect one it will cause problems!

1.3 - Client Side Requirements

Spoiler

The only real requirement is that you must verify that the network adapter you have or plan to use supports PXE/iPXE. If your network adapter is built into your motherboard the feature is usually disabled by default. You'll have to enter your BIOS and enable it.

For the best results & performance though you will want to use at least a 10Gig capable adapter. Any of the four mentioned below would be appropriate and will work:

Mellanox ConnectX-2 MNPA19-XTR
Mellanox ConnectX-3 CX311A (Recommended)
Intel X520-T1 or Intel X520-T2 (Highly Recommend)
Intel X540-T1 or Intel X540-T2 (Recommend)

2. Preparing Installation Media

Spoiler

Bootable USB sticks can be created for both the server & client operating systems. Just download the .ISO file from each project site and follow the USB setup instructions for your platform.

Server OS Project Site
- FreeBSD
Client OS Project Site
- Ubuntu Server

2.1 - On Windows

Spoiler

After the .ISO finishes downloading start by installing an application known as Rufus. Rufus is a free and easy utility for Windows to create bootable USB media.

Start by selecting the thumb drive you want to install to in the Device field.

Now click SELECT to the right and navigate to where you saved the .ISO file. Click the .ISO file then click Open. From here Rufus should auto-populate the remaining fields with appropriate values.

Now click START. You may receive a prompt that the .ISO is of a Hybrid type. It's fine to leave the value below on it's default setting. Click OK.

You may receive a prompt about out of date files. If you do go ahead and press Yes to update them.

You'll be prompted that all data on the drive will be erased. If you're certain that that's OK go ahead and click OK.

Allow the installer to run to completion. This will be denoted by the following status on the program:

Your installer USB is now ready.

2.2 - On Linux

Spoiler

After the .ISO finishes downloading start by opening a Terminal in the directory where the .ISO file is located. We're going to use the terminal application DD which is included with manu GNU/Linux distributions. If you don't have DD this can likely be downloaded from your distributions repositories.

Start by listing what devices are available with the lsblk command.

user@workstation:~$ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sde           8:64   1  57.3G  0 disk 
└─sde1        8:65   1  57.3G  0 part /media/user/64 GB
nvme0n1     259:0    0 953.9G  0 disk 
├─nvme0n1p1 259:1    0   512M  0 part /boot/efi
└─nvme0n1p2 259:2    0 953.4G  0 part /run/timeshift/backup

We can see a 64GB thumb-drive with the drive name sde. Start by unmounting any & all volumes on the drive with umount /dev/sde*:

user@workstation:~$ umount /dev/sde*
umount: /dev/sde: not mounted.

Now to create a file-system on it use sudo mkfs.ext4 /dev/sde:

user@workstation:~$ sudo mkfs.ext4 /dev/sde
[sudo] password for user: 
mke2fs 1.45.5 (07-Jan-2020)
Found a dos partition table in /dev/sde
Proceed anyway? (y,N) y
Creating filesystem with 15022080 4k blocks and 3760128 inodes
Filesystem UUID: 31afea12-4adf-42f8-8656-a812307e4f40
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
	4096000, 7962624, 11239424

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (65536 blocks): done
Writing superblocks and filesystem accounting information: done

From here we can use DD to write the .ISO image to the USB using sudo dd if=ubuntu-20.04.3-live-server-amd64.iso of=/dev/sde status=progress:

user@workstation:~$ sudo dd if=ubuntu-20.04.3-live-server-amd64.iso of=/dev/sde status=progress 
1249743360 bytes (1.2 GB, 1.2 GiB) copied, 84 s, 14.9 MB/s 
2463616+0 records in
2463616+0 records out
1261371392 bytes (1.3 GB, 1.2 GiB) copied, 99.1369 s, 12.7 MB/s

Your installer USB is now ready.

3. Prepping the Server Operating System

Spoiler

Installing FreeBSD is a easy but tedious process with many many configurable parameters throughout the setup. I will however walk us through most/all of the setup process including the initial system configuration.

3.1 - Installing the Operating System

Spoiler

When you first start the installer after seeing the splash screen above you'll be greeted by a menu with the options to Install, exit to Shell, or run the Live CD. Go ahead and click install.

After this you'll be told to pick a keyboard keymap (default is QWERTY English).

Next up is to choose a name for your server.

You can name yours whatever you prefer.

The next few menus can all be left on default unless you otherwise need something.

Now select which disk you want to install the OS to. This can be done with the arrow keys + the space bar for selection.

You'll be asked if you're sure as all data on this disk will be erased.

If you're sure. Proceed.

Now pick a password for your new root user:

FreeBSD Installer
========================

Please select a password for the system management account (root):
Typed characters will not be visible.
Changing local password for root
New Password:

Time to configured the network interfaces. Now the FreeBSD installer only lets you configure one interface, any others will need to be configured within the OS. Select the adapter that's going to be part of your management network. If you make a mistake here it's OK. We can fix it once we're booted into the OS.

You'll be prompted as to if you want to configure IPv4. Under most circumstances the answer is yes.

Next up is do you want to leave it up to DHCP (Dynamic Host Configuration Protocol - ie. your Router) to setup the interface? Although you can, I don't recommend it. You'll want to configure the interface manually so the IP doesn't have the possibility of changing on you. It's important to note though whatever IP you assign it isn't able to be handed out by your router anymore to other clients as this can lead to serious network conflict.

If you don't know what to enter below reference your router's DHCP page or ask a question to find out what subnet your network uses as not all computer networks are setup like mine.

You'll be prompted as to if you want to configure IPv6. This is optional and subject to your needs. So I won't be going over it here.

Now we need to setup DNS (Domain Name Server/Service). You can use your ISP provided DNS or a 3rd party like Google (8.8.8.8, 8.8.4.4), or CloudFlare (1.1.1.1, 1.0.0.1).

Time Zone Setup

The following menus can all be left on default values.

If you would like to add a user you may do so now.

When adding a user you will have to fill out the following details:

FreeBSD Installer
========================
Add Users

Username: ipxe
Full name: ipxe
Uid (Leave empty for default):
Login group [ipxe]:
Login group is ipxe. Invite ipxe into other groups? []:
Login class [default]:
Shell (sh csh tcsh nologin) [sh]:
Home directory [/home/ipxe]:
Home directory permissions (Leave empty for default):
Use password-based authentication? [yes]:
Use an empty password? (yes/no) [no]:
Use a random password? (yes/no) [no]:
Enter password:
Enter password again:
Lock out the account after creation? [no]:
Username   : ipxe
Password   : *****
Full Name  : ipxe
Uid        : 1001
Class      :
Groups     : ipxe
Home       : /home/ipxe
Home Mode  :
Shell      : /bin/sh
Locked     : no
OK? (yes/no):

Afterwords you'll be asked if you have any final changes you'd like to make:

If you'd like the enter shell:

And finally Reboot or enter Live CD.

Go ahead and reboot the system. OS installation is now complete. You will see the following when the OS finishes loading:

3.2 - Initial System Configuration

Spoiler

Once FreeBSD has completed it's installation start by logging in as root with your root user's password.

3.2.1 - Installing the Package Manager

Spoiler

To install the package manager query it by running the command pkg.

root@ipxe-server:~ $ pkg
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]:

When complete you will see the following output:

Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:13:amd64/quarterly, please wait...
Verifying signature with trusted certificate pkg.freebsd.org.2013102301... done
Installing pkg-1.17.2...
Extracting pkg-1.17.2: 100%
pkg: not enough arguments
Usage: pkg [-v] [-d] [-l] [-N] [-j <jail name or id>|-c <chroot path>|-r <rootdir>] [-C <configuration file>] [-R <repo config dir>] [-o var=value] [-4|-6] <command> [<args>]

For more information on available commands and options see 'pkg help'.

3.2.2 - Installing & Configuring Sudo

Spoiler

Sudo is a helpful tool for allowing regular users to run commands they'd ordinarily have to be root in order to use. To install it run pkg install sudo.

root@ipxe-server:~ # pkg install sudo
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
The following 3 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
	gettext-runtime: 0.21
	indexinfo: 0.3.1
	sudo: 1.9.8p2

Number of packages to be installed: 3

The process will require 7 MiB more space.
2 MiB to be downloaded.

Proceed with this action? [y/N]:

When complete you should see this output:

[1/3] Fetching sudo-1.9.8p2.pkg: 100%    1 MiB   1.5MB/s    00:01    
[2/3] Fetching gettext-runtime-0.21.pkg: 100%  166 KiB 169.9kB/s    00:01    
[3/3] Fetching indexinfo-0.3.1.pkg: 100%    6 KiB   5.7kB/s    00:01    
Checking integrity... done (0 conflicting)
[1/3] Installing indexinfo-0.3.1...
[1/3] Extracting indexinfo-0.3.1: 100%
[2/3] Installing gettext-runtime-0.21...
[2/3] Extracting gettext-runtime-0.21: 100%
[3/3] Installing sudo-1.9.8p2...
[3/3] Extracting sudo-1.9.8p2: 100%
root@ipxe-server:~ #

To allow users to use sudo we need to edit the sudoers file. This can be done with visudo.

root@ipxe-server:~ # visudo

## sudoers file.
##
## This file MUST be edited with the 'visudo' command as root.
## Failure to use 'visudo' may result in syntax or file permission errors
## that prevent sudo from running.
##
## See the sudoers man page for the details on how to write a sudoers file.
##

##
## Host alias specification
##
## Groups of machines. These may include host names (optionally with wildcards),
## IP addresses, network numbers or netgroups.
# Host_Alias    WEBSERVERS = www1, www2, www3

##
## User alias specification
##
## Groups of users.  These may consist of user names, uids, Unix groups,
## or netgroups.
# User_Alias    ADMINS = millert, dowdy, mikef

##
## Cmnd alias specification
##
## Groups of commands.  Often used to group related commands together.
# Cmnd_Alias    PROCESSES = /usr/bin/nice, /bin/kill, /usr/bin/renice, \
#                           /usr/bin/pkill, /usr/bin/top
# Cmnd_Alias    REBOOT = /sbin/halt, /sbin/reboot, /sbin/poweroff

##
## Defaults specification
##
## Uncomment if needed to preserve environmental variables related to the
## FreeBSD pkg utility and fetch.
# Defaults     env_keep += "PKG_CACHEDIR PKG_DBDIR FTP_PASSIVE_MODE"
##
## Additionally uncomment if needed to preserve environmental variables
## related to portupgrade
# Defaults     env_keep += "PORTSDIR PORTS_INDEX PORTS_DBDIR PACKAGES PKGTOOLS_CONF"
##
## You may wish to keep some of the following environment variables
## when running commands via sudo.
##
## Locale settings
# Defaults env_keep += "LANG LANGUAGE LINGUAS LC_* _XKB_CHARSET"
##
## Run X applications through sudo; HOME is used to find the
## .Xauthority file.  Note that other programs use HOME to find
## configuration files and this may lead to privilege escalation!
# Defaults env_keep += "HOME"
##
## X11 resource path settings
# Defaults env_keep += "XAPPLRESDIR XFILESEARCHPATH XUSERFILESEARCHPATH"
##
## Desktop path settings
# Defaults env_keep += "QTDIR KDEDIR"
##
## Allow sudo-run commands to inherit the callers' ConsoleKit session
# Defaults env_keep += "XDG_SESSION_COOKIE"
##
## Uncomment to enable special input methods.  Care should be taken as
## this may allow users to subvert the command being run via sudo.
# Defaults env_keep += "XMODIFIERS GTK_IM_MODULE QT_IM_MODULE QT_IM_SWITCHER"
##
## Uncomment to use a hard-coded PATH instead of the user's to find commands
# Defaults secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
##
## Uncomment to send mail if the user does not enter the correct password.
# Defaults mail_badpass
##
## Uncomment to enable logging of a command's output, except for
## sudoreplay and reboot.  Use sudoreplay to play back logged sessions.
# Defaults log_output
# Defaults!/usr/bin/sudoreplay !log_output
# Defaults!/usr/local/bin/sudoreplay !log_output
# Defaults!REBOOT !log_output

##
## Runas alias specification
##

##
## User privilege specification
##
root ALL=(ALL) ALL

## Uncomment to allow members of group wheel to execute any command
# %wheel ALL=(ALL) ALL

## Same thing without a password
# %wheel ALL=(ALL) NOPASSWD: ALL

## Uncomment to allow members of group sudo to execute any command
# %sudo ALL=(ALL) ALL

## Uncomment to allow any user to run sudo if they know the password
## of the user they are running the command as (root by default).
# Defaults targetpw  # Ask for the password of the target user
# ALL ALL=(ALL) ALL  # WARNING: only use this together with 'Defaults targetpw'

## Uncomment to show on password prompt which users' password is being expected
# Defaults passprompt="%p's password:"

## Read drop-in files from /usr/local/etc/sudoers.d
@includedir /usr/local/etc/sudoers.d

To add users start by pressing the letter I for Insert, now append to the end of the file user ALL=(ALL) ALL substituting user for your username(s).

...
## Read drop-in files from /usr/local/etc/sudoers.d
@includedir /usr/local/etc/sudoers.d

ipxe ALL=(ALL) ALL

To save & exit press ESC then the characters :wq followed by pressing Enter.

:wq
/usr/local/etc/sudoers.tmp: 109 lines, 3654 characters.

Now to test if sudo is working start by changing users with su - user.

root@ipxe-server:~ # su - ipxe
To do a fast search for a file, try

	 locate filename

locate uses a database that is updated every Saturday (assuming your computer
is running FreeBSD at the time) to quickly find files based on name only.
ipxe@ipxe-server:~ $

Now if we try to run the sudo editor as normal we get a permission denied error:

ipxe@ipxe-server:~ $ visudo
visudo: /usr/local/etc/sudoers: Permission denied

But with sudo we are offered to enter a password so we can run visudo among many other functions as if we were root:

ipxe@ipxe-server:~ $ sudo visudo

We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:

    #1) Respect the privacy of others.
    #2) Think before you type.
    #3) With great power comes great responsibility.

Password:

NOTE: If this is being deployed in any kind of business or public place enabling sudo introduces the potential for hackers to gain unauthorized access more easily. Use this power wisely. Of course the option to revoke permissions is as simple as re-ruinning visudo and removing users appropriately.

3.2.3 - Installing Nano (optional)

Spoiler

Nano is an easy to use CLI text editor which we will use to create and/or edit configuration files although Vi/Vim among other popular editors will work just as well.

To install nano run sudo pkg install nano.

ipxe@ipxe-server:~ $ sudo pkg install nano
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
The following 1 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
	nano: 5.8

Number of packages to be installed: 1

The process will require 3 MiB more space.
556 KiB to be downloaded.

Proceed with this action? [y/N]:

You should see the output:

[1/1] Fetching nano-5.8.pkg: 100%  556 KiB 569.1kB/s    00:01    
Checking integrity... done (0 conflicting)
[1/1] Installing nano-5.8...
[1/1] Extracting nano-5.8: 100%
ipxe@ipxe-server:~ $

3.2.4 - Adding Additional Network Interfaces (Important!)

Spoiler

Since PXE/iPXE relies on a DHCP server to initialize the connection unless this server is going to act as your primary DHCP server for your home/business network it needs to be virtually or physically segregated from your LAN so that FreeBSD's DHCP server doesn't conflict with your router's DHCP server.

To check what network interfaces you have available run the command ifconfig.

ipxe@ipxe-server:~ $ ifconfig
vtnet0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=4c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,TXCSUM_IPV6>
	ether 22:de:1a:f0:29:c7
	inet 192.168.0.200 netmask 0xffffff00 broadcast 192.168.0.255
	media: Ethernet autoselect (10Gbase-T <full-duplex>)
	status: active
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
vtnet1: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=4c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,TXCSUM_IPV6>
	ether 1a:30:11:9d:77:04
	media: Ethernet autoselect (10Gbase-T <full-duplex>)
	status: active
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
	inet6 ::1 prefixlen 128
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
	inet 127.0.0.1 netmask 0xff000000
	groups: lo
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

Here we have a 1Gig LAN network (vtnet0) and an unconfigured 10Gig network (vtnet1).

To add additional interfaces to the system configuration we need to edit rc.conf in /etc/rc.conf. We can do this by running sudo nano /etc/rc.conf.

sudo nano /etc/rc.conf

hostname="ipxe-server"
ifconfig_vtnet0="inet 192.168.0.200 netmask 255.255.255.0"
defaultrouter="192.168.0.1"
sshd_enable="YES"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="AUTO"
zfs_enable="YES"

Note this line.

ifconfig_vtnet0="inet 192.168.0.200 netmask 255.255.255.0"

This is what was configured when we installed the OS.

To add our other interface(s) simply add another line with a different IP address.

NOTE: This IP needs to be on a different subnet from your other interface(s). Feel free to ask questions if that confuses you.

ifconfig_vtnet0="inet 192.168.0.200 netmask 255.255.255.0"
ifconfig_vtnet1="inet 10.0.0.254 netmask 255.255.255.0"

Now press Ctrl+O to save then Ctrl+X to exit.

To apply our changes restart the server. We can view the changes after logging back in with the command ifconfig.

ipxe@ipxe-server:~ $ ifconfig
vtnet0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=4c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,TXCSUM_IPV6>
	ether 22:de:1a:f0:29:c7
	inet 192.168.0.200 netmask 0xffffff00 broadcast 192.168.0.255
	media: Ethernet autoselect (10Gbase-T <full-duplex>)
	status: active
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
vtnet1: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=4c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,TXCSUM_IPV6>
	ether 1a:30:11:9d:77:04
	inet 10.0.0.254 netmask 0xffffff00 broadcast 10.0.0.255
	media: Ethernet autoselect (10Gbase-T <full-duplex>)
	status: active
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
	inet6 ::1 prefixlen 128
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
	inet 127.0.0.1 netmask 0xff000000
	groups: lo
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

As we can see our interface acquired it's new IP. If yours did not double-check /etc/rc.conf for spelling errors.

3.3 - Installing & Configuring Services

Spoiler

With the OS cleanly installed with basic utilities we can start adding the services we need to boot a computer from the network.

3.3.1 - Installing & Configuring DHCP (Dynamic Host Configuration Protocol)

Spoiler

3.3.1.1 - Installation

Spoiler

Start by running the following commands:

NOTE: The first two commands will take some time to complete. This is normal.

sudo portsnap fetch
sudo portsnap extract
cd /usr/ports/net/isc-dhcp44-server
sudo make

When it completes you'll see the following prompt:

On this menu select BIND_SYMBOLS with the space bar and de-select IPV6. Then hit Enter.

After this there will be one more prompt but it can be left on default. Just hit Enter. You will see a lot of text but you will eventually be brought back to your prompt:

ipxe@ipxe-server:/usr/ports/net/isc-dhcp44-server $

From here run the command sudo make install:

ipxe@ipxe-server:/usr/ports/net/isc-dhcp44-server $ sudo make install
Password:
===>  Installing for isc-dhcp44-server-4.4.2P1_1
===>  Checking if isc-dhcp44-server is already installed
===>   Registering installation for isc-dhcp44-server-4.4.2P1_1
Installing isc-dhcp44-server-4.4.2P1_1...
===> Creating groups.
Creating group 'dhcpd' with gid '136'.
===> Creating users
Creating user 'dhcpd' with uid '136'.
****  To setup dhcpd, please edit /usr/local/etc/dhcpd.conf.

****  This port installs the dhcp daemon, but doesn't invoke dhcpd by default.
      If you want to invoke dhcpd at startup, add these lines to /etc/rc.conf:

	    dhcpd_enable="YES"				# dhcpd enabled?
	    dhcpd_flags="-q"				# command option(s)
	    dhcpd_conf="/usr/local/etc/dhcpd.conf"	# configuration file
	    dhcpd_ifaces=""				# ethernet interface(s)
	    dhcpd_withumask="022"			# file creation mask

****  If compiled with paranoia support (the default), the following rc.conf
      options are also supported:

	    dhcpd_chuser_enable="YES"		# runs w/o privileges?
	    dhcpd_withuser="dhcpd"		# user name to run as
	    dhcpd_withgroup="dhcpd"		# group name to run as
	    dhcpd_chroot_enable="YES"		# runs chrooted?
	    dhcpd_devfs_enable="YES"		# use devfs if available?
	    dhcpd_rootdir="/var/db/dhcpd"	# directory to run in
	    dhcpd_includedir="<some_dir>"	# directory with config-
						  files to include

****  WARNING: never edit the chrooted or jailed dhcpd.conf file but
      /usr/local/etc/dhcpd.conf instead which is always copied where
      needed upon startup.

===> SECURITY REPORT: 
      This port has installed the following files which may act as network
      servers and may therefore pose a remote security risk to the system.
/usr/local/lib/libomapi.a(listener.o)
/usr/local/lib/libdhcp.a(socket.o)
/usr/local/sbin/dhcpd
/usr/local/bin/omshell
/usr/local/lib/libdhcp.a(icmp.o)

      If there are vulnerabilities in these programs there may be a security
      risk to the system. FreeBSD makes no guarantee about the security of
      ports included in the Ports Collection. Please type 'make deinstall'
      to deinstall the port if this is a concern.

      For more information, and contact details about the security
      status of this software, see the following webpage: 
https://www.isc.org/dhcp/

3.3.1.2 - Configuration

Spoiler

With ISC_DHCP_SERVER installed we can start configuring it by adding two lines to the system startup config file in /etc/rc.conf:

dhcpd_enable="YES"
dhcpd_ifaces="vtnet1"

NOTE: Replace vtnet1 with the name of your interface(s). In the event of multiple interfaces they can be concatenated on one line. Ie. dhcpd_ifaces="vtnet1 vtnet2 vtnet3 etc."

Now to designate the subnet we want to advertise start by editing dhcpd.conf in /usr/local/etc:

sudo nano /usr/local/etc/dhcpd.conf

At the end of the configuration file we need to designate the following parameters:

subnet - The network/interface to be advertised on.
netmask - The subnet mask to tell where network & client addresses separate.
range - The range of addresses within the subnet to be handed out.
next-server - For chain-loading purposes (will be discussed later).
host - For assigning specific IP addresses to specific clients.
hardware ethernet - When listening for a particular Layer 2 MAC address.
fixed-address - For assigning a specific address to the MAC address.
filename - For chain-loading purposes (will be discussed later).

Now all of that may sound like a lot but in the config file all of it boils down to:

subnet 10.0.0.0 netmask 255.255.255.0 {
range 10.0.0.50 10.0.0.253;
next-server 10.0.0.254; }

host client-name {
hardware ethernet 00:02:c9:56:ab:aa;
fixed-address 10.0.0.50;
filename "undionly.kpxe"; }

Explained quite simply:

subnet 10.0.0.0 netmask 255.255.255.0 { - This tells DHCP that the subnet we want to broadcast is in the Class A range with a Class C netmask. This means there will be a maximum of 255 host bits for the network clients.
- range 10.0.0.50 10.0.0.253; - This tells DHCP that within the 10.0.0.0 subnet we only want to pass out IP addresses in-between 10.0.0.50 & 10.0.0.253.
- next-server 10.0.0.254; } - This is for our TFTP server which will be touched on later. For now set it to the same IP you configured in /etc/rc.conf.
host client-name { - This tells DHCP that for a specific client you want it to respond to their requests with specific designated parameters.
- hardware ethernet 00:02:c9:56:ab:aa; - This tells DHCP that the host you're looking out for is the one with a MAC address matching what you designate. You will have to find the MAC address of your NIC and populate this field. There are a number of ways to go about this if you need help.
- fixed-address 10.0.0.50; - This tells DHCP to always give the host the IP 10.0.0.50. This is important come later.
- filename "undionly.kpxe"; } - This tells DHCP to let PXE/iPXE know the name of the file for chain-loading for this client. This will also be touched on later.

Once you have your configuration built save & exit the file. From here we can start the server without rebooting using the command:

sudo /usr/local/etc/rc.d/isc-dhcpd start

You should see the output:

Starting dhcpd.
Internet Systems Consortium DHCP Server 4.4.2-P1
Copyright 2004-2021 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
Config file: /usr/local/etc/dhcpd.conf
Database file: /var/db/dhcpd/dhcpd.leases
PID file: /var/run/dhcpd/dhcpd.pid
Wrote 0 class decls to leases file.
Wrote 0 deleted host decls to leases file.
Wrote 0 new dynamic host decls to leases file.
Wrote 0 leases to leases file.
Listening on BPF/vtnet1/1a:30:11:9d:77:04/10.0.0.0/24
Sending on   BPF/vtnet1/1a:30:11:9d:77:04/10.0.0.0/24
Sending on   Socket/fallback/fallback-net

We can then verify the service is running with:

/usr/local/etc/rc.d/isc-dhcpd status

You should see an output similar but not identical to:

dhcpd is running as pid 98771.

Our DHCP server is ready for PXE/iPXE.

3.3.2 - Installing & Configuring TFTP (Trivial File Transfer Protocol)

Spoiler

TFTP is a network file transfer protocol for simple files. In our case PXE/iPXE is a one time executable usually baked into the network cards firmware. What TFTP enables us to do however is perform an operation known as Chain-loading where-in PXE or iPXE loads the latest version of iPXE from the network then loads whatever service is configured to come next.

3.3.2.1 - Installation

Spoiler

To start we need to add another line to /etc/rc.conf:

inetd_enable="YES"

This will start the TFTP service at system startup.

From here we need to un-comment two TFTP lines in /etc/inetd.conf:

...
# ntalk is required for the 'talk' utility to work correctly
#ntalk	dgram	udp	wait	tty:tty	/usr/libexec/ntalkd	ntalkd
tftp	dgram	udp	wait	root	/usr/libexec/tftpd	tftpd -l -s /tftpboot
tftp	dgram	udp6	wait	root	/usr/libexec/tftpd	tftpd -l -s /tftpboot
#bootps	dgram	udp	wait	root	/usr/libexec/bootpd	bootpd
...

To start the service without rebooting:

sudo /etc/rc.d/inetd start

To check if the service is working run the command:

netstat -an | grep "*.69"

You should see the output:

udp6       0      0 *.69                   *.*                    
udp4       0      0 *.69                   *.*

3.3.2.2 - Configuration

Spoiler

Run the following command to create the TFTP directory:

sudo mkdir /tftpboot

Now we need to create our iPXE images for PXE/iPXE to load from TFTP at system startup.

3.3.2.3 - Creating Custom iPXE Image Files

Spoiler

To make the iPXE files that our clients will boot from we have to compile them from source with an embedded script. Unfortunately at this time there isn't good documentation on how to achieve this using UNIX/FreeBSD. We will need to compile it on Linux. Any Debian based or similar distributions should suffice.

NOTE: If you do not have a machine running Linux running it in a Virtual Machine should be sufficient for the task so long as it get's an IP from your router and not through NAT.

From a Linux machine start by installing the Git package if you don't already have it. Distros like Ubuntu can accomplish this with:

sudo apt install git

If you are running something else where apt isn't your package manager ask for help and we'll get it sorted.

Now download the iPXE repository with:

git clone git://git.ipxe.org/ipxe.git

NOTE: If you intend to network boot multiple machines create a backup of the ipxe folder now!

From here we need to create our script. For each client we want to network boot we will need to create a unique iPXE file compiled from source with a custom script in each.

To create the script file open nano or vi/vim and write the following lines:

#!ipxe

dhcp
set initiator-iqn iqn.2021-11.ipxe.com:lun1
sanboot iscsi:10.0.0.254::::iqn.2021-11.ipxe.com:lun1

These tell iPXE what to do during the network boot process. You can copy this file character for character if you like with the exception that the IP address in the 5th line needs to point to your iSCSI server (more on this later). For now change the IP to match the Interface we added in /etc/rc.conf. Now save & exit the file editor naming the file anything you like.

To compile iPXE with the custom script run the commands:

cd ipxe/src
make bin/undionly.kpxe EMBED=/path/to/script

NOTE: You may run into errors stating you lack dependencies. In the event of this comment below to discuss what you're missing.

When the compiler finishes you can copy undionly.kpxe over the network using SFTP and drop it into our FreeBSD servers /tftpboot directory. If you need help with this let me know.

If you need multiple clients for network boot from here you would edit the script files last two lines to reflect the next available disk from iSCSI:

set initiator-iqn iqn.2021-11.ipxe.com:lun2
sanboot iscsi:10.0.0.254::::iqn.2021-11.ipxe.com:lun2

set initiator-iqn iqn.2021-11.ipxe.com:lun3
sanboot iscsi:10.0.0.254::::iqn.2021-11.ipxe.com:lun3

etc...

This is why you created a backup of the ipxe folder. Once ipxe is compiled you can't re-compile the same folder. You'll need as many copies of the source folder as you have planned clients but in the event you make a mistake you can always re-download the source from Git.

NOTE: With multiple clients each instance of the undionly.kpxe file will require a unique name in both the tftpboot folder & our DHCP server config this being denoted by the "filename":

host client-name1 {
hardware ethernet 00:02:c9:56:ab:aa;
fixed-address 10.0.0.50;
filename "undionly.kpxe0"; }

host client-name2 {
hardware ethernet 00:02:c9:56:ab:ab;
fixed-address 10.0.0.51;
filename "undionly.kpxe1"; }

host client-name3 {
hardware ethernet 00:02:c9:56:ab:ac;
fixed-address 10.0.0.52;
filename "undionly.kpxe2"; }

DHCP will tell the network adapter to go fetch the corresponding file name from TFTP. Since each has a custom script they need to be differentiated from one another.

The good thing about this method is each unique file isn't tied to the clients in any permanent manor. Which makes this not a bad method of going about managing multiple clients.

3.3.3 - Configuring iSCSI (Internet Small Computer Systems Interface)

Spoiler

On our FreeBSD server we need to create network accessible storage for our clients to connect to. PXE/iPXE supports a few different protocols for this but today this tutorial will focus on iSCSI. What iSCSI does is it provides block level storage over a TCP/IP network. One of the advantages of this being it can utilize both physical and/or virtual disks.

Getting started run this command to list available physical disks:

geom disk list

You will see an output similar to:

Geom name: da0
Providers:
1. Name: da0
   Mediasize: 68719476736 (64G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r3w3e5
   descr: QEMU QEMU HARDDISK
   ident: (null)
   rotationrate: unknown
   fwsectors: 63
   fwheads: 255

Geom name: cd0
Providers:
1. Name: cd0
   Mediasize: 971974656 (927M)
   Sectorsize: 2048
   Mode: r0w0e0
   descr: QEMU QEMU DVD-ROM
   ident: (null)
   rotationrate: unknown
   fwsectors: 0
   fwheads: 0

Geom name: da1
Providers:
1. Name: da1
   Mediasize: 137438953472 (128G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: QEMU QEMU HARDDISK
   ident: (null)
   rotationrate: unknown
   fwsectors: 63
   fwheads: 255

Geom name: da2
Providers:
1. Name: da2
   Mediasize: 137438953472 (128G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: QEMU QEMU HARDDISK
   ident: (null)
   rotationrate: unknown
   fwsectors: 63
   fwheads: 255

Each da device (next to Geon name: is a HDD/SSD. How you setup your server will affect how this appears to you.

Getting started an iSCSI service on FreeBSD comes pre-installed so all we have to do is enable & configure it. Let's start by creating a /etc/ctl.conf file with:

sudo nano /etc/ctl.conf

Within this file we're going to designate a series of parameters. Below is an example config file:

portal-group pg0 {
        discovery-auth-group no-authentication
        listen 10.0.0.254
}

target iqn.2021-11.ipxe.com:lun1 {
        auth-group no-authentication
        portal-group pg0

        lun 0 {
                path /dev/da1
        }
}

Breaking down what we're looking at above we have:

portal group pg0 - This is effectively our iSCSI disk manager to which you can designate a series of parameters that the chosen disks will abide by.
- discovery-auth-group no-authentication - You can opt to have security on the iSCSI volume so only people with login credentials may access the drive. This is known as CHAP authentication. CHAP is beyond the scope of this tutorial however so we're using no-authentication meaning anyone can connect to the drive.
- listen - This tells iSCSI which network interfaces to listen on for incoming requests. It's done by interface IP.
target iqn.2021-11.ipxe.com:lun1 - This designation tells iSCSI that you're advertising a disk as being hosted for clients to connect to. LUN# is the default name commonly used to specify which disk you want your clients to use but you can change this to another naming convention if desired.
- auth-group no-authentication - Similar to what was discussed before if you want security on your drives you can investigate CHAP but declaring no-authentication means anybody can connect to this disk.
- portal-group pg0 - This is the interconnect to say what rules this target drive abides by. In this example file it says there's no authentication required to connect & the disk is only accessible out the network interface with the assigned IP 10.0.0.254.
- lun 0 - This is where we connect iSCSI to what drive we want it to use.
  - path /dev/da1 - This tells iSCSI that for the target iqn.2021-11.ipxe.com:lun1 it will use the raw disk /dev/da1.

After this is created save & exit the file. Now to start iSCSI at system startup edit /etc.rc.conf and add the line:

ctld_enable="YES"

Save & exit.

To start the iSCSI service without rebooting run the command:

sudo service ctld start

You should see the output:

Starting ctld.
ctld: /etc/ctl.conf is world-readable

With an iSCSI Initiator you should now be able to connect to the disk we assigned and view it as an unformatted drive.

4. Installing & Modifying the Client Operating System

Spoiler

At this point in time if you start your client PXE/iPXE should be able to get an IP from the FreeBSD DHCP server which will in turn provide the IP to the FreeBSD TFTP server with the filename of our custom undionly.kpxe image file. PXE/iPXE will then download (chain-load) the updated version of iPXE from TFTP and go through the same initialization process starting with acquiring an IP. Setting the iSCSI Initiator. Then attempting a sanboot where it will fail.

*The image below is for demonstration purposes only and may not reflect what you will see with your hardware.

The fact that it failed is fine though as we haven't installed an OS to the SAN disk but that will change in this chapter.

4.1 - Installation Setup

Spoiler

Start by inserting our Ubuntu Server Installer USB into the target server/client. Start the computer and go into the BIOS. Before we boot from the USB stick we want to set the BIOS/CSM parameters under the Boot Menu to Legacy/BIOS only. We don't want UEFI. The reason being the undionly.kpxe file is for Legacy boot only. iPXE supports a ipxe.efi file for UEFI capable systems but I could never make it work. Once you save your changes and reboot if there's no other boot media the USB should start. You'll see a lot of text on a black screen but if you just hang back and wait you will eventually drop to the menu depicted below:

On the next menu you can choose weather to update the installer or opt not to. It won't change the behavior of what we need to do.

Now pick your keyboard layout.

From here things get a little interesting. What you see below will vary depending on how you setup your server but you should see at least one interface with the IP you specified in the DHCP service on the FreeBSD server. If you have multiple interfaces you can choose to change the other interface(s) to Static based on your needs but leave DHCP enabled on the NIC that iPXE is using.

If you need a proxy to reach the internet you can enter those details below. Otherwise click Done.

Now you can change the default Ubuntu archive if you need to. NOTE: At this point DO NOT CLICK DONE!

From here we need to connect to the iSCSI drive, otherwise the installer cannot see the disk on the network. To do this from the current menu press Tab until the highlighted menu option is the Help icon in the upper right corner then press Enter.

From the drop-down menu use the arrow keys to select Enter Shell then press Enter.

This will drop you into a live shell:

From here we're going to run a couple commands starting with a discovery request.

iscsiadm -m discovery -t sendtargets -p 10.0.0.254

Breaking this down we use the iscsiadm command and make a discovery request for the target names hosted on the server IP 10.0.0.254. In layman's terms we're yelling out over the network to our iSCSI/DHCP/TFTP FreeBSD server and asking it if it's hosting any disks that we can use.

You should get a output reply similar to as follows:

10.0.0.254:3260,257 iqn.2021-11.ipxe.com:lun1

If you setup multiple disks on the FreeBSD iSCSI service you will have multiple lines of output. This lets Ubuntu Server know that there is a iSCSI disk available to use on the network.

Now to connect to this disk we run the command:

iscsiadm --mode node --targetname iqn.2021-11.ipxe.com:lun1 --portal 10.0.0.254 --login

You should see the output:

Logging in to [iface: default, target: iqn.2021-11.ipxe.com:lun1, portal: 10.0.0.254,3260] (multiple)
Login to [iface: default, target: iqn.2021-11.ipxe.com:lun1, portal: 10.0.0.254,3260] successful.

If you look at the FreeBSD server's console you should also see a response to the client request:

Dec  8 12:18:38 ipxe-server ctld[6791]: 10.0.0.50 (iqn.1993-08.org.debian:01:e0741deca62): read: Connection reset by peer

From our Ubuntu Server installer console you can now type exit and press enter. This will bring us back to the Archive mirror page we were on. You can now go ahead and press Done. This will bring up the following menu:

You may have to select it from the drop-down menu if you have any disks connected to the system but you will want to see the disk following a format similar to iqn.2021-11.ipxe.com:lun1,lun,0 local disk. You can chose if you want to setup a LVM group but it isn't necessary so you can un-checkmark it if desired.

From here click Done.

NOTE: What you do not want to see!

If you see that the installer wants to create a /boot/efi 512.000M partition then the system will not boot when we finish the installation. This is UEFI which isn't compatible with undionly.kpxe. You will need to go back into the BIOS and recheck the Boot CSM settings and verify that you set it to Legacy/BIOS boot only.

NOTE: What you do want to see!

If you see that the installer wants to create a 1.000M BIOS grub spacer then you're all set to continue. Press Done. You'll receive a pop-up warning you that all the data on the disk will be lost. If you're certain that there's nothing on the physical/virtual disk that you want to preserve press Continue.

On the next menu setup username, system name, and password information.

After this you can choose to install OpenSSH for remote access purposes if you want it.

From here you can choose to install any listed snap packages if your application needs apply:

\

Now just wait for the installer to finish. This will be denoted by the option to Reboot Now. NOTE: Do not reboot, we need to make some additional changes!

4.2 - Modifying Installation

Spoiler

Now that the OS is installed we have to modify a couple of things so that:

The iSCSI driver loads at OS startup.
The OS itself knows where to go to connect to the SAN disk.
GRUB knows where to go to connect to the SAN disk.

Going back into the console (again that being Help -> Enter Shell) we need to switch the current session over to the iSCSI volume and that will be done by running the commands:

mount --bind /dev /target/dev
mount -t proc proc /target/proc
mount -t sysfs sys /target/sys
chroot /target
hostname -F /etc/hostname

This makes it so when we make changes to files the changes wont happen to the OS on our USB but the OS that's on the drive over the SAN.

4.2.1 - Editing Modules

Spoiler

To make sure that the open-iscsi driver loads when the system boots we need to add iscsi to the modules file. This can be done quite simply with:

echo "iscsi" >> /etc/initramfs-tools/modules

4.2.2 - Editing GRUB

Spoiler

For the uninitiated GRUB for Ubuntu/Linux is basically the equivalent to the Windows Boot-loader. Similar to Windows it hands off control of the hardware from the BIOS to the Linux Kernel and subsequently the OS. Because of how the software works it needs to be told where to go after iPXE handed control of the system over to it. This can be done by editing the GRUB file. To do this open the GRUB file in an editor with:

nano /etc/default/grub

You will see the following file:

# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
#   info -f grub -n 'Simple configuration'

GRUB_DEFAULT=0
GRUB_TIMEOUT_STYLE=hidden
GRUB_TIMEOUT=0
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="maybe-ubiquity"
GRUB_CMDLINE_LINUX=""

# Uncomment to enable BadRAM filtering, modify to suit your needs
# This works with Linux (no patch required) and with any kernel that obtains
# the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)
#GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"

# Uncomment to disable graphical terminal (grub-pc only)
#GRUB_TERMINAL=console

# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo'
#GRUB_GFXMODE=640x480

# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
#GRUB_DISABLE_LINUX_UUID=true

# Uncomment to disable generation of recovery mode menu entries
#GRUB_DISABLE_RECOVERY="true"

# Uncomment to get a beep at grub start
#GRUB_INIT_TUNE="480 440 1"

All we're interested in is the small section of lines that don't start with #.

GRUB_DEFAULT=0
GRUB_TIMEOUT_STYLE=hidden
GRUB_TIMEOUT=0
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="maybe-ubiquity"
GRUB_CMDLINE_LINUX=""

From here edit the line GRUB_CMDLINE_LINUX_DEFAULT to include the following information:

ip=dhcp ISCSI_INITIATOR=iqn.2021-11.ipxe.com:lun1 ISCSI_TARGET_NAME=iqn.2021-11.ipxe.com:lun1 ISCSI_TARGET_IP=10.0.0.254 ISCSI_TARGET_PORT=3260

NOTE: What you type will not be 100% identical. This is only an example. Replace the target names & IP's where appropriate for your configuration.

It should look similar to this when you're done:

GRUB_CMDLINE_LINUX_DEFAULT="ip=dhcp ISCSI_INITIATOR=iqn.2021-11.ipxe.com:lun1 ISCSI_TARGET_NAME=iqn.2021-11.ipxe.com:lun1 ISCSI_TARGET_IP=10.0.0.254 ISCSI_TARGET_PORT=3260"

From here save & exit the file.

To make our changes to GRUB permanent run the command:

update-grub

You'll see an output similar to:

Sourcing file `/etc/default/grub'
Sourcing file `/etc/default/grub.d/init-select.cfg'
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.4.0-91-generic
Found initrd image: /boot/initrd.img-5.4.0-91-generic
done

4.2.3 - Editing Initiator Name

Spoiler

The last thing we need to modify is the Initiator Name so the OS knows where the disk is. This is done by running the three commands below:

echo "InitiatorName=iqn.2021-11.ipxe.com:lun1" > /etc/iscsi/initiatorname.iscsi
touch /etc/iscsi/iscsi.initramfs
update-initramfs -u

You should see an output similar to:

update-initramfs: Generating /boot/initrd.img-5.4.0-91-generic

If all was done correctly we're ready to reboot the server. Type exit again then choose Reboot Now from the installer menu.

So what should happen now if you have PXE/iPXE set as your primary boot device is:

PXE/iPXE queries the FreeBSD DHCP service for it's assigned IP address meanwhile collecting the IP & Filename for TFTP.
PXE/iPXE queries the FreeBSD TFTP service for the file. Chainloads (downloads) it, and boots it.
The now updated iPXE image runs the special embedded script we wrote and:
1. Queries FreeBSD DHCP for it's assigned IP again.
2. Sets the initiator-iqn.
3. Makes a sanboot request to the initiator-iqn.
This starts loading GRUB (the bootloader). GRUB queries the FreeBSD iSCSI service based on how we edited GRUB_CMDLINE_LINUX_DEFAULT.
As the OS starts to load it starts the iscsi module and queries the FreeBSD iSCSI service based on how we edited /etc/iscsi/initiatorname.iscsi.

And finally after this process finishes the system wraps up what other data it needs to load into memory from disk. We have a successful SAN booted PC!

There is no GUI but one can be installed. Comment below if you're interested in that. Otherwise enjoy!