Multi Operating System Boot Management for G4 Workstation Clusters (v. 0.2) * DHCP server * TFTP server * Open Firmware * Boot script * Blessed folder * symbolic link * boot-device * boot-partition * kernel So.. When the power is turned on, Open Firmware is executed. It has a large number of tasks to perform, including probing for devices, initializing devices, assigning devices to input and output handlers and finally, it must load and execute the desired operating system. In the powerpc we can run a number of different operating systems including Linux, MacOS-X, MacOS9, and FreeBSD. The problem: ============ The machines we wish to use to form a workstation cluster for research purposes are primarily intended to be used by students of the University. For their academic projects they may be required to use MacOS9. Certainly their instructors expect these machines to boot into MacOS9 flawlessly in the mornings. For our cluster we wish to use Linux as this is probably the best supported platform and certainly the most open for creating workstation clusters. It is possible and we intend to have MacOS9, MacOSX, and Linux all installed on each computers hard drive. One can change which operating system is booted up at power on by changing the setting in the non-volatile memory. This poses a problem, since the procedure to do this resembles magical incantations, requiring specialized knowledge and precise operations in a most user unfriendly interface. Were someone to leave the computers set to boot linux, there would many upset students, instructors and system administrators the next day. The solution: ============= We can control which operating system gets loaded by giving commands from the network. We set the computers open firmware to retrieve a boot script from a master control server upon power up. This boot script then determines which OS is boot up. Issues: ======= If the master control server is offline then the machines will be stumped. Experience varies, sometimes they give up after 5 minutes and boot to macos, sometimes they keep trying. Two machines were observed to wait 18 minutes before being reset. This is not a new problem however, as all the machines will start up very slowly - if at all if the dhcp server is offline. If the dhcp and tftp services run on the same server, then this is essentially the identical problem. This can issue can be addressed by having a backup DHCP server, that comes online if the other one goes offline and also runs a backup tftp server. One could also broadcast tftp ERR packets to the lab to cause the open firmware to error out on the TFTP. This of course requires having a second machine monitor the status of the TFTP server. It may simply be possible to instead specify two TFTP servers in the DHCP response packet. Then the second server could instruct open firmware to boot macos9 by default (thus assuming that there is some failure with the master control server). A better solution would be to patch the open firmware with a nvramrc script that would limit the TFTP retries and timeout period, and pushing the default boot when hit. This could be done by someone with more knowledge and experience in forth than myself. [snip - The machines can also be told where to look for the tftp server by the dhcp server.] The boot process in detail: =========================== The most important settings in open firmware for the purposes of this discussion are: "boot-device" "auto-boot?" These settings are saved in a non-volatile storage device called nvram or sometimes pram. These setting (and all others) can be restored to factory defaults by holding down the C-O-p-r keys simultaneously during power up for two chime cycles. They can also be changed from within the operating system. atuo-boot? - if set to true, the computer will stop the boot process at the open firmware prompt, awaiting further human interaction. - if set to false, open firmware will try to find and executee a boot script at the device pointed to by the boot-device setting. The factory default boot-device setting is: "hd:,\\:tbxi" hd is an alias for device node which represents the master IDE drive on the first IDE interface. The ',' seperates the device from the "file" on the device. In this case, "\\:tbxi" does not represent a file but rather the "blessed system folder". The standard macos boot process is to have only one blessed system folder on a hard drive at a time, and this folder is (required?) normally in the root of the partition (file-system). Thus it is in a sense a "deferred variable" (forth terminology). At any rate, if we want to boot linux from the hard drive the correct boot-device setting is one that includes a partition number after the first colon. Thus we break normality and indeed have two blessed folders. I am quite sure how this works but it does. For optimum ease of use a "bootstrap" partition is used to hold the linux blessed folder, which is usually the root folder. The easiest way to configure this folder is when running linux to use the "ybin" program which reads configuration information from /etc/yaboot.conf, copies over the necessary boot files to the bootstrap partition (defined in yaboot.conf) and then blesses the partition/folder with "holy penguin pee". It can also set the boot-device in nvram. Thus on the first system we set up the boot-device setting is: "ultra1:9,\\:tbxi" 9 is the first useable partition on the ultra1 drive, which is an alias for the slave IDE drive on the first IDE interface. Linux boot process using yaboot: This is a two stage process, the first stage is a (blessed?) boot script written in forth that can present a boot menu giving the user a chance to boot multiple operating systems including macos9 or X, as well as linux. The implementation: =================== For the workstation cluster we wish to create, we will set the boot device to be "enet:130.60.61.34,bootme". "bootme" is the boot script written in forth, and it grabbed by open firmware from the tftp server indicated by the IP address "130.60.61.34". If we left the IP address unspecified (:0), then it would look on the tftp server of the same IP address of the DHCP server - or the IP address specified in the "next-server" parameter of the DHCP response packet. "enet" of course represents that we are using the ethernet device to perform these network operations. So, by changing the contents of the forth boot script named "bootme" we have complete control over which operating system gets booted. This process is a very rapid one requiring only a couple seconds for the second stage boot process to begin. In the case of macos9 this second stage involves loading of any macos boot scripts found and proceeding with the loading of the macos operating system. I am not too familiar with the details. In the case of linux, the second stage requires the yaboot program which is an ELF executable, and yaboot then looks for yaboot.conf, which should specify a linux kernel to be loaded, and a root partition to be mounted. This root partition can also be a ramdisk that gets loaded into ram. All these files are searched for at the same place that the "bootme" script was retrieved from, unless of course the bootme script pointed us elsewhere. This mostly completes the analysis I think. Wrapup: ======= From the cluster users perspective, the only operations which are necessary to bring up the cluster are to set a symbolic link to the proper bootme script, and then reboot all the workstations. This can be done either by walk through the lab and pressing the reset switches (probably frowned upon by the sysadmins as it could result in filesystem corruption on the currently running operating system) or by using the MacAdmin software which can be used to reboot macos9 clients. To reboot macosX and linux, one only needs to use ssh to connect to the machines and issue a shutdown or restart command. Peter Jordan 2001-10-04