How to install SmartOS on a datacenter machine

This blogpost documents how to install SmartOS in a datacenter machine, with no remote KVM neither physical access.

SmartOS used to need a CD, DVD or USB pendrive to boot the Hypervisor (saving hacks like Installing/booting SmartOS in/from a harddisk without physical access), but since late 2020 you can install it in the ZFS ZPOOL and directly boot from there, using the new utility piadm.

1   Create a template ZPOOL image

Nota

You don't need to do this by yourself if you trust me. I host a SmartOS HardDisk image at https://download.jcea.es/tamiza/dump_SmartOS_10GB_20250124T134639Z_pass_jcea2025.xz. It is only 180 MB in size. The root password is jcea2025. Its SHA256 hash is f7ae2a59d58737a01850c9cb8696bbfe68b0a4b37d21b322005d076d054da562.

First step is to create a zpool image to deploy on the server:

  1. Download the mostly current version (I usually prefer to avoid last version and use the previous version, if not marked as toxic) of SmartOS.

  2. Launch Virtualbox, create a new machine with 4 GB of RAM and a HardDisk of 10 GB. The virtual machine CPU must be configured as 64 bits.

    Provide a virtual NIC card. That is, a network card.

    Use the SmartOS image as the virtual CD drive.

  3. Boot the virtual machine. SmartOS should enter a configuration script. Provide dummy data for most of it, we will override later.

    At the end, enable booting from the zones zpool in order to do a SmartOS install on the virtual HardDisk.

    Advertencia

    When creating the zones zpool, BE SURE it is created with ashift=12. Boot must be compatible with BIOS and UEFI machines. The easiest way to be sure is:

    1. When asked about the zpool, enter manual.

    2. Create the zpool manually:

      [root@ /]# zpool create -B -o ashift=12 zones c1t0d0

    3. Exit the shell and continue the configuration process.

  4. Eject the SmartOS virtual CD and reboot the virtual machine. Be sure it boots from the virtual HardDisk.

  5. We must destroy dataset zones/dump and recreate it with the right size. What is the right size? It is difficult to say. It depends of the RAM size, but it is not linear. The SmartOS configuration script uses dumpadm -e just after reset and multiply it by two.

    In this SmartOS image I will use 4 GB. For a 64 GB machine, SmartOS seems to require around 1.5 GB, so we are quite safe for now.

    [root@ /]# dumpadm -d none
[root@ /]# zfs destroy zones/dump
[root@ /]~# zfs create -V 4G zones/dump

  1. Be sure the machine can boot under BIOS and UEFI:

    [root@ /]# piadm bootable
zones                          ==> BIOS and UEFI

  2. Shutdown the virtual Machine and boot with a Linux, for instance, a Debian Live CD.

  3. Upload an image of the 10 GB virtual HardDisk with SmartOS installed to your cloud or file hosting service. Since most of the virtual HardDisk is empty, compressing it will be highly effective.

  4. You can now destroy the virtual machine. You don't need it anymore.

The image just created will be a template to deploy any number of remote datacenter machines.

I already did this for you. You can use the SmartOS HardDisk image I host at https://download.jcea.es/tamiza/dump_SmartOS_10GB_20250124T134639Z_pass_jcea2025.xz. Compressed, it is only 180 MB. The root password is jcea2025. Its SHA256 hash is f7ae2a59d58737a01850c9cb8696bbfe68b0a4b37d21b322005d076d054da562.

Advertencia

DO NOT FORGET to change that default password!.

2   Overwrite and configure the remote machine in a datacenter

  1. Rent a machine in a remote datacenter. SmartOS kernel is Illumos, a Solaris derivative. It is quite compatible with most server hardware. I advice you to use an Intel NIC, performant and well supported, but many other popular brands are supported.

  2. Boot the machine in rescue mode. This will be, typically, a Linux environment.

  3. Access to the remote machine in the rescue mode. This will tipically run on a ramdisk, leaving the HardDisks alone.

  4. Download the SmartOS you prepared. This will typically be a ramdisk but you will probably have plenty of RAM available and the SmartOS compressed HardDisk is tiny.

    You can download my SmartOS HardDisk image: https://download.jcea.es/tamiza/dump_SmartOS_10GB_20250124T134639Z_pass_jcea2025.xz. Its SHA256 hash is f7ae2a59d58737a01850c9cb8696bbfe68b0a4b37d21b322005d076d054da562. Feel free to use it.

    For instance, you can do something like:

    root@RESCUE:~# wget https://download.jcea.es/tamiza/dump_SmartOS_10GB_20250124T134639Z_pass_jcea2025.xz
root@RESCUE:~# sha256sum dump_SmartOS_10GB_20250124T134639Z_pass_jcea2025.xz
f7ae2a59d58737a01850c9cb8696bbfe68b0a4b37d21b322005d076d054da562  dump_SmartOS_10GB_20250124T134639Z_pass_jcea2025.xz

  5. Destroy any GPT label in the HardDisk. You can use gdisk /dev/sda. Press x for expert mode and z to delete any trace of GPT in the disk.

    Do this for all the server HardDisks.

    Advertencia

    BEWARE: This step will DESTROY the content of your remote HardDisk. Be sure that is what you want.

  6. Overwrite the HardDisk with the image. For instance, do something like this:

    root@RESCUE:~# xzcat dump_SmartOS_10GB_20250124T134639Z_pass_jcea2025.xz | \
               dd of=/dev/sda bs=65536 status=progress

root@RESCUE:~# partprobe

    Notice that this command expand the image directly over the HardDisk, you don't need to decompress first. that is nice because it is faster and you could run out of RAM in you rescue ramdisk.

    Advertencia

    BEWARE: This step will DESTROY the content of your remote HardDisk. Be sure that is what you want.

  7. My reference datacencer Linux rescue system doesn't have ZFS preinstalled, but it installs it at first use:

    root@RESCUE:~# zpool
The Hetzner Rescue System does not come with preinstalled ZFS support,
however, we will attempt to compile and install the latest release for you.
Please read the information below thoroughly before entering any response.

ATTENTION

This script will attempt to install the current OpenZFS release
which is available in the OpenZFS git repository to the Rescue
System. If this script fails, do not contact Hetzner Support, as
it is provided AS-IS and Hetzner will not support the installation
or usage of OpenZFS due to License incompatiblity (see below).
Due to github.com limitations, this script only works via IPv4.

Licenses of OpenZFS and Linux are incompatible

OpenZFS is licensed under the Common Development and Distribution
License (CDDL), and the Linux kernel is licensed under the GNU General
Public License Version 2 (GPL-2). While both are free open source
licenses they are restrictive licenses. The combination of them causes
problems because it prevents using pieces of code exclusively available
under one license with pieces of code exclusively available under the
other in the same binary.

Please be aware that distributing of the binaries may lead to infringing.

Press y to accept this.

    Press y and wait a few minutes.

  8. If you already have ZFS available, import the zpool you just wrote on your HardDisk:

    root@RESCUE:~# zpool import zones -f -R /mnt

    Nota

    Remember to use the root option: -R when you import the zpool.

  1. Be sure the ashift of the zpool is 12 (4096 bytes):

    root@RESCUE:~# zpool get ashift zones
NAME   PROPERTY  VALUE   SOURCE
zones  ashift    12      local

  2. Check the zpool:

    root@RESCUE:~# zpool list
NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
zones  9.50G  1.57G  7.93G        -     3.63T     0%    16%  1.00x    ONLINE  /mnt

root@RESCUE:~# zpool status
  pool: zones
 state: ONLINE
status: Some supported and requested features are not enabled on the pool.
        The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
config:

        NAME        STATE     READ WRITE CKSUM
        zones       ONLINE       0     0     0
          sda       ONLINE       0     0     0

errors: No known data errors

  3. All the SmartOS hypervisor configuration resides in zones/usbkey dataset. This dataset is tagged as legacy in the zpool by SmartOS, so was not automatically mounted. You must do it by hand:

    root@RESCUE:~# mkdir /tmp/x
root@RESCUE:~# mount -t zfs zones/usbkey /tmp/x
root@RESCUE:~# cd /tmp/x
root@RESCUE:/tmp/x#

  4. Configure there the file config with the necessary details, IPs, NIC MAC address, netmasks, hostname, default gateway, dns domain, and so on. You can get those details from the rescue Linux.

    You need to know how to configure SmartOS hypervisor. Check, for instance:

    You can add a SSH public key in that directory, and you can provide a new shadow file with a new password for root.

    Pay attention to the edits you do. If you make a mistake, SmartOS will not boot.

  5. When done, reboot!

3   First SmartOS boot!

  1. First thing you MUST do when first booting is to change the root password and do that change persistent. This is not trivial because SmartOS hypervisor is inmutable. You can see the necessary steps (and why they are needed) in Changing the Root Password in the Global Zone. Log into your new SmartOS machine and do:

    [root@xXx ~]# umount /etc/shadow
[root@xXx ~]# cp /usbkey/shadow /etc/shadow
[root@xXx ~]# passwd root
New Password:
Re-enter new Password:
passwd: password successfully changed for root
[root@xXx ~]# cp /etc/shadow /usbkey/shadow

  2. Expand the zpool, currently 10 GB in size, to the entire HardDisk:

    [root@xXx ~]# zpool online -e zones c1t0d0
[root@xXx ~]# zpool list
NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
zones  3,64T  4,58G  3,63T        -         -     0%     0%  1.00x    ONLINE  -

  3. Add the mirror HardDisk:

    [root@xXx ~]# zpool attach zones c1t0d0 c1t1d0
Make sure to wait until resilver is done before rebooting.

    There is almost no data yet, so the ZFS resilvering is fast:

    [root@xXx ~]# zpool status
  pool: zones
 state: ONLINE
  scan: resilvered 4,58G in 0 days 00:00:29 with 0 errors on Mon Feb 10 08:38:07 2025
config:

        NAME        STATE     READ WRITE CKSUM
        zones       ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            c1t0d0  ONLINE       0     0     0
            c1t1d0  ONLINE       0     0     0

errors: No known data errors

  4. Be sure we can boot from the mirror HardDisk, just in case:

    [root@xXx ~]# piadm bootable -r zones

  5. If the SmartOS image you are using is old, consider updating the installed SmartOS hypervisor using piadm.

  6. When you are done messing with the configuration, reboot just to be sure you are golden and enjoy your new SmartOS machine!

4   Things to consider

  1. You can retrieve the configuration details of your remote machine and edit the SmartOS virtual machine in VirtualBox. The new configuration will not correctly boot in the virtual machine but, if you did everything correct, I should boot when copied over the target remote machine HardDisk.

    In this case: you must size the ZFS zpool bigger, at least 10GB plus RAM size of the real machine. dump

  2. You will most probably want to expand the swap space. You can follow instructions in Spanish at Cómo cambiar el tamaño del espacio de intercambio (SWAP) en SmartOS.

  3. Remember to change the SmartOS default "root" password I provided in the image!.