3. Installing PC-BSD®¶

3.3.2.1. ZFS Overview¶

ZFS is an enterprise grade file-system, which provides many features including: support for high storage capacities, high reliability, the ability to quickly take snapshots, boot environments, continuous integrity checking and automatic repair, RAIDZ which was designed to overcome the limitations of hardware RAID, and native NFSv4 ACLs.

If you are new to ZFS, the Wikipedia entry on ZFS provides an excellent starting point to learn about its features. Additionally, FreeBSD Mastery: ZFS by Michael W Lucas and Allan Jude is a very helpful resource and is more specific to ZFS as it is implemented in FreeBSD.

These resources are also useful to bookmark and refer to as needed:

• ZFS Evil Tuning Guide
• FreeBSD ZFS Tuning Guide
• ZFS Best Practices Guide
• ZFS Administration Guide
• Becoming a ZFS Ninja (video)
• blog post explaining how ZFS simplifies the storage stack

The following is a glossary of terms used by ZFS:

Pool: a collection of devices that provides physical storage and data replication managed by ZFS. This pooled storage model eliminates the concept of volumes and the associated problems of partitions, provisioning, wasted bandwidth, and stranded storage. Thousands of filesystems can draw from a common storage pool, each one consuming only as much space as it actually needs. The combined I/O bandwidth of all devices in the pool is available to all filesystems at all times. The Storage Pools Recommendations of the ZFS Best Practices Guide provides detailed recommendations for creating the storage pool.

Mirror: a form of RAID where all data is mirrored onto two or more disks, creating a redundant copy should a disk fail.

RAIDZ: ZFS software solution that is equivalent to RAID5 in that it allows one disk to fail without losing data. Requires a minimum of 3 disks.

RAIDZ2: double-parity ZFS software solution that is similar to RAID6 in that it allows two disks to fail without losing data. Requires a minimum of 4 disks.

RAIDZ3: triple-parity ZFS software solution. RAIDZ3 offers three parity drives and can operate in degraded mode if up to three drives fail with no restrictions on which drives can fail.

Dataset: once a pool is created, it can be divided into datasets. A dataset is similar to a folder in that it supports permissions. A dataset is also similar to a filesystem in that you can set properties such as quotas and compression.

Snapshot: a read-only point-in-time copy of a filesystem. Snapshots can be created quickly and, if little data changes, new snapshots take up very little space. For example, a snapshot where no files have changed takes 0MB of storage, but if you change a 10GB file it will keep a copy of both the old and the new 10GB version. Snapshots provide a clever way of keeping a history of files, should you need to recover an older copy or even a deleted file. For this reason, many administrators take snapshots often (e.g. every 15 minutes), store them for a period of time (e.g. for a month), and store them on another system. Such a strategy allows the administrator to roll the system back to a specific time or, if there is a catastrophic loss, an off-site snapshot can restore the system up to the last snapshot interval (e.g. within 15 minutes of the data loss). Snapshots can be cloned or rolled back, but the files on the snapshot can not be accessed independently.

Clone: a writable copy of a snapshot which can only be created on the same ZFS volume. Clones provide an extremely space-efficient way to store many copies of mostly-shared data such as workspaces, software installations, and diskless clients. Clones do not inherit the properties of the parent dataset, but rather inherit the properties based on where the clone is created in the ZFS pool. Because a clone initially shares all its disk space with the original snapshot, its used property is initially zero. As changes are made to the clone, it uses more space.

ZIL: is effectively a filesystem journal that manages writes. The ZIL is a temporary storage area for sync writes until they are written asynchronously to the ZFS pool. If the system has many sync writes, such as from a database server, performance can be increased by adding a dedicated log device known as a SLOG (Secondary LOG). If the system has few sync writes, a SLOG will not speed up writes. When creating a dedicated log device, it is recommended to use a fast SSD with a supercapacitor or a bank of capacitors that can handle writing the contents of the SSD’s RAM to the SSD. If you decide to create a dedicated log device, the SSD should be half the size of system RAM as anything larger than that is unused capacity. Note that a dedicated log device can not be shared between ZFS pools and that the same device cannot hold both a log and a cache device.

L2ARC: ZFS uses a RAM cache to reduce read latency. If an SSD is dedicated as a cache device, it is known as an L2ARC and ZFS uses it to store more reads which can increase random read performance. However, adding a cache device will not improve a system with too little RAM and will actually decrease performance as ZFS uses RAM to track the contents of L2ARC. RAM is always faster than disks, so always add as much RAM as possible before determining if the system would benefit from a L2ARC device. If you have a lot of applications that do large amounts of random reads on a dataset small enough to fit into the L2ARC, read performance may be increased by adding a dedicated cache device. SSD cache devices only help if your working set is larger than system RAM, but small enough that a significant percentage of it will fit on the SSD. Note that a dedicated L2ARC device can not be shared between ZFS pools.

3.3.2.2. ZFS Layout¶

In “Advanced Mode”, the disk setup wizard allows you to configure your ZFS layout. The initial ZFS configuration screen is seen in Figure 3.3.5: ZFS Configuration.

If your system contains multiple drives and you would like to use them to create a ZFS mirror or RAIDZ*, check the box “Add additional disks to zpool” which will enable this screen. Any available disks will be listed in the box below the “ZFS Virtual Device Mode” drop-down menu. Select the desired level of redundancy, then the desired number of disks.

While ZFS will let you use disks of different sizes, this is discouraged as it will decrease storage capacity and ZFS performance.

The PC-BSD® installer supports the following ZFS configurations:

• mirror: requires a minimum of 2 disks.
• RAIDZ1: requires a minimum of 3 disks. For best performance, a maximum of 9 disks is recommended.
• RAIDZ2: requires a minimum of 4 disks. For best performance, a maximum of 10 disks is recommended.
• RAIDZ3: requires a minimum of 5 disks. For best performance, a maximum of 11 disks is recommended.
• stripe: requires a minimum of 2 disks. Note that a stripe does NOT provide ANY redundancy. If any disk fails in a stripe, all data in the pool is lost!

The installer will not let you save a configuration if your system does not meet the minimum number of disks required by that configuration. As you select a configuration, a message will indicate how many more disks you need to select.

To use multiple disks, select the type of configuration from the “ZFS Virtual Device Mode” drop-down menu, then check the box for each disk that you would like to add to that configuration. When finished, click the “Next” button to see the screen shown in Figure 3.3.6: Add L2ARC or ZIL.

This screen can be used to specify an SSD to use as an L2ARC or a secondary log device. Any available devices will be listed in the boxes in this screen. Note that a separate SSD is needed for each type of device. Refer to the descriptions for ZIL and L2ARC in the ZFS Overview to determine if your system would benefit from any of these devices before adding them in this screen. When finished, click “Next” to see the screen shown in Figure 3.3.7: Configure Encryption.

This screen can be used to configure full-disk encryption which is meant to protect the data on the disks should the system itself be lost or stolen. This type of encryption prevents the data on the disks from being available during bootup, unless the correct passphrase is typed at the bootup screen. Once the passphrase is accepted, the data is unencrypted and can easily be read from disk.

To configure full disk encryption, check the box “Encrypt disk with GELI”. This option will be greyed out if “GPT (Best for new hardware)” is not selected as GELI does not support MBR partitioning at this time. If needed, use the “Back” button to go back to the Advanced Mode screen and select the option “GPT (Best for new hardware)”. Once the “Encrypt disk with GELI” box is checked, input a good passphrase twice into the “Password” fields. This password should be long and something that you will not forget, but hard for other users, especially those who know you, to guess.

When finished, click “Next” to move on the screen shown in Figure 3.3.8: Default ZFS Layout.

Regardless of how many disks you selected for your ZFS configuration, the default layout will be the same. ZFS does not require separate partitions for /usr, /tmp, or /var. Instead, you create one ZFS partition (pool) and specify a mount for each dataset. A /boot partition is not mandatory with ZFS as the PC-BSD® installer puts a 64k partition at the beginning of the drive.

You can use the “Add” button to add additional mount points. You will only be prompted for the name of the mount point as size is not limited at creation time. Instead, the data on any mount point can continue to grow as long as space remains within the ZFS pool.

If you wish to set the swap size, click the “Swap Size” button. This will prompt you to enter a size in MB. If you have created a RAIDZ or mirror, a swap partition of the specified size will be created on each disk and mirrored between the drives. For example, if you specify a 2048MB swap size, a 2GB swap partition will be created on all of the specified disks, yet the total swap size will be 2GB, due to redundancy.

If you right-click any mount point, you can toggle between enabling or disabling any of the following ZFS properties.

• atime: when set to “on”, controls whether the access time for files is updated when they are read. When set to “off”, this property avoids producing write traffic when reading files and can result in significant performance gains, though it might confuse mailers and some other utilities.
• canmount: if set to “off”, the filesystem can not be mounted.
• checksum: automatically verifies the integrity of the data stored on disks. Setting this property to “off” is highly discouraged.
• compression: if set to “on”, automatically compresses stored data to conserve disk space.
• exec: if set to “off”, processes can not be executed from within this filesystem.
• setuid: if set to “on”, the set-UID bit is respected.

Once you click “Next”, the wizard will show a summary of your selections. If you wish to change anything, use the “Back” button to return to a previous screen. Otherwise, click “Finish” to leave the wizard and return to the “Disk Selection” screen.