Install a version of docker with gpu support (docker-ce = 19.03) # 2. Enter the project directory and build the wav2lip image: # docker build -t wav2lip. Allow root user to connect to the display. # xhost +local:root. Docker Desktop stores Linux containers and images in a single, large “disk image” file in the Mac filesystem. This is different from Docker on Linux, which usually stores containers and images in the /var/lib/docker directory. Where is the disk image file?
A Virtual Machine (VM) is an environment on a host computer thatcan be used as if it were a separate physical computer. VMs can beused to run multiple operating systems simultaneously on a singlecomputer. Operating systems running inside a VM see emulated virtualhardware rather than the actual hardware of the host computer. Thisprovides more isolation than Jails, although there isadditional overhead. A portion of system RAM is assigned to each VM,and each VM uses a zvol for storage. While a VMis running, these resources are not available to the host computer orother VMs.
FreeNAS® VMs use thebhyve(8)virtual machine software. This type of virtualization requires anIntel processor with Extended Page Tables (EPT) or an AMD processorwith Rapid Virtualization Indexing (RVI) or Nested Page Tables (NPT).
To verify that an Intel processor has the required features, useShell to run grepVT-x/var/run/dmesg.boot. If theEPT and UG features are shown, this processor can be used withbhyve.
To verify that an AMD processor has the required features, useShell to run grep POPCNT /var/run/dmesg.boot. If theoutput shows the POPCNT feature, this processor can be used withbhyve.
Note
AMD K10 “Kuma” processors include POPCNT but do not supportNRIPS, which is required for use with bhyve. Production of theseprocessors ceased in 2012 or 2013.
By default, new VMs have thebhyve(8)-H option set. This causes the virtual CPU thread to yieldwhen a HLT instruction is detected and prevents idle VMs from consumingall of the host CPU.
Virtual Machinesshows a list of installed virtual machines.
The (Options) menu has options for controlling and modifying VMs:
Start boots a VM. VMs can also be started by clicking theslide toggle on the desired VM.
An option is provided to Overcommit Memory. Memoryovercommitment allows multiple VMs to be launched when there is notenough free memory for all of them to run at the same time. Thisoption should be used with caution.
When active, the VM State changes toRUNNING. To start a VM when the host system boots, setAutostart.
Edit changes VM settings.
Delete removes the VM. Zvols used indisk devices and image files used inraw file devices are not removed when a VMis deleted. These resources can be removed manually inStorage ‣ Pools after it is determined that thedata in them has been backed up or is no longer needed.
Devices is used to add, remove, or edit devices attachedto a virtual machine.
Clone copies the VM. The new clone has
_cloneNappended to the name, whereNis the clone number.
These additional options in (Options) are available when a VM isrunning:
Power off immediately halts the VM. This is equivalentto unplugging the power cord from a computer.
Stop shuts down the VM.
Restart shuts down and immediately starts the VM.
VMs with Web Interface enabled show a VNCbutton. VNC connections permit remote graphical access to the VM.
Serial opens a connection to a virtual serial port on theVM.
/dev/nmdm1Bis assigned to the first VM,/dev/nmdm2Bis assigned to the second VM, and so on. Thesevirtual serial ports allow connections to the VM console from theShell.Tip
The nmdmdevice is dynamically created. The actual
nmdmXYnamevaries on each VM.To connect to the first VM, type
cu-l/dev/nmdm1B-s9600in the Shell. Seecu(1)for more information.
16.1. Creating VMs¶
Click ADD to open the wizardin Figure 16.1.1:
Select a virtual machine type from theVirtual Machine (VM) Wizard type. The choices areVirtual Machine (VM) and Docker Host.
The configuration options fora Virtual Machine (VM) type are described inTable 16.1.1.
| Screen # | Setting | Value | Description |
|---|---|---|---|
| 1 | Virtual Machine(VM) Wizard type | drop-down menu | Select the type of VM to create. |
| 2 | Guest OperatingSystem | drop-down menu | Choose the VM operating system type. Choices are: Windows, Linux, or FreeBSD. Seethis guidefor detailed instructions about using a different guest OS. |
| 2 | Name | string | Name of the VM. Alphanumeric characters and _ are allowed. The name must beunique. |
| 2 | Boot Method | drop-down menu | Select UEFI for newer operating systems, or UEFI-CSM (Compatibility Support Mode) forolder operating systems that only understand BIOS booting. VNC connections are only availablewith UEFI. |
| 2 | Start on Boot | checkbox | Set to start the VM when the system boots. |
| 2 | Enable VNC | checkbox | Add a VNC remote connection. Requires UEFI booting. |
| 2 | Bind | drop-down menu | VNC network interface IP address. The primary interface IP address is the default. Adifferent interface IP address can be chosen. |
| 3 | Virtual CPUs | integer | Number of virtual CPUs to allocate to the VM. The maximum is 16 unless limited by the hostCPU. The VM operating system might also have operational or licensing restrictions on thenumber of CPUs. |
| 3 | Memory Size (MiB) | integer | Allocate the amount of RAM inmebibytes for the VM. |
| 4 | Disk image | check optionwith customfields | Select Create new disk image to create a new zvol on an existing dataset.This is used as a virtual hard drive for the VM. Select Use existing disk imageand choose an existing zvol from the Select Existing zvol drop-down. |
| 4 | Select Disk Type | drop-down menu | Select the disk type. Choices are AHCI and VirtIO. Refer toDisk Devices for more information about these disk types. |
| 4 | Size (GiB) | integer | Allocate the amount of storage in GiB for the new zvol. |
| 4 | Select zvol | drop-down menu | When Create new disk image is chosen, select a pool or dataset for the new zvol.When Use existing disk image is chosen, select an existing zvol for the VM. |
| 5 | Adapter Type | drop-down menu | Intel e82545 (e1000) emulates the same Intel Ethernet card. Thisprovides compatibility with most operating systems. VirtIO providesbetter performance when the operating system installed in the VM supports VirtIOparavirtualized network drivers. |
| 5 | MAC Address | string | Enter the desired MAC address to override the auto-generatedrandomized MAC address. |
| 5 | Attach NIC | drop-down menu | Select the physical interface to associate with the VM. |
| 6 | Optional: Chooseinstallation mediaimage | browse button | Click (Browse) to select an installer ISO or image file on the FreeNAS® system. |
| 6 | Upload ISO | checkbox andbuttons | Set to upload an installer ISO or image file to the FreeNAS® system. |
The final screen of the Wizard displays the chosen options for the newVirtual Machine (VM) type. Click SUBMIT to create the VM orBACK to change any settings.
This example creates a FreeBSD VM:
- Virtual Machine (VM) Wizard type is set toVirtual Machine (VM).
- Guest Operating System is set to FreeBSD.Name is set to samplevm. Other options are left atdefaults.
- Virtual CPUs is set to 2 andMemory Size (MiB) is set to 2048.
- Create new disk image is selected. The zvol size is setto 20 GiB and stored on the pool named pool1.
- Network settings are left at default values.
- A FreeBSD ISO installation image has been selected and uploaded tothe FreeNAS® system. The Choose installation media imagefield is populated when the upload completes.
- After verifying the VM Summary is correct,SUBMIT is clicked.
Figure 16.1.2 shows the confirmation stepand basic settings for the new virtual machine:
Fig. 16.1.2 Creating a Sample Virtual Machine
16.2. Adding Devices to a VM¶
Go toVirtual Machines, (Options) ‣ Devices,and click ADD to add a new VM device.
Select the new device from the Type field. These devices areavailable:
- VNC Interface (only available on virtual machineswith Boot Loader Type set to UEFI)
Virtual Machines ‣ (Options) ‣ Devicesis also used to edit or delete existing devices. Click (Options) fora device to display Edit, Delete,Change Device Order, and Details options:
- Edit modifies a device.
- Delete removes the device from the VM.
- Change Device Order sets the priority number for bootingthis device. Smaller numbers are higher in boot priority.
- Details shows additional information about the specificdevice. This includes the physical interface and MAC address in aNIC device, the path to the zvol in a DISK device, and the pathto an
.isoor other file for a CDROM device.
16.2.1. CD-ROM Devices¶
Adding a CD-ROM device makes it possible to boot the VM from a CD-ROMimage, typically an installation CD. The image must be present on anaccessible portion of the FreeNAS® storage. In this example, a FreeBSDinstallation image is shown:
Note
VMs from other virtual machine systems can be recreated foruse in FreeNAS®. Back up the original VM, then create a new FreeNAS®VM with virtual hardware as close as possible to the original VM.Binary-copy the disk image data into the zvolcreated for the FreeNAS® VM with a tool that operates at the levelof disk blocks, likedd(1).For some VM systems, it is best to back up data, install theoperating system from scratch in a new FreeNAS® VM, and restore thedata into the new VM.
16.2.2. NIC (Network Interfaces)¶
Figure 16.2.3 shows the fields that appear aftergoing toVirtual Machines ‣ (Options) ‣ Devices,clicking ADD, and selecting NIC as theType.
The Adapter Type can emulate an Intel e82545 (e1000)Ethernet card for compatibility with most operating systems. VirtIOcan provide better performance when the operating system installed inthe VM supports VirtIO paravirtualized network drivers.
By default, the VM receives an auto-generated random MAC address. Tooverride the default with a custom value, enter the desired addressin MAC Address. Click GENERATE MAC ADDRESS toautomatically populate MAC Address with a new randomizedMAC address.
If the system has multiple physical network interface cards, use theNIC to attach drop-down menu to specify whichphysical interface to associate with the VM.
Set a Device Order number to determine the boot order ofthis device. A lower number means a higher boot priority.
Tip
To check which interface is attached to a VM, start the VMand go to the Shell. Type ifconfig and find thetap interface that showsthe name of the VM in the description.
16.2.3. Disk Devices¶
Zvols are typically used as virtual hard drives.After creating a zvol, associate it with the VMby clickingVirtual Machines ‣ (Options) ‣ Devices,clicking ADD, and selecting Disk as theType.
Open the drop-down menu to select a created Zvol, then setthe disk Mode:
- AHCI emulates an AHCI hard disk for best software compatibility.This is recommended for Windows VMs.
- VirtIO uses paravirtualized drivers and can provide betterperformance, but requires the operating system installed in the VM tosupport VirtIO disk devices.
If a specific sector size is required, enter the number of bytes inDisk sector size. The default of 0 uses an autotune scriptto determine the best sector size for the zvol.
Set a Device Order number to determine the boot order ofthis device. A lower number means a higher boot priority.
16.2.4. Raw Files¶
Raw Files are similar to Zvol disk devices,but the disk image comes from a file. These are typically used withexisting read-only binary images of drives, like an installer diskimage file meant to be copied onto a USB stick.
After obtaining and copying the image file to the FreeNAS® system,clickVirtual Machines ‣ (Options) ‣ Devices,click ADD, then set the Type to Raw File.
Click (Browse) to select the image file. If a specific sector sizeis required, choose it from Disk sector size. The Defaultvalue automatically selects a preferred sector size for the file.
Setting disk Mode to AHCI emulates an AHCI hard diskfor best software compatibility. VirtIO uses paravirtualized driversand can provide better performance, but requires the operating systeminstalled in the VM to support VirtIO disk devices.
Set a Device Order number to determine the boot order ofthis device. A lower number means a higher boot priority.
Set the size of the file in GiB.
A Docker Host also has a password field. This is the loginpassword for the Docker Host.
16.2.5. VNC Interface¶
VMs set to UEFI booting are also given a VNC (Virtual NetworkComputing) remote connection. A standardVNCclient can connect to the VM to provide screen output and keyboard andmouse input.
Each VM can have a single VNC device. A|dockerhost| does not support VNC connections.An existing VNC interface can be changed by clicking (Options) andEdit.
Note
Using a non-US keyboard with VNC is not yet supported. As aworkaround, select the US keymap on the system running the VNC client,then configure the operating system running in the VM to use akeymap that matches the physical keyboard. This will enablepassthrough of all keys regardless of the keyboard layout.
Figure 16.2.6 shows the fields that appearafter going toVirtual Machines ‣ (Options) ‣ Devices,and clicking (Options) ‣ Editfor VNC.
Setting Port to 0 automatically assigns a port when the VMis started. If a fixed, preferred port number is needed, enter it here.
Set Delay VM Boot until VNC Connects to wait to start the VMuntil a VNC client connects.
Resolution sets the default screen resolution used for theVNC session.
Use Bind to select the IP address for VNC connections.
To automatically pass the VNC password, enter it into thePassword field. Note that the password is limited to 8characters.
To use the VNC web interface, set Web Interface.
Tip
If a RealVNC 5.X Client shows the errorRFBprotocolerror:invalidmessagetype, disable theAdapt to network speed option and move the slider toBest quality. On later versions of RealVNC, selectFile ‣ Preferences,click Expert, ProtocolVersion, thenselect 4.1 from the drop-down menu.
Set a Device Order number to determine the boot order ofthis device. A lower number means a higher boot priority.
16.3. Docker Host VMs¶
Dockeris open source software for automating application deploymentinside containers. A container provides a complete filesystem,runtime, system tools, and system libraries, so applications alwayssee the same environment.
Rancheris a web-based tool for managing Docker containers.
FreeNAS® runs the Rancher web interface within the Docker Host.
16.3.1. Docker Host Requirements¶
The system BIOS must have virtualization support enabled for aDocker Host to work properly. On Intel systems this is typically anoption called VT-x. AMD systems generally have an SVM option.
20 GiB of storage space is required for the Docker Host.
For setup, the SSH service must be enabled.
The Docker Host requires 2 GiB of RAM while running.
16.3.2. Creating Docker Host¶
Figure 16.3.1 shows the Wizard that appearsafter going toVirtual Machines,clicking ADD, and selecting Docker Host as theVirtual Machine (VM) Wizard type.
Docker Host configuration options are described inTable 16.3.1.
| Screen # | Setting | Value | Description |
|---|---|---|---|
| 1 | Virtual Machine(VM) Wizard type | drop-down menu | Choose the type of VM to create. |
| 2 | Name | string | A descriptive name for the Docker Host. Alphanumeric characters and _are allowed. |
| 2 | Start on Boot | checkbox | Set to start this Docker Host when the FreeNAS® system boots. |
| 3 | Virtual CPUs | integer | Number of virtual CPUs to allocate to the Docker Host. The maximum is 16 unlessthe host CPU limits the maximum.The VM operating system can also have operational or licensing restrictions onthe number of CPUs. |
| 3 | Memory Size (MiB) | integer | Allocate this amount of RAM in MiB for the Docker Host. A minimum 2048 MiB ofRAM is required. |
| 4 | Adapter Type | drop-down menu | Intel e82545 (e1000) emulates the same Intel Ethernet card. Thisprovides compatibility with most operating systems. VirtIO providesbetter performance when the operating system installed in the VM supports VirtIOparavirtualized network drivers. |
| 4 | MAC Address | string | Enter the desired MAC address to override the auto-generatedrandomized MAC address. |
| 4 | Attach NIC | drop-down menu | Select the physical interface to associate with the VM. |
| 5 | Raw filename | string | Name of the disk image for the Docker Host to use as storage. |
| 5 | Raw filenamepassword | string | Alphanumeric password added to the raw file. This is used to log in to theDocker Host. The default is docker. |
| 5 | Raw file size | integer | Set the size of the new raw file. |
| 5 | Raw file location | browse button | Select a directory to store the new raw file. |
| 5 | Disk sector size | integer | Define the disk sector size in bytes. Default leaves the sector size unset. |
Choose the base options for the VM at each step of the wizard.Virtual CPUs is set to 1. Memory Size mustbe set to at least 2048 MiB.
The Network Interface step is automatically populated withsystem defaults. Customize these fields as necessary and pressNEXT to continue.
The Storage Files section of the wizard contains optionsto create and store a raw file. Add a filename by typing an .imgname in the Raw filename field. Enter a number ofgigabytes for the Raw file size. Set the raw file locationwith the folder button or by typing a directory in the field.
Docker Raw Format Examples
The final screen of the Wizard displays the chosen options for the newDocker Host. Click SUBMIT to create the Host orBACK to change any settings. Click CANCEL at anytime to return to theVirtual Machinespage.
Fig. 16.3.2 Docker Host Configuration
Docker Raw Format Software
Click (Options) and Serial tolog in to the Docker Host. Enter rancher for the user nameand docker for the password.
The default password is changed in the Devices by stoppingthe Docker Host, clicking (Options), and Devices. Click (Options) and Edit for the RAW device andenter a new value in the password field. Passwords cannotcontain spaces.
16.3.3. Start the Docker Host¶
Go toVirtual Machinesand find the entry for the new Docker Host. Click (Options) andStart to boot the Docker Host.
A Docker Host can take several minutes to boot. Click (Options) and –> Serial to view the Docker Hostactivity during startup. Use this console to configure Rancher insidethe Docker Host.
When the RancherOS console graphic is shown, press Enter to seethe ClientHostlogin: prompt. Enter the usernamerancher and press Enter. If a custom password was setin the raw file, enter it now. Otherwise, enter the default password ofdocker. The [rancher@ClientHost~]$ prompt isshown.
16.3.4. SSH to the Docker Host¶
Go toVirtual Machines,find the Docker Host entry, and locate the Com Port. Comport names have the format /dev/nmdm1B, where {1B}is unique for each VM.
Connect to the FreeNAS® server with an SSH client. The SSH servicemust be running with Login as Root with Password enabled.
At the FreeNAS® console prompt, connect to the Docker Host withcu-l/dev/nmdm1B, replacing {1B} with theDocker Host Com Port.
If the terminal does not immediately show a rancherlogin:prompt, press Enter. The Docker Host can take several minutes tostart and display the login prompt.
16.3.5. Installing and Configuring Rancher¶
Docker.raw File Size
Ensure Rancher has functional networking and can ping anoutside website.
If ping returns an error, adjust the VMNetwork Interface and reboot the VM.
Download and install the Rancher server withsudo docker run -d --restart=unless-stopped -p 8080:8080 rancher/server.
If a CannotconnecttotheDockerdaemon error is shown,enter sudo dockerd and trysudo docker run -d --restart=unless-stopped -p 8080:8080 rancher/serveragain. Installation time varies with processor and network connectionspeed. [rancher@ClientHost~]$ is shown when the installationis finished.
Enter ifconfig eth0 | grep 'inet addr' to view the RancherIP address. Enter the IP address followed by :8080 into a webbrowser to connect to the Rancher web interface. For example, if the IPaddress is 10.231.3.208, enter 10.231.3.208:8080in the browser.
The Rancher web interface takes a few minutes to start. The web browsermight show a connection error while the web interface starts. If aconnectionhastimedout error is shown, wait one minute andrefresh the page.
When the Rancher web interface loads, click Add a host fromthe banner across the top of the screen. Verify thatThis site’s address is chosen and click Save.
Follow the steps shown in the Rancher web interface and copy the fullsudodockerrun command from the text box. Paste it in theDocker Host shell. The Docker Host will finish configuring Rancher. A[rancher@ClientHost~]$ prompt is shown when theconfiguration is complete.
Go to the Rancher web interface and clickINFRASTRUCTURE ‣ Hosts.When a host with the Rancher IP address is shown, configuration iscomplete and Rancher is ready to use.
For more information on Rancher, see the Rancherdocumentation.
16.3.6. Configuring Persistent NFS-Shared Volumes¶
Rancher supports using a single persistent volume with multiplecontainers. This volume can also be shared with FreeNAS® using NFS.FreeNAS® must be configured with specific NFS permissions and aRancher NFS servermust have a properly configured stack scoped volume.
A stack scoped volume is data that is managed by a single Rancher stack.The volume is shared by all services that reference it in the stack.
Configure NFS sharing for a stack scoped volume by setting specificoptions in the command line of the Rancher NFS server and the FreeNAS®system:
- Log in to the Rancher NFS server and modify
/etc/exports. Addan entry for the NFS shared directory, typically/nfs, withseveral permissions options:/nfsIP(rw,sync,no_root_squash,no_subtree_check).IP is the IP address of the client and can also be set to thewildcard*. - In the FreeNAS® web interface, go toServices ‣ NFS Configure.Set Enable NFSv4 andNFSv3 ownership model for NFSv4. Click SAVEand restart the NFS service.
- Add
:nocopyto the end of the pool to be mounted:mount-tnfspool:/mnt/pool1:nocopy~nfsmounts/pool1_mount
Docker.raw
I wanted to write it as a comment toarticle of @Evgeny Shvarov . But it happens to be so long, so, decided to post it separately.
I would like to add a bit of clarification about how docker uses disk space and how to clean it. I use macOS, so, everything below, is mostly for macOS, but docker commands suit any platform.
While docker comes from Linux, it works there on the same filesystem by default. But on Windows and macOS, it works in a small virtual machine with its own Linux inside. And disk's space there is limited my settings in docker. In my case, I have configured it to use up to 112 GB.
So, when you actively working with Docker, your space inside will go out. You can check how docker spends all that space with the command.
In macOS with the latest versions of Docker, used the raw format of the disk (previously was qcow2). And together with APFS filesystem in macOS, this file may take up less physical space than file size itself. Look at these two commands.
ls command shows the size of my Docker.raw file as 104Gb, while du command shows real size on the disk and it is 88Gb
Well, docker system df showed me, that I can reclaim some space. Let's do this.
This command deletes any stopped containers and any non-tagged images which not in use by any tagged images. And can be deleted safely.
You maybe noticed that it was reclaimed just only 5.5 GB, while docker system df said about 55GB. That's because, df counts all not active images, not just dangled ones. If you want to delete all those images as well, you can use this command. Which deletes any images which not used in running containers at the moment. So, if you don't have any running containers, it will delete all local images.
I just reclaimed only dangled images and stopped containers. How much space my docker use now.
Docker Raw Format Download
As you can see, it uses already less size. ls will show the same result. The size of the file mostly grows.
But for macOS is mostly important how much space is used on a physical disk.
And as you can see it now 69GB, and it is about 19GB less than it was before.
So, for macOS users, it does not really matter how much size of the file, with APFS optimizations it can be less in reality.
Docker Raw Format Example
Another way to prune old images with some filter by creation date. Like this example, will delete all images which were created more than 10 days ago, but will keep images which currently in use by containers.