1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
|
---
title: "Use systemd-nspawn to Create a Development Sandbox"
date: 2025-03-04T23:22:23+08:00
lastmod: 2025-03-27T17:46:24+08:00
---
*systemd-nspawn* is a great tool for creating development sandboxes. Compared to
other similar technologies, it's lightweight, flexible, and easy to use. In this
blog, I'll present a simple guide to using it.
<!--more-->
## Advantages
I've been using traditional VMs and Docker for creating development
environments. While both work fine, regardless of the performance, they suffer
from being overly isolated. Two big headaches for me are host network sharing in
traditional VMs and the immutability of Docker container ports and mounts.
*systemd-nspawn* is much more flexible. Every feature can be configured
granularly and dynamically. For example, filesystem sharing can be configured to
work like bind mounts, and network isolation can be disabled entirely, which
exactly solves the two headaches mentioned above. Additionally, being part of
*systemd*, it has the same excellent design as other *systemd* components.
Debian has a similar powerful tool called *schroot*. It is the official tool for
automatic package building. Unfortunately, it seems to be a tool specific to
Debian.
## Usage
*systemd-nspawn* consists of two parts that work together to achieve its VM
functionality:
1. The program `systemd-nspawn`, which runs other programs in an isolated
environment with user-specified settings. Each running VM is essentially a
group of processes launched via `systemd-nspawn`.
2. Components for defining and managing VMs, possibly utilizing
`systemd-nspawn`.
*systemd-nspawn* has a user interface similar to *systemd service*:
- `[name].service` => `[name].nspawn`: Define VMs.
- Should be placed in `/etc/systemd/nspawn/`, where `machinectl` scans for VM
definitions.
- `[name]` serves as the VM's name. Use it to specify the VM when calling
`machinectl`. Note: You'd better use a valid hostname (avoid illegal
characters like `.`) to prevent weird errors.
- The options available roughly mirror `systemd-nspawn`'s CLI arguments, with
some adjustments to better fit VM semantics.
- Isolation-related options are usually prefixed with `Private` (e.g.,
`PrivateUsers=`).
- `systemctl` => `machinectl`: Manage VMs.
- `enable`/`disable`: Set whether the VM starts automatically at system boot.
- `start`/`poweroff`/`reboot`/`terminate`/`kill`: Control the VM's running
state.
- `login`/`shell`: Do things inside the VM.
I'll demonstrate how to create a Debian-based VM on Arch Linux as an example.
You should adjust the commands based on your own situation.
### Create Root Filesystem
The root filesystem of a distribution can be created using a special tool from
its package manager. For Debian-based distributions, it's `debootstrap`. If your
OS uses a different package manager ecosystem, the target distribution's one and
its keyrings (which might reside somewhere else) have to be installed first.
```bash-session
# pacman -S debootstrap debian-archive-keyring ubuntu-keyring
```
Regular directories work perfectly as root filesystems, but other directory-like
things should work, too, such as `btrfs` subvolume.
```bash-session
# btrfs subvolume create /var/lib/machines/[name]
```
Now, run `debootstrap` to create a minimal filesystem. Update the command with
the target distribution's codename and one of its mirrors you select.
```bash-session
# debootstrap --include=dbus,libpam-systemd [codename] \
/var/lib/machines/[name] [mirror]
```
At this point, the filesystem contains only the distribution's essential
packages, much like a base Docker image (e.g., `debian`), so you can customize
it in a similar way.
### Configure and Customize
I'll present my personal configuration here as a reference. You can create a new
one based on it or from scratch.
1. Disable user isolation: `[Exec] PrivateUsers=no`
2. Disable network isolation: `[Network] Private=no`
3. Create a user with the same username, group name, UID and GIDs: should be
done inside the VM.
4. Only bind a subdirectory under *home*: `[Files] Bind=/home/[user]/[subdir]`
5. Set the hostname: `[Exec] Hostname=[hostname]`
I disable user isolation because it's implemented using the kernel's user
namespace, which adds many inconveniences due to UID/GID mapping.
So, the final `.nspawn` file is like:
```systemd
/etc/systemd/nspawn/[name].nspawn
---
[Exec]
PrivateUsers=no
Hostname=[hostname]
[Files]
Bind=/home/[user]/[subdir]
[Network]
Private=no
```
If `machinectl` can already start the VM, you can log in to customize it
further. Otherwise, you can use `systemd-nspawn` directly to enter the VM and
run commands inside it. `--resolv-conf=bind-host` binds the host's
`/etc/resolv.conf` file to make the network work.
```bash-session
# systemd-nspawn --resolv-conf=bind-host -D /var/lib/machines/[name]
```
Now, inside the VM, you can do whatever you like. In my configuration, a correct
user must be created manually.
```bash-session
# apt install locales sudo nano vim less man bash-completion curl wget \
build-essential git
# dpkg-reconfigure locales
# useradd -m -G sudo -s /usr/bin/bash [user]
# passwd [user]
```
Some setup may need to be done manually, especially those usually handled by the
distribution's installer.
1. Update `/etc/hostname` with the VM's real hostname.
2. Update `/etc/hosts`.
```plain
/etc/hosts
---
127.0.0.1 localhost [hostname]
::1 localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
```
**Ubuntu 20.04 specific:** Due to [a bug in
systemd](https://github.com/systemd/systemd/issues/22234), the backport source
has to be added.
```plain
/etc/apt/sources.list
---
deb https://mirrors.ustc.edu.cn/ubuntu focal main restricted universe multiverse
deb https://mirrors.ustc.edu.cn/ubuntu/ focal-updates main restricted universe multiverse
deb https://mirrors.ustc.edu.cn/ubuntu/ focal-backports main restricted universe multiverse
deb https://mirrors.ustc.edu.cn/ubuntu/ focal-security main restricted universe multiverse
```
### Use
The following command starts a new shell session for the specified user inside
the VM, where you can run commands and perform tasks.
```bash-session
# machinectl shell [user]@[name]
```
Another way is to use `login` command to enter the *login console*. From there,
you can log in as a user to start a shell session.
```bash-session
# machinectl login [name]
```
To exit a VM session (especially in the *login console*), press `CTRL+]` three
times quickly in a row.
### Snapshot
The easiest way to backup/snapshot a VM is to create an archive of the VM's
filesystem. You can use any archive tool you prefer, such as the simple `tar`.
If the VM's filesystem is a `btrfs` subvolume, native `btrfs` snapshots can be
used here. Before creating a snapshot, you should power off the VM to avoid
archiving runtime files.
```bash-session
# machinectl poweroff [name]
# btrfs subvolume snapshot /var/lib/machines/[name] \
/var/lib/machines/btrfs-snapshots/[name]/[snapshot-name]
```
`machinectl` also provides an *image* feature similar to Docker, though I've
never tried it. Feel free to explore it if you're interested!
|
