7.6. Troubleshooting K3s
This section offers troubleshooting information when the K3s installer or the K3s configuration checker fail.
7.6.1. Troubleshooting the K3s Installer
The LOCKSS Installer's install-k3s script installs K3s by executing Rancher's official K3s installer after making sure many firewall and DNS issues are resolved 1. However, the installer can still run into issues and fail. Some of the error messages you might encounter are documented below, but you may need to refer to the official K3s documentation or use a search engine to look up the specific error message.
7.6.1.1. Failed to apply container_runtime_exec_t
In some Fedora systems, you may see an error message similar to the following:
[ERROR] Failed to apply container_runtime_exec_t to /usr/local/bin/k3s, please install:
yum install -y container-selinux selinux-policy-base
yum install -y https://rpm.rancher.io/k3s/stable/common/centos/7/noarch/k3s-selinux-0.2-1.el7_8.noarch.rpm
The specific commands and version numbers may vary from the example above.
To resolve this problem:
7.6.1.2. k3s-selinux requires container-selinux
In some Oracle Linux 7 systems, you may see an error message similar to the following:
Error: Package: k3s-selinux-0.3-0.el7.noarch (rancher-k3s-common-stable)
Requires: container-selinux >= 2.107-3
You could try using --skip-broken to work around the problem
You could try running: rpm -Va --nofiles --nodigest
The specific commands and version numbers may vary from the example above.
This can occur in environments where the Oracle Linux 7 Addons Yum repository is not enabled by default, so Rancher's official K3s installer is unable to install the package container-selinux
automatically.
To resolve this problem:
7.6.2. Troubleshooting the K3s Configuration Checker
After installing K3s with install-k3s 1 and successfully running check-k3s 2, you can run the following command as root
6:
k3s check-config
This configuration checker runs through a more extensive series of tests, covering "required", "generally necessary", and "optional" aspects for K3s to operate.
As a rule of thumb, if k3s check-config ends successfully with STATUS: pass
, there is a good chance the K3s cluster is configured correctly.
Some failures, especially in "optional" aspects, may not actually prevent the cluster from working normally in the limited ways the LOCKSS system uses Kubernetes, but if possible they should be addressed. Some of the error messages you might encounter are documented below, but you may need to refer to the official K3s documentation or use a search engine to look up the specific error message.
7.6.2.1. iptables should be older than v1.8.0 or in legacy mode
In some instances, you may encounter an error message similar to the following:
iptables v1.8.4 (nf_tables): should be older than v1.8.0 or in legacy mode (fail)
This error message is generally spurious, because the LOCKSS Installer should have previously detected and offered to correct this issue in the circumstances where it applies, and Rancher has a documented bug report that the K3s configuration checker keeps reporting this issue even in circumstances where it does not apply 4.
If check-k3s ran successfully 2, your K3s cluster is probably running normally and you can ignore this error message even if you receive it.
If your system is running iptables version 1.8.0 or later in
nf_tables
mode via Alternatives, as can be the case in some Debian or Ubuntu systems, iptables needs to be switched tolegacy
mode via Alternatives. The configure-firewall script called by install-k3s is supposed to detect this condition and offer to fix it for you 1. See Troubleshooting iptables.
7.6.2.2. User namespaces disabled
In the RHEL/CentOS family of operating systems, you may receive the following error message:
RHEL7/CentOS7: User namespaces disabled; add 'user_namespace.enable=1' to boot command line
To resolve this issue 5:
Edit the file
/etc/default/grub
asroot
6.Look for the line beginning with
GRUB_CMDLINE_LINUX=
, for example:GRUB_CMDLINE_LINUX="no_timer_check console=tty0 console=ttyS0,115200n8 net.ifnames=0 biosdevname=0 elevator=noop crashkernel=auto"
Add
user_namespace.enable=1
to the space-separated list of boot arguments, for instance:GRUB_CMDLINE_LINUX="user_namespace.enable=1 no_timer_check console=tty0 console=ttyS0,115200n8 net.ifnames=0 biosdevname=0 elevator=noop crashkernel=auto"
Run the following command as
root
:grub2-mkconfig -o /boot/grub2/grub.cfg
Reboot the system.
Re-run k3s check-config 3.
7.6.2.3. swap should be disabled
This warning can be ignored:
swap: should be disabled
7.6.2.4. CONFIG_INET_XFRM_MODE_TRANSPORT missing
This warning can be ignored:
CONFIG_INET_XFRM_MODE_TRANSPORT: missing
Footnotes
- 1
See install-k3s.
- 2
See check-k3s.
- 3
See k3s-check-config.
- 4
References:
- 5
References:
- 6