k3s/tests/TESTING.md

Testing Standards in K3s

Testing in K3s comes in 5 forms:

• Unit
• Integration
• Smoke
• Performance
• End-to-End (E2E)
• Distros-test-framework

This document will explain when each test should be written and how each test should be generated, formatted, and run.

Note: all shell commands given are relative to the root k3s repo directory.

---

Unit Tests

Unit tests should be written when a component or function of a package needs testing. Unit tests should be used for "white box" testing.

Framework

All unit tests in K3s follow a Table Driven Test style. Specifically, K3s unit tests are automatically generated using the gotests tool. This is built into the Go vscode extension, has documented integrations for other popular editors, or can be run via command line. Additionally, a set of custom templates are provided to extend the generated test's functionality. To use these templates, call:

gotests --template_dir=<PATH_TO_K3S>/contrib/gotests_templates

Or in vscode, edit the Go extension setting Go: Generate Tests Flags
and add --template_dir=<PATH_TO_K3S>/contrib/gotests_templates as an item.

To facilitate unit test creation, see tests/util/runtime.go helper functions.

Format

All unit tests should be placed within the package of the file they test.
All unit test files should be named: <FILE_UNDER_TEST>_test.go.
All unit test functions should be named: Test_Unit<FUNCTION_TO_TEST> or Test_Unit<RECEIVER>_<METHOD_TO_TEST>.
See the etcd unit test as an example.

Running

go test ./pkg/... -run Unit

Note: As unit tests call functions directly, they are the primary drivers of K3s's code coverage metric.

---

Integration Tests

Integration tests should be used to test a specific functionality of k3s that exists across multiple Go packages, either via exported function calls, or more often, CLI commands. Integration tests should be used for "black box" testing.

See integration/README.md for more info.

---

Smoke Tests

Smoke tests are a collection of tests defined under the tests and fall into two categories: install and snapshotter. These tests are used to validate the installation and operation of K3s on a variety of operating systems. The sub-directories therein contain fixtures for running simple clusters to assert correct behavior for "happy path" scenarios. The test themses are Vagrantfiles describing single-node installations that are easily spun up with Vagrant for the libvirt and virtualbox providers:

• Install Script ➡️ scheduled nightly and on an install script change
  • CentOS 7 (stand-in for RHEL 7)
  • Rocky Linux 8 (stand-in for RHEL 8)
  • Rocky Linux 9 (stand-in for RHEL 9)
  • Fedora 37
  • Leap 15.5 (stand-in for SLES)
  • Ubuntu 22.04
• Snapshotter ➡️ on any code change
  • BTRFS (containerd built-in)
    • Leap 15.4

Format

When adding new installer test(s) please copy the prevalent style for the Vagrantfile. Ideally, the boxes used for additional assertions will support the default libvirt provider which enables them to be used by our Github Actions Workflow(s). See:

• install.yaml.
• snapshotter.yaml.

Framework

If you are new to Vagrant, Hashicorp has written some pretty decent introductory tutorials and docs, see:

• https://learn.hashicorp.com/collections/vagrant/getting-started
• https://www.vagrantup.com/docs/installation

Plugins and Providers

The libvirtprovider cannot be used without first installing the vagrant-libvirt plugin. Libvirtd service must be installed and running on the host machine as well.

Additionally, the vagrant-scp and vagrant-k3s plugins are required.

All three can be and can be installed with:

vagrant plugin install vagrant-scp vagrant-k3s vagrant-libvirt

Environment Variables

These can be set on the CLI or exported before invoking Vagrant:

• TEST_VM_CPUS (default ➡️ 2)
  The number of vCPU for the guest to use.
• TEST_VM_MEMORY (default ➡️ 2048)
  The number of megabytes of memory for the guest to use.
• TEST_VM_BOOT_TIMEOUT (default ➡️ 600)
  The time in seconds that Vagrant will wait for the machine to boot and be accessible.

Running

The Install Script tests can be run by changing to the fixture directory and invoking vagrant up, e.g.:

cd tests/install/rocky-8
vagrant up
# The following provisioners are optional. In GitHub Actions CI they are invoked
# explicitly to avoid certain timeout issues on slow runners
vagrant provision --provision-with=k3s-wait-for-node
vagrant provision --provision-with=k3s-wait-for-coredns
vagrant provision --provision-with=k3s-wait-for-local-storage
vagrant provision --provision-with=k3s-wait-for-metrics-server
vagrant provision --provision-with=k3s-wait-for-traefik
vagrant provision --provision-with=k3s-status
vagrant provision --provision-with=k3s-procps

The Snapshotter test requires that k3s binary is built at dist/artifacts/k3s. It is invoked similarly, i.e. vagrant up, but with different sets of named shell provisioners. Take a look at the individual Vagrantfiles and/or the Github Actions workflows that harness them to get an idea of how they can be invoked.

---

Performance Tests

Performance tests use Terraform to test large scale deployments of K3s clusters.

See perf/README.md for more info.

---

End-to-End (E2E) Tests

E2E tests cover multi-node K3s configuration and administration: bringup, update, teardown etc. across a wide range of operating systems. E2E tests are run nightly as part of K3s quality assurance (QA).

See e2e/README.md for more info.

---

Distros Test Framework

The acceptance tests from distros test framework are a customizable way to create clusters and perform validations on them such that the requirements of specific features and functions can be validated.

See distros-test-framework/README for more info.

---

Contributing New Or Updated Tests

We gladly accept new and updated tests of all types. If you wish to create a new test or update an existing test, please submit a PR with a title that includes the words <NAME_OF_TEST> (Created/Updated).