Building External Kernel Modules With Torizon
Introduction
It is often necessary to add a kernel module to a TorizonCore image. There are two possibilities:
- Add a kernel module that is available in the kernel source, or in-tree. In this case, please make a request in our community. If you tell us the exact kernel config that must be enabled and it does not have a negative impact on TorizonCore, we'll add it as a module.
To check whether a given kernel config is already enabled, you can simply check the kernel config of a module running TorizonCore (zcat /proc/config.gz) or you can search for the configuration on the toradex-kernel-cache repository, which stores the kernel config for official TorizonCore images. When searching in the repository, be sure to choose the correct branch for your TorizonCore version.
- Add an out-of-tree kernel module, in other words, a module that is external to the Linux kernel source tree. In this article, you will learn how to do it.
Some examples where this is necessary:
- When adding support to some device/peripheral you're using.
- When including some proprietary code.
You'll need to compile the kernel module's source code against the kernel source to make it compatible.
The typical way to do this in embedded Linux would be to add your kernel module via the Yocto Project/OpenEmbedded build system. However, this requires quite a bit of knowledge, and as we deliver TorizonCore as a binary distribution, we try to simplify this process greatly in a way that you don't need to fall back to a Yocto build.
This article complies with the Typographic Conventions for Torizon Documentation.
Pre-requisites
The pre-requisite to complete these instructions are:
- Prerequisites from Configure Build Environment for Torizon Containers installed.
- A Toradex SoM with Torizon installed.
- Having read the Torizoncore Builder - Getting Started article can be helpful.
- A TorizonCore image downloaded.
- Source code for whatever kernel module you want to build. We will provide an example code in the exercise below.
Installing TorizonCore Builder
To install TorizonCore Builder, read our statements on OS and shell compatibility, then follow the instructions below, in order.
Download the setup script into some writable directory in your system (here we use
~/tcbdir/):$ mkdir -p ~/tcbdir/ && cd ~/tcbdir/
$ wget https://raw.githubusercontent.com/toradex/tcb-env-setup/master/tcb-env-setup.shnoteIf you did this before then you can skip this step unless you want to update the setup script. When you source the script (next step) it will show you a warning message if it determines an update is advisable.
Source the script:
$ cd ~/tcbdir/
$ source tcb-env-setup.shMake sure to do this from every shell instance that you intend to use with TorizonCore Builder. For advanced usage, run
source tcb-env-setup.sh -h, or see the project README. The latter has information about using the early-access version of the tool for those interested.tip- Remember to source the setup script every time you start a new terminal (or shell instance).
- The setup script installs a Bash completion script for TorizonCore Builder, making it possible to autocomplete commands and parameters by just pressing the TAB key.
Beware that under Windows, you must pass extra parameters to the script when the use of the following commands is intended:
ostree serve: consult the ostree serve reference.
Verify that the command
torizoncore-builderis available:$ torizoncore-builder --help
Hello World Kernel Module Example
Begin by organizing your kernel module source code into a single directory; let's call this your hello-mod. At a minimum, your source code should have at least one source C file and one Makefile.
To get the kernel module example, called hello-mod, from our Github repository, run the command below:
$ git clone https://github.com/toradex/hello-mod
Ihe files below are the source C file and the Makefile, respectively, contained in this hello-mod example:
#include <linux/module.h>
int init_module(void)
{
printk("Hello World!\n");
return 0;
}
void cleanup_module(void)
{
printk("Goodbye Cruel World!\n");
}
MODULE_LICENSE("GPL");
Take extra notice of the Makefile. In particular, notice the environment variable $(KERNEL_SRC). The TorizonCore Builder tool uses this variable, so include this in your Makefile as shown below. Alternatively, you can also use $(KDIR) instead. You do not need to define both these variables, but including at least one is mandatory.
obj-m := hello.o
SRC := $(shell pwd)
all:
$(MAKE) -C $(KERNEL_SRC) M=$(SRC) modules
clean:
$(MAKE) -C $(KERNEL_SRC) M=$(SRC) clean
Development and Production: Add External Kernel Modules with TorizonCore Builder
Applying an External Kernel Module to a Custom Image
There are two possible approaches to apply the customization and generate a custom Toradex Easy Installer image, described in the next two sections Approach 1 and Approach 2. These approaches in some cases are interchangeable and in some not as described in the next sections.
To learn about TorizonCore Builder workflow and the different approaches to use the tool, with explanatory diagrams, please refer to the TorizonCore Builder - Workflow article.
Both approaches generate a custom Toradex Easy Installer image as output, so the approaches should be followed alternatively and not in sequence.
Approach 1: Applying an External Kernel Module to a Custom Image Using the Build Command
TorizonCore Builder build command generates a custom TorizonCore image with the external kernel module built, ready to be installed with Toradex Easy Installer, named torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM in the example below. This is achieved using a configuration YAML file, tcbuild.yaml as default.
This is the recommended approach on production programming and on CI/CD (continuous integration / continuous development) pipelines.
To learn about TorizonCore Builder workflow and the different approaches to use the tool, with explanatory diagrams, please refer to the TorizonCore Builder - Workflow article.
It requires a Toradex Easy Installer image of TorizonCore (preferably without containers), torizon-core-docker-colibri-imx6-Tezi_5.3.0+build.7.tar in this case, as input. The directory with the kernel module source code is passed as customization: kernel: modules: source-dir.
# Sample configuration file:
input:
easy-installer:
local: images/torizon-core-docker-colibri-imx6-Tezi_5.3.0+build.7.tar
# Sample customization: build hello-mod module into the image
customization:
kernel:
modules:
- source-dir: hello-mod/
autoload: no
output:
easy-installer:
local: torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM
Notice the kernel: modules: autoload parameter of the hello-mod kernel module defined to no. This means that the kernel module will not be automatically loaded on boot time. To enable loading on boot time just change this parameter to yes.
The directory with the kernel module source code hello-mod should be available before running the build command.
Build The Custom Image
To generate the TorizonCore image, run the command below, in the same directory where the tcbuild.yaml file is:
$ torizoncore-builder build
...
1091 metadata, 12741 content objects imported; 412.2 MB content written
Pulling done.
Deploying OSTree with checksum 58629613a342197c31c5911d0874aac1b0fcb46b68a63f59760c03bacc4df08a
Deploying done.
Copy files not under OSTree control from original deployment.
Packing rootfs...
Packing rootfs done.
=>> Build command successfully executed!
In case of using a configuration file with a different name than tcbuild.yaml, run the command specifying the configuration file name:
$ torizoncore-builder build --file <configuration_file_name>
Deploy The Custom Toradex Easy Installer Image
To deploy the custom Toradex Easy Installer image to the board, click on the link below and choose between the available options.
Deploy The Custom Toradex Easy Installer Image
The output image can be deployed in three ways to the board:
- Using Toradex Easy Installer, the recommended method for production programming.
- Directly on the board through SSH, the recommended method during development.
- Through Torizon Platform Services server, the recommended method to perform updates on a device already deployed to the field.
Despite the recommendations, it is also possible to use a different method, using the Torizon Platform Services method during development for example.
To learn more about when to use each method, please refer to the Deployment Methods Comparison section of the TorizonCore Builder - Workflow article.
Toradex Easy Installer
Toradex Easy Installer Tool allows users to install an image into the internal flash memory of Toradex modules in an extremely simple way.
Copy the output image into a USB stick and follow the steps in the Toradex Easy Installer article linked above. The copy of the output image can be done with the command below:
$ cp -a torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM /media/user/myUSBstick
Where, in my case, /media/user/myUSBstick is the path to USB stick mount point and torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM is the directory containing the custom Toradex Easy Installer image.
Directly, through SSH
In this case, before deployment the output image needs to be unpacked, using the command below:
$ torizoncore-builder images unpack torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM
Change the argument torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM to the directory containing your custom Toradex Easy Installer image.
For more details on how this command works, please check the images unpack command in the commands manual.
To deploy it directly to the board, through SSH, use the command below:
$ torizoncore-builder deploy --remote-host 192.168.1.117 --remote-username torizon --remote-password torizon --reboot
For more details on how this command works, please check the deploy command in the commands manual.
Change the arguments --remote-host,--remote-username and --remote-password to your board IP Address, username and password, respectively.
From TorizonCore Builder 3.1.0 after, the default value of --remote-username and --remote-password is torizon, so if the username or the password is torizon the argument can be omitted.
This way does not support the deployment of the pre-provisioned containers.
Torizon Platform Services
In this case, before deployment the output image needs to be unpacked, using the command below:
$ torizoncore-builder images unpack torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM
Change the argument torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM to the directory containing your custom Toradex Easy Installer image.
For more details on how this command works, please check the images unpack command in the commands manual.
To deploy it to the Torizon Platform Update Service, use the command below:
$ torizoncore-builder platform push --credentials credentials.zip --package-name base-package --package-version base-package-1.0 base
For more details on how this command works, please check the platform push command documentation in the Commands Manual.
This way does not support the deployment of the pre-provisioned containers, so their Docker Compose file will need to be deployed separately in the server.
To deploy the image from the Torizon Platform Remote Update Service to the board, please follow the steps in the OTA Web Interface article.
To obtain credentials (credentials.zip file) and to obtain more information on how to sign and securely deploy a custom image using Torizon Platform Updates, check the Signing and pushing the image to Torizon OTA article.
Approach 2: Applying an External Kernel Module to a Custom Image Using Standalone Commands
In this second approach, instead of using a configuration YAML file and a one-step command, the generation of the custom TorizonCore with the external kernel modules is done using standalone commands, each performing one step towards this generation.
This approach is especially useful when making incremental changes, generating multiple images with different external kernel modules (or other customizations like different device tree overlays). As you will see, including an external kernel module on an image of TorizonCore that has other customizations (including other external kernel modules) is just a matter of performing the apply, merge and deploy stages. If you want to remove one or more kernel modules, it is also necessary to come back to the initial image of TorizonCore by executing the images unpack command again.
To learn about TorizonCore Builder workflow and the different approaches to use the tool, with explanatory diagrams, please refer to the TorizonCore Builder - Workflow article.
To generate a custom Toradex Easy Installer image with the desired external kernel modules follow the sequence of steps below.
Unpack an Input Image
You just need to execute this again if you want to remove one or more kernel modules that you have applied before to the image. Then, you are ready to apply multiple changes to the image. For example, in addition to applying external kernel modules, you can also apply a custom splash screen, a new device tree, among other possibilities.
If you have not unpacked an image yet, download a base TorizonCore image (preferably without containers) inside the TorizonCore Builder working directory, then run the command below to unpack it. In the example below the torizon-core-docker-colibri-imx6-Tezi_5.3.0+build.7.tar image is used as a reference:
$ torizoncore-builder images unpack torizon-core-docker-colibri-imx6-Tezi_5.3.0+build.7.tar
If you want to change the TorizonCore base image, download the new image and run the images unpack command again, passing the new image as the argument.
For more details about the images unpack command, please check the images unpack command in the commands manual.
Instead of using the images unpack you can use the images download command. This command checks which is the connected Toradex SoM, downloads the compatible latest quarterly release of a TorizonCore image without containers, and unpacks this image.
$ torizoncore-builder images download --remote-host 192.168.1.117 --remote-username torizon --remote-password torizon
Change the arguments --remote-host,--remote-username and --remote-password to your board IP Address, username and password, respectively.
For more details on how the images download command works, please check the images download command in the commands manual.
Build a Kernel Module
To build a kernel module from a directory with the kernel source code, hello-mod in this case, run the command kernel build_module below. After some time, if the compilation was successful, you'll see output similar to the following:
$ torizoncore-builder kernel build_module hello-mod
make: Entering directory '/workdir/hello-mod'
make -C /storage/linux M=/workdir/hello-mod
make[1]: Entering directory '/storage/linux'
AR /workdir/hello-mod/built-in.a
CC [M] /workdir/hello-mod/hello.o
Building modules, stage 2.
MODPOST 1 modules
CC [M] /workdir/hello-mod/hello.mod.o
LD [M] /workdir/hello-mod/hello.ko
make[1]: Leaving directory '/storage/linux'
make: Leaving directory '/workdir/hello-mod'
Kernel module(s) successfully built and ready to deploy.
All kernel module(s) have been built and prepared.
It is important to notice that the example directory with the kernel module source code named hello-mod builds a module named hello, as it can be seen in the line below, taken from the output of the kernel build_module command:
LD [M] /workdir/hello-mod/hello.ko
If you're building from two or more different sets of source code, you can run kernel build_module multiple times targeting each source directory separately. Each kernel module will be added in succession.
If you want a kernel module to load on boot, you should pass the argument --autoload to the kernel build_module command.
For more details about the kernel build_module command, please check the kernel build_module command in the commands manual.
Merge Changes
Merge the external kernel modules (as well as other customizations like a splash screen or a new device tree) into the base Toradex Easy Installer image of TorizonCore - use whatever branch name you want.
As an example, to commit changes into a branch named custom-branch use the command below, accordingly with the TorizonCore Builder version:
$ torizoncore-builder union custom-branch
Applying changes from STORAGE/dt.
Commit 58629613a342197c31c5911d0874aac1b0fcb46b68a63f59760c03bacc4df08a has been generated for changes and is ready to be deployed.
$ torizoncore-builder union --union-branch=custom-branch
We recommend that you switch to the latest version of TorizonCore Builder to enjoy its simpler and more consistent user interface besides other improvements and bug fixes.
For more details about the union command, please check the union command in the commands manual.
Deploy The Custom Toradex Easy Installer Image
To deploy the custom Toradex Easy Installer image to the board, click on the link below and choose between the available options.
Deploy The Custom Toradex Easy Installer Image
The output image can be deployed in three ways to the board:
- Using Toradex Easy Installer, the recommended method for production programming.
- Directly on the board through SSH, the recommended method during development.
- Through Torizon Platform Services server, the recommended method to perform updates on a device already deployed to the field.
Despite the recommendations, it is also possible to use a different method, using the Torizon Platform Services method during development for example.
To learn more about when to use each method, please refer to the Deployment Methods Comparison section of the TorizonCore Builder - Workflow article.
However, the commands have differences in comparison to the deploy in Approach 1.
Toradex Easy Installer
Toradex Easy Installer Tool allows users to install an image into the internal flash memory of Toradex modules in an extremely simple way.
First, to generate a Toradex Easy Installer image, into the output directory torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM, use the command below:
$ torizoncore-builder deploy custom-branch --output-directory torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM
Change the arguments custom-branch to the branch where you merged the changes (with the union command).
Copy the output image into a USB stick and follow the steps in the Toradex Easy Installer Tool article linked above. The copy of the output image can be done with the command below:
$ cp -a torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM /media/user/myUSBstick
Where /media/user/myUSBstick is, in my case, the path to USB stick mount point.
Directly, through SSH
To deploy it directly to the board, through SSH, use the command below:
$ torizoncore-builder deploy custom-branch --remote-host 192.168.1.117 --remote-username torizon --remote-password torizon --reboot
Change the arguments custom-branch, --remote-host,--remote-username and --remote-password to the branch where you merged the changes (with the union command), your board IP Address, username and password, respectively.
From TorizonCore Builder 3.1.0 after, the default value of --remote-username and --remote-password is torizon, so if the username or the password is torizon the argument can be omitted.
For more details on how this command works, please check the deploy command in the commands manual.
This way does not support the deployment of the pre-provisioned containers.
Torizon Platform Remote Update Service
To deploy it to the Torizon Platform Remote Update Service, use the command below:
$ torizoncore-builder push --credentials credentials.zip custom-branch
Change the argument custom-branch to the branch where you merged the changes (with the union command).
This way does not support the deployment of the pre-provisioned containers, so their Docker Compose file will need to be deployed separately in the server.
To deploy the image from the Torizon Platform Remote Update Service to the board, please follow the steps in the OTA Web Interface article.
To obtain credentials (credentials.zip file) and to obtain more information on how to sign and securely deploy a custom image using Over-the-Air (OTA) updates, check the Signing and pushing the image to Torizon OTA article.
For more details on how this command works, please check the push command in the commands manual.
Verifying The New Custom Image On The Device
After rebooting, in your target device's terminal, verify that your new custom image of TorizonCore is active on the device with the command below:
# sudo ostree admin status
Password:
* torizon 58629613a342197c31c5911d0874aac1b0fcb46b68a63f59760c03bacc4df08a.0
Version: 5.3.0+build.7-tcbuilder.20211008140217
origin refspec: tcbuilder:58629613a342197c31c5911d0874aac1b0fcb46b68a63f59760c03bacc4df08a
torizon 36ad904617b170339b6ded7b9dce87ed8cf0f76473b897fdd832d91e82eb1ddc.0 (rollback)
Version: 5.3.0+build.7
origin refspec: tcbuilder:36ad904617b170339b6ded7b9dce87ed8cf0f76473b897fdd832d91e82eb1ddc
Where 58629613a342197c31c5911d0874aac1b0fcb46b68a63f59760c03bacc4df08a is the OSTree commit hash and should be the same as:
- The one in the output of the
unioncommand in the case of the standalone commands - The one in the "Deploying OSTree with checksum ..." part of the output of the
buildcommand.
Load The External Kernel Module
After deploying the image, you should be able to use the modprobe command on the board to load your kernel module. As a note, by default, the kernel module will not load on boot. If you are using the interactive approach, you can change this behavior by adding the --autoload argument to the build_module command. If you are using the build command approach, you can set the autoload parameter to yes in the configuration file.
To load your kernel module, run the command below. It is important to notice that the example directory with the kernel module source code named hello-mod builds a module named hello, therefore we should enable a module named hello and not hello-mod.
# sudo modprobe hello
Differently from defining the kernel module to autoload on TorizonCore Builder, every time you reboot the device you should execute the modprobe command to load the kernel module.
Check The External Kernel Module
After deploying the image and loading the kernel module, you should be able to see your kernel module in the loaded kernel modules list. To do it, you can use the command below:
# lsmod
Module Size Used by
joydev 20480 0
stmpe_adc 16384 0
industrialio 65536 1 stmpe_adc
imx_sdma 28672 2
virt_dma 16384 1 imx_sdma
coda_vpu 73728 0
imx_vdoa 16384 1 coda_vpu
videobuf2_vmalloc 16384 1 coda_vpu
uio_pdrv_genirq 16384 0
uio 16384 1 uio_pdrv_genirq
hello 16384 0
openvswitch 110592 0
nsh 16384 1 openvswitch
nf_conncount 16384 1 openvswitch
nf_nat 36864 1 openvswitch
libcomposite 53248 0