Build Torizon OS from Source With Yocto Project/OpenEmbedded
Introduction
Torizon OS (formerly TorizonCore) is the base operating system in which Torizon is built. It is a minimal Yocto-based embedded Linux image and, even though customers may usually stick to application development on top of containers, sometimes they may need to customize the base image for their needs.
If you just want to start using Torizon OS as-is with no customization in the original image, see the Quickstart Guide for information on how to install it on your Toradex Computer-on-Module.
Before trying to build Torizon OS using Yocto Project/Open Embedded, briefly read what TorizonCore Builder can do. It's a Toradex-provided tool that can help in customizing your Torizon OS image, without having to deal with Yocto.
The Toradex manifest repository contains a Repo manifest and setup scripts for the Torizon OS build system. If you want to modify, extend or port Torizon OS to a new hardware platform, this is the manifest repository to use.
The build system uses various components from the Yocto Project, most importantly the OpenEmbedded build system, the bitbake task executor and various application and BSP layers.
You have two options to build Torizon OS (and possibly any Yocto-based image):
- Use a native Linux distribution like Ubuntu to build Torizon OS.
- Pros: has the advantage that you can easily understand what is going on.
- Cons: you may be stuck on a certain Distribution Version which is supported by the Yocto Release where Torizon OS is built on; you need to install all dependencies by yourself in every new development machine.
- Use a Docker container and build everything inside that container.
- Pros: it has the advantage that it will work independently of the version of the host distribution; it works under Windows; the commands used inside the container are the exact same as if you would build on the native distribution.
- Cons: it may seem to be more complicated at the first look.
You can build Torizon OS natively under your favorite Linux flavor. However, not all distribution version/Yocto version combos are allowed. If your distribution is not supported you will get a warning when running your first bitbake command. In such cases, it may happen that your current distribution can't build Torizon OS. You may consider trying to use a Docker container then.
This article complies with the Typographic Conventions for Torizon Documentation.
Common Prerequisites
The following information is something that you should determine before proceeding with either build option.
Manifest Branch
The manifest branch refers to the git branch of the manifest repository that will be used for setup. The available branches related to Torizon OS images are:
| Torizon OS version | Branch |
|---|---|
| 7.x.y | scarthgap-7.x.y |
| 6.x.y1 | kirkstone-6.x.y |
| 5.x.y2 | dunfell-5.x.y |
| 0.0.0 | master |
- Versions 6.3.0 and earlier were released with the TorizonCore name.
- All Versions 5.x.y were released with the TorizonCore name.
These branches come from the official Toradex Manifest Repository.
master is maintained to monitor changes in the external layers. Builds can be broken for a longer period of time. This branch is not recommended for productization.
Manifest File
Beware that the path to the manifest files has changed for Torizon 7! Before they were torizoncore/<manifest>.xml and now it is torizon/<manifest>.xml.
The manifest file refers to the specific file in the manifest repository that will be used for setup. The available files related to Torizon images are:
torizon/default.xmltorizon/integration.xmltorizon/next.xml
It is recommended to use torizon/default.xml. The other two manifest files are used by Toradex to monitor future changes and may not be stable.
Torizon OS Distro
Within the greater Torizon Cloud (formerly Torizon Platform), Toradex provides different variations of Torizon OS. In the OpenEmbedded community these are known as "Distros". For Torizon OS, Toradex provides the following Distros:
| Distro | Kernel Base | Kernel Config |
|---|---|---|
| torizon | Downstream kernel from SoC vendor (NXP) (default for iMX8* based modules) | Default (without the PREEMPT_RT patch) |
| torizon-upstream | Mainline kernel (default for iMX6/iMX7 based modules) | Default (without the PREEMPT_RT patch) |
| torizon-rt | Downstream kernel from SoC vendor (NXP) | Fully preemptive (real-time Linux PREEMPT_RT patch) |
| torizon-upstream-rt | Mainline kernel | Fully preemptive (real-time Linux PREEMPT_RT patch) |
Machine
Supported MACHINE targets (officially tested by Toradex):
| Machine Name | Corresponding Toradex Module |
|---|---|
apalis-imx6 | Apalis iMX6 |
apalis-imx8 | Apalis iMX8 |
colibri-imx7-emmc | Colibri iMX7D 1GB (equipped with eMMC flash) |
colibri-imx6 | Colibri iMX6 |
colibri-imx6ull-emmc | Colibri iMX6ULL 1GB (equipped with eMMC flash) |
colibri-imx8x | Colibri iMX8X V1.0C or newer |
verdin-imx8mm | Verdin iMX8M Mini |
verdin-imx8mp | Verdin iMX8M Plus |
verdin-am62 | Verdin AM62 (supported on 6.4.0 or newer Torizon OS releases) |
Torizon OS Image
Supported image targets for bitbaking:
| Image | Description |
|---|---|
| torizon-minimal | Torizon OS reference minimal image without a container engine All other Torizon OS features are included, such as remote and offline OS updates, device monitoring, and more Refer to Torizon OS Reference Minimal Image Technical Overview |
| torizon-docker | Default image supported by Toradex - Network Manager: NetworkManager - Init system: systemd - Container Engine: Docker - OTA Framework: OSTree & Uptane |
| torizon-podman | Experimental image using Podman instead of Docker Same Features as above except Container Engine is Podman |
Build Process
Choose from the tabs below to either do a native or a containerized build:
Prerequisites
You must have a host PC configured with the following prerequisites:
- Linux host with prerequisites for building a Yocto/OpenEmbedded image.
Optionally, if you plan to build an image with Secure Boot, refer to the article Secure Boot on Torizon OS, where additional dependencies and build configuration are listed.
Download Metadata
To configure the scripts and download the build metadata, do:
$ mkdir ~/bin
$ PATH=~/bin:$PATH
$ curl http://commondatastorage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
$ chmod a+x ~/bin/repo
In addition, make sure you have Git user name and e-mail configured:
$ git config --global user.email "you@example.com"
$ git config --global user.name "Your Name"
Run repo init to bring down the latest stable version of Repo. You must
specify a URL for the manifest, which specifies the various repositories that
will be placed within your working directory. Furthermore, a branch must be
specified. Available manifest branches are listed in the above table.
To check out the latest Torizon OS release, including monthly pre-releases:
$ repo init -u git://git.toradex.cn/toradex-manifest.git -b scarthgap-7.x.y -m torizon/default.xml
Alternatively, if you want to work on a specific version, check out using tags. Find out available tags on the Git repository. See the example below for 6.0.0:
repo init -u git://git.toradex.cn/toradex-manifest.git -b refs/tags/7.0.0 -m torizon/default.xml
For reproducible builds, you should at least lock in on a specific version. If reproducible builds are critical for your use case, consider fetching all source code and storing it on a local machine after the setup.
A successful initialization will end with a message stating that Repo is initialized in your working directory. Your client directory should now contain a .repo directory where files such as the manifest will be kept.
To pull down the metadata sources to your working directory from the repositories as specified in the repo manifest, run:
$ repo sync --no-clone-bundle
When downloading from behind a proxy (which is common in some corporate environments), it might be necessary to explicitly specify the proxy that is then used by repo:
$ export HTTP_PROXY=http://<proxy_user_id>:<proxy_password>@<proxy_server>:<proxy_port>
$ export HTTPS_PROXY=http://<proxy_user_id>:<proxy_password>@<proxy_server>:<proxy_port>
More rarely, Linux clients experience connectivity issues, getting stuck in the middle of downloads (typically during "Receiving objects"). It has been reported that tweaking the settings of the TCP/IP stack and using non-parallel commands can improve the situation. You need root access to modify the TCP setting:
$ sudo sysctl -w net.ipv4.tcp_window_scaling=0
$ repo sync -j1
Setup Environment
The default distribution (DISTRO) variable is automatically set based on the chosen MACHINE (either torizon or torizon-upstream). To choose another DISTRO, specify so in the below command, similar to MACHINE. For more information on available DISTROs please see the previous section Torizon OS Distro.
Setup the work environment by using the setup-environment script:
$ [MACHINE=<MACHINE>] source setup-environment [BUILDDIR]
If MACHINE is not provided, the script will list all possible machines and force one to be selected. For more information on available machines please consult the Machine list.
Additionally, if ACCEPT_FSL_EULA="1" was not previously added to conf/local.conf, a prompt will appear asking if you would like to read NXP's EULA (if a NXP MACHINE was selected) and if you accept the terms. If accepted, ACCEPT_FSL_EULA="1" will be automatically added to conf/local.conf, if not the build will fail until the EULA is accepted by adding ACCEPT_FSL_EULA="1" to conf/local.conf.
There's also a way to automatically accept the EULA and skip this whole prompt (very useful for automating tasks), by using the EULA variable:
$ [MACHINE=<MACHINE>] EULA=1 source setup-environment [BUILDDIR]
This will automatically accept NXP's EULA and add ACCEPT_FSL_EULA="1" to conf/local.conf.
Start Building
To build the Torizon OS image:
The above setup script should properly prepare the environment with correct defaults.
$ bitbake torizon-docker
The Torizon OS Build Environment is a project that extends CROPS to build a Torizon OS image. CROPS is a Docker image for building Yocto based distros which is maintained by the Yocto Project.
Prerequisites
The following prerequisites are mandatory:
- Linux / Windows host with Docker installed.
Additionally, the following is required on a Windows Machine only:
- WSL2
- A suitable WSL2 Linux Distribution (Ubuntu 18.04 was tested)
For more information on setting up WSL2 please see the section Install the Windows Subsystem for Linux (WSL2).
Basic Build Configuration
Linux Setup
Create a working directory where the Yocto build will be stored. For example:
$ cd ~
$ mkdir ~/yocto-workdir
Then run the following minimal docker run command to bring up the build container. Don't forget to substitute <machine> and <image> based on the module and Torizon OS image you want to build:
$ docker run --rm -it --name=crops -v ~/yocto-workdir:/workdir --workdir=/workdir -e MACHINE=<machine> -e IMAGE=<image> torizon/crops:scarthgap-7.x.y startup-tdx.sh
Windows Setup
For Windows due to the heavy Linux filesystem dependencies of OpenEmbedded it will be easier to setup a working directory via WSL2. This working directory will be used to store the resulting build files on your host machine.
First, open a Windows command prompt and execute wsl. This should put you in a prompt "inside" your installed WSL2 Linux distribution. From here you can setup a folder where you want to store the build files. For example, if you want to set this up inside the home directory then do as follows:
$ cd
$ explorer.exe .
This will open up your WSL2 home directory with Windows File Explorer. From here you can copy the exact file path for later use (i.e. \wsl$\Ubuntu\home\coj)
Once you've setup your desired working directory you can now run the following minimal docker run command via Windows command prompt (don't forget to substitute <machine> and <image> based on the module and Torizon OS image you want to build):
$ docker run --rm -it --name=crops -v \wsl$\Ubuntu\home\coj:/workdir --workdir=/workdir -e MACHINE=<machine> -e IMAGE=<image> torizon/crops:kirkstone-6.x.y startup-tdx.sh
Build Information
For either Windows or Linux, once you've ran the appropriate command, this will then start to setup and build a Torizon image of IMAGE for module MACHINE. For more information on valid options for these variables please consult the Common Prerequisites at the beginning of this article.
Before the build begins you will be prompted to accept the NXP/Freescale EULA which is required for some packages. Make sure to accept this prompt before leaving to get coffee!
Once the build is complete you can view the build output from either in the container or your host machine's working directory that was setup up prior.
Container Arguments
The above docker run commands were just minimal commands required to get the container to setup and start a build. However, there are additional arguments that can be passed to either further configure the build or change its behavior. These arguments are passed into the container as environment variables via the -e flag of docker run:
-e MACHINE=<machine>: This is the target machine that the image will be built for. This is required and will not default to any value if it is not set. For a list of all compatible machines see the Machine list.-e DISTRO=<distro>: This is the Torizon DISTRO that will be built. This will default to eithertorizonortorizon-upstreamdepending on the chosenMACHINEif this variable is not explicitly set. For other valid options please see the Torizon OS Distro list.-e BRANCH=<branch>: This is the manifest repository branch. If not passed it defaults toscarthgap-7.x.y. For a list of other options see the Manifest Branch. Alternatively, if you want to work on a specific version, check out using tags. Find out available tags on the Git repository. For example, to use 6.7.0 add-e BRANCH=refs/tags/7.0.0.-e MANIFEST=<manifest>: This is the specific manifest file that is to be used. If not passed it defaults totorizon/default.xml. For other options see the Manifest File.-e IMAGE=<IMAGE>: This is the Yocto build recipe that is to be built. If not passed the container will simply setup the build environment and then run a shell inside the without starting any build, for if you want to do further configurations before building. For a list of valid image recipes see the Torizon OS Images list.
Customization
The Yocto Project can be quite defying and hard to use. In the following documentation, we cover some basic functionality that is likely you will have to go through during your development:
It does not replace the official Yocto Project documentation, though. At the end of the article Build a Reference Image with Yocto Project, we provide links to it, and you will most likely need to study it to some degree.
Custom Layers
On Torizon OS all your custom layers must be version controlled by Git, due to how we include layer revision information with OSTree. More details are provided in the aforementioned article Hello World integration into OpenEmbedded.
Deploy to Torizon Cloud using Yocto
Prerequisites
You must have Java installed on your host PC:
$ sudo apt install openjdk-8-jre
Preparing Setup
Before building your custom Torizon OS image using Yocto, it is possible to set variables in order to automatically push the image to your Torizon Cloud account.
Adding the path to your credential.zip, which can be obtained following How to Get Torizon Cloud credential.zip, and specifying the hardware for the image build. This configuration will be added to conf/local.conf:
SOTA_PACKED_CREDENTIALS = "/path/to/credential.zip"
SOTA_HARDWARE_ID = "verdin-imx8mp"
After building the image with Yocto, your image will be automatically pushed to your Torizon Cloud account and can be checked on the Recent Packages tab.
Deploy to Torizon Cloud Using TorizonCore Builder
Request access to evaluate the Torizon Cloud premium features |
To be able to deploy your custom image to Torizon Cloud, you need to meet the following criteria:
- Maintain the OSTree and Aktualizr references unchanged at your Yocto Build setup. They are responsible for the system update and OTA process.
Then, all you need from Yocto is the resulting image in the Toradex Easy Installer format, which you will use with the TorizonCore Builder Tool to unpack the image and then push it to Torizon Cloud.
Read the article Signing and Pushing Torizon OS Images to Torizon Cloud for more details about this process.
Keep in mind, though, that once you build your own Torizon OS image you are fully responsible for regularly building and deploying updates as well.