Search by Tags

Build TorizonCore With Yocto

 

Article updated at 16 Apr 2020
Compare with Revision




Subscribe for this article updates

Introduction

TorizonCore is the base operating system in which Torizon is built. It is a minimal Yocto-based embedded Linux image and, even though customers may usually stick to application development on top of containers, sometimes they may need to customize the base image for their needs.

The toradex-torizon-manifest project on Github repository contains a Repo manifest and setup scripts for the TorizonCore build system. If you want to modify, extend or port TorizonCore to a new hardware platform, this is the manifest repository to use.

The build system uses various components from the Yocto Project, most importantly the OpenEmbedded build system, the bitbake task executor and various application and BSP layers.

Toradex TorizonCore is based on Linux microPlatform from foundries.io and therefore requires the meta-lmp layer.

You have two options to build TorizonCore. One is to use a native distribution like Ubuntu to build TorizonCore (see Native TorizonCore Build). This has the advantage that you can easily understand what is going on. However, you may be stuck on a certain Distribution Version which is supported by the Yocto Release where TorizonCore is build on. Second, you can use a Docker container and build everything inside that container (see Containerized TorizonCore Build). This option first seems more complicated but has the advantage that it will work independent of the version of the host distribution. It even works under Windows. The commands used inside the container are the exact same as if you would build on the native distribution.

This article complies to the Typographic Conventions for Torizon Documentation.

You can build TorizonCore native under your favorite Linux flavor. However, not all distribution version/Yocto version combos are allowed. If your distribution is not supported you will get a warning when running your first bitbake command. In such cases, it may happen that your current distribution can't build TorizonCore. We recommend following the "Containerized TorizonCore Build" article if this happens.

Prerequisites

You must have a host PC configured with the following prerequisites:

Download Metadata

To configure the scripts and download the build metadata, do:

$ mkdir ~/bin
$ PATH=~/bin:$PATH

$ curl http://commondatastorage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
$ chmod a+x ~/bin/repo

In addition, make sure you have Git user name and e-mail configured:

$ git config --global user.email "you@example.com"
$ git config --global user.name "Your Name"

Run repo init to bring down the latest stable version of Repo. You must specify a URL for the manifest, which specifies the various repositories that will be placed within your working directory. Furthermore a branch must be specified.

TorizonCore version Branch
4.0.0 zeus
0.0.0 master

Note: master is maintained to monitor changes in external layers. Builds can be broken for a longer period of time. This branch is not recommended for productization.

To check out the latest TorizonCore release:

$ repo init -u https://github.com/toradex/toradex-torizon-manifest -b zeus

A successful initialization will end with a message stating that Repo is initialized in your working directory. Your client directory should now contain a .repo directory where files such as the manifest will be kept.

To pull down the metadata sources to your working directory from the repositories as specified in the repo manifest, run:

$ repo sync --no-clone-bundle

When downloading from behind a proxy (which is common in some corporate environments), it might be necessary to explicitly specify the proxy that is then used by repo:

$ export HTTP_PROXY=http://<proxy_user_id>:<proxy_password>@<proxy_server>:<proxy_port>
$ export HTTPS_PROXY=http://<proxy_user_id>:<proxy_password>@<proxy_server>:<proxy_port>

More rarely, Linux clients experience connectivity issues, getting stuck in the middle of downloads (typically during "Receiving objects"). It has been reported that tweaking the settings of the TCP/IP stack and using non-parallel commands can improve the situation. You need root access to modify the TCP setting:

$ sudo sysctl -w net.ipv4.tcp_window_scaling=0
$ repo sync -j1

Setup Environment

Supported MACHINE targets (officially tested by Toradex):

  • colibri-imx7 - Colibri iMX7 (eMMC & raw NAND*)
  • colibri-imx6 - Colibri iMX6
  • colibri-imx6ull - Colibri iMX6ULL*
  • colibri-imx8x - Colibri iMX8X
  • apalis-imx6 - Apalis iMX6
  • apalis-imx8 - Apalis iMX8
  • apalis-imx8x - Apalis iMX8X
  • verdin-imx8mm - Verdin iMX8MM

Note: Raw NAND-based modules are currently supported only by experimental releases and they may not be supported in the stable versions of TorizonCore.

Supported image targets:

  • torizon-core-docker
  • torizon-core-podman

The default distribution (DISTRO) variable is automatically set to torizon, which is provided by the meta-toradex-torizon layer.

Setup the work environment by using the setup-environment script:

$ [MACHINE=<MACHINE>] source setup-environment [BUILDDIR]

If MACHINE is not provided, the script will list all possible machines and force one to be selected.

Start Building

To build the TorizonCore image:

The above setup script should properly prepare the environment with correct defaults. Additionally you'll want to modify the local.conf file by adding ACCEPT_FSL_EULA="1".

$ bitbake torizon-core-docker

The TorizonCore Build Environment is a project that extends CROPS to build a TorizonCore image. CROPS is a Docker image for building Yocto which is maintained by the YoctoProject. Our adaption for TorizonCore is available on Github.

Prerequisites

The following prerequisites are mandatory:

  • Linux / Windows host with Docker installed.

The following prerequisites are recommended, though optional:

The current article does not replace, but rather extends the following information. Carefully going through it is recommended:

Note: the instructions provided have been tested on Linux and Windows hosts. Things are expected to work on Mac OS but specific instructions are not provided.

Basic Build Configuration

Since Docker allows for great flexibility, as well as the Torizon/CROPS image used for the Yocto build, this article has a basic section where you are guided through building the Torizon image, and an advanced section, which in fact is just a compendium of additional or alternative configuration options.

Setup

A few steps are required before you are ready to stat a build. You need to execute those steps once every time you want to start a new build, but if you want to modify, access or continue an existing build those can be skipped.

On Windows

Open a Power Shell and create a Docker volume. Change the UID/GID to your user instead of root user, as it's required by the Torizon/CROPS container:

$ docker volume create yocto-workdir
$ docker run -it --rm -v yocto-workdir:/workdir busybox chown -R 1000:1000 /workdir

On Linux

Open a shell using a terminal emulator and create a directory to be shared with Torizon/CROPS container. Create the directory as user, don't use sudo:

$ cd ${HOME}
$ mkdir ${HOME}/yocto-workdir

Modify

You will most likely want to make changes to the default image, otherwise you would be using the pre-built release provided by Toradex. It is convenient that you have access from your host OS instead of inside a container, because then you can use your favorite IDE, text editor, graphical tools, etc.

On Windows

Since the filesystem / container is not a native Windows container, it is not possible to use the bind-mount method for accessing the build contents. The workaround proposed by the CROPS project is to serve the build directory using a Samba mount.

Open another Power Shell window and type:

$ docker run --rm -d -it --net host --name samba -v yocto-workdir:/workdir crops/samba

Now you can close this Power Shell window, since Samba container is running in the background. Open the File Explorer and access your workdir from local network. Use the address below:

\\10.0.75.2/workdir

On Linux

Since you are sharing a local directory with the container, you can access the files inside it right away. Just open a new terminal or the file explorer of your choice and go to the previously created ${HOME}/yocto-workdir.

Build

Now you are ready to start a build. You need to know for which Toradex SoM you are building and use the correct machine value in the next steps. Please consult the respective Torizon or regular BSP articles to get the correct machine value:

For instance, if you have a Colibri iMX6 you'll use MACHINE=colibri-imx6 in the next steps whenever required.

You will also have to decide what target image you want to build. See the resources below for some options:

For instance, if you want to build the default Torizon image with Docker runtime you'll use TARGET=torizon-core-docker in the next steps.

Follow the steps according to your host OS, machine and (optionally) target image.

On Windows

Start the Torizon/CROPS container using the recently created volume as working directory. Give the container a name, you might use it later. In our example the name is crops:

$ docker run --rm -it --name=crops -v yocto-workdir:/workdir torizon/crops-toradex --workdir=/workdir --cmd="MACHINE=<machine> TARGET=<target> startup-tdx-torizon.sh"
 
$ docker run --rm -it --name=crops -v yocto-workdir:/workdir torizon/crops-toradex --workdir=/workdir --cmd="MACHINE=colibri-imx6 TARGET=torizon-core-docker startup-tdx-torizon.sh"

On Linux

Start the Torizon/CROPS container using the recently created directory as working directory. Give the container a name, you might use it later. In our example the name is crops:

$ docker run --rm -it --name=crops -v ${HOME}/yocto-workdir:/workdir torizon/crops-toradex --workdir=/workdir --cmd="MACHINE=<machine> TARGET=<target> startup-tdx-torizon.sh"
 
$ docker run --rm -it --name=crops -v ${HOME}/yocto-workdir:/workdir torizon/crops-toradex --workdir=/workdir --cmd="MACHINE=colibri-imx6 TARGET=torizon-core-docker startup-tdx-torizon.sh"

Once the build finishes, you will see a prompt similar to the one presented below, which means you are inside a container shell. It is an environment ready for Yocto builds where you can run or re-run builds, build single components and pretty much anything related to Yocto.

pokyuser@3bbac563cacd:/workdir$

Install

Once the build is done, there are two ways to flash the image to the SoM:

  • Bring up a container that serves the image over HTTP and install over Ethernet.
  • Copy the image to your host machine and use an SD card or USB flash drive.

Serve Over HTTP

Open a new shell to bring-up a trivial plain HTTP file server using a Python container. Notice that this server is a convenience for a quick and dirty test, but is not meant for a production environment:

$ docker run --rm -it -v yocto-workdir:/workdir --expose 8080 -p 8080:80 python:3.7-alpine python3 -m http.server --directory=/workdir/torizon/build-torizon/ 80

Find the IP of your computer. Hint: on Power Shell you can type ipconfig and on Linux shell you can type ip addr.

On the graphic interface of Toradex Easy Installer running on the board, add your local server to the feeds:

http://<your PC IP>:8080/image_list.json

Once the list of images is refreshed, you will see your image ready to be installed on the SoM.

Use an SD Card or USB Stick

You can access the final image either from the Samba mount or your bind-mount directory. Decompress it into an SD card or USB flash drive, insert it into the board and follow instructions how to install an image using the Toradex Easy Installer.

If for some reason you don't have the Samba container running on Windows, or have chosen to use a volume on Linux instead of a bind-mount, you can copy the compressed Toradex Easy Installer to your host machine using docker cp. See the instructions in the collapsible section below:

Copy image from Docker container to your computer

Advanced Build Configuration

This section goes into more detail about using the Torizon/CROPS container.

Setup

This section talks about how to keep different mount points for different parts of the Yocto setup:

Mount Downloads, Deploy and sstate-cache in Separate Directories or Volumes

The basic setup uses a single volume or bind-mount for your entire workdir. You could, for instance, have separate ones for downloads, sstate-cache and the deploy directory that could be shared with other builds or backed-up in case you want to wipe the rest of your workdir:

$ mkdir ${HOME}/yocto-downloads
$ mkdir ${HOME}/yocto-sstate-cache
$ mkdir ${HOME}/yocto-deploy

You would need to mount those inside the Torizon/CROPS container, in the root directory of the Yocto setup. For Torizon it would be /workdir/torizon/:

$ docker run --rm -it --name=crops -v ${HOME}/yocto-workdir:/workdir -v ${HOME}/yocto-downloads:/workdir/torizon/downloads -v ${HOME}/yocto-sstate-cache:/workdir/torizon/sstate-cache -v ${HOME}/yocto-deploy:/workdir/torizon/build-torizon/deploy torizon/crops-toradex --workdir=/workdir --cmd="MACHINE=<machine> TARGET=<target> startup-tdx-torizon.sh"

Modify

This section has some remarks about how to modify the Yocto setup.

Use Vim Inside the Build Container

The Torizon/CROPS container includes the Vim tiny editor, which you can use for quick tweaks without having to use external tools. Vim is known as hard for first steps. There are several tutorials, including an interactive tutorial and a game. For instance, to quickly edit the conf/local.conf:

$$ vim.tiny conf/local.conf

Use a Helper Container to Make Edits

In addition, you could as well bring up any helper container to edit files, such as busybox:

$ docker run --rm -it -v torizon-workdir:/workdir busybox sh

But then things start to become much easier if you use your host machine as described before in the basics section.

Automatically Bring Up the Samba Container

If you want to automatically bring up the Samba container running as a daemon, a few flags must be changed:

$ docker run -d -it --restart always --net host --name samba -v yocto-workdir:/workdir crops/samba

Build

This section presents some different ways to do things using the Torizon/CROPS container.

Only Configure But Don't Build

You can just bring up the Torizon/CROPS container without starting a build, but having everything configured for building a Torizon image. Just omit the TARGET variable:

$ docker run --rm -it --name=crops -v ${HOME}/yocto-workdir:/workdir torizon/crops-toradex --workdir=/workdir --cmd="MACHINE=<machine> startup-tdx-torizon.sh"

Only Bring Up But Don't Configure

You can just bring up the Torizon/CROPS container without having it configured for building a Torizon image:

$ docker run --rm -it --name=crops -v ${HOME}/yocto-workdir:/workdir torizon/crops-toradex --workdir=/workdir

You will then be in a shell inside the container where you can follow steps as described by any Yocto setup, for instance OpenEmbedded (core) or Build TorizonCore.

Multiple Build Directories

You can use the helper scripts from the Torizon/CROPS container to setup a build directory with a different name, but still sharing the setup (layers, downloads, build directory, etc). You just have to pass the build directory name after the helper script in --cmd. See the examples below, where the first is generic and the second uses build-torizon-colibri-imx6 as build directory:

$ docker run --rm -it --name=crops -v ${HOME}/yocto-workdir:/workdir torizon/crops-toradex --workdir=/workdir --cmd="MACHINE=<machine> startup-tdx-torizon.sh <build directory>"
 
$ docker run --rm -it --name=crops -v ${HOME}/yocto-workdir:/workdir torizon/crops-toradex --workdir=/workdir --cmd="MACHINE=<machine> startup-tdx-torizon.sh build-torizon-colibri-imx6"

Build the Poky-based Embedded Linux BSP

Throughout the article, the helper script startup-tdx-torizon.sh was invoked when starting a Torizon/CROPS container to setup and/or build a TorizonCore image. For convenience, a script named startup-tdx-poky.sh is provided as well for building the Poky-based Embedded Linux BSP. You can simply replace one with the other in the instructions. An example is provided for reference:

Note: other things might change, such as the directory structure of the build, which might require tweaks in the examples given throughout the article.

$ docker run --rm -it --name=crops -v ${HOME}/yocto-workdir:/workdir torizon/crops-toradex --workdir=/workdir --cmd="MACHINE=<machine> TARGET=<target> startup-tdx-poky.sh"
 
$ docker run --rm -it --name=crops -v ${HOME}/yocto-workdir:/workdir torizon/crops-toradex --workdir=/workdir --cmd="MACHINE=colibri-imx7 TARGET=console-tdx-image startup-tdx-poky.sh"

Build for a Specific Release Version

It is also possible to build any other version of the Embedded Linux BSP, for instance the release 2.8 which is sometimes referred to as ångström-based Embedded Linux BSP:

$ docker run --rm -it --name=crops -v ${HOME}/yocto-workdir:/workdir torizon/crops-toradex --workdir=/workdir --cmd="MACHINE=<machine> TARGET=<target> BRANCH=<repo manifest branch> startup-tdx-poky.sh"
 
$ docker run --rm -it --name=crops -v ${HOME}/yocto-workdir:/workdir torizon/crops-toradex --workdir=/workdir --cmd="MACHINE=colibri-imx7 TARGET=console-tdx-image BRANCH=LinuxImage2.8 startup-tdx-poky.sh"

Install

This section has comments on the install process described earlier.

HTTP Server Remarks

To install an image over the network all you need is a plain HTTP file server. It could be an Nginx, Apache, Node.js + Express or any other server or framework that serves files over HTTP. We have chosen the built-in Python simple HTTP server, which is a quick setup in a development environment.

The Torizon/CROPS container already setup a generic image_list.json file with all possible machines. It is located in /etc/image_list.json and copied to each build directory you create, in /workdir///image_list.json. You can modify it at your will.

You can find more information in the Toradex Easy Installer article, for instance in the section Unattended Flashing Over Ethernet.

Additional Resources

Cross Platform Enablement With Containers, or CROPS, was created by Randy Witt from Intel in c.a. October 2015 (according to initial commit on GitHub). For a better understanding of what CROPS is, check the talk Cross Platform Enablement for the Yocto Project with Containers from Randy Witt, Intel at Embedded Linux Conference 2017. The slides from the talk are available in the Yocto Project website.

The CROPS repository on GitHub has quite a few projects. For the scope of this article we identify a few relevant ones:

  • yocto-dockerfiles: the base for the other projects. It has dockerfiles for many different Linux distros and versions, and is not directly used by us, but rather indirectly from the CROPS project poky-container.
  • poky-container: this is the base container actually uses for building the Yocto images. Its base is the latest Ubuntu image supported by the Yocto Project, from yocto-dockerfiles.

Troubleshoot

On Windows host, the following have been observed:

  • Issue: sometimes it seems that the terminal becomes unresponsible during the build, while in fact things are still running appropriately.
    • Solution: hit Enter key a few times and have patience.
  • Issue: when building heavy packages such as Firefox, it was observed that the build failed due to lack of resources.
    • Solution: the issue was solved by re-building everything from scratch after re-configuring Docker with more CPU, RAM and swap resources.