How to Use Secure Offline Updates with TorizonCore
This article will cover how to use the Secure Offline Update feature of our Torizon Platform Services. If you need conceptual context, please refer to our Torizon Updates Overview article on Secure Updates with Torizon.
Throughout the article, we will use terminology specific to Offline Updates. For this information and more on what Offline Updates are and the concepts surrounding them please refer to our First Steps article on the feature.
Otherwise, this article will be a purely how-to guide on the feature.
- TorizonCore 5.7.0 or newer
- Commercial account on our Torizon Platform Services website
- An appropriate update medium (USB or SD Card drive)
- An understanding of the basic concepts of Offline Updates
- Basic understanding of our TorizonCore Builder Tool
Preparing Your Device
Two configuration tweaks are required to enable offline updates:
- Provision device: add public keys for your update repository's root of trust
- Configure Aktualizr: add a configuration file with offline updates instructions
Provision Device for Offline Updates
Before your device can perform any Offline Update it needs to be able to trust the software repository that generates the update. You can provision this data in during production programming; please refer to our article on Production Programming & Provisioning to learn how to do this.
You can also provision the device manually if it has network connectivity. The easiest way to do so is to provision it into your repository; please refer to Provision a Single Device for detailed instructions.
If you don't want to provision the device for online platform access (or are unable to), you can also use TorizonCore Builder to get your shared provisioning data, then extract it into
/var/sota/import/ on your device (note that you'll need root access on the device):
$ torizoncore-builder platform provisioning-data --credentials <path/to/your/credentials.zip> --shared-data shared-data.tar.gz
Fetching 'root.json' from image repository.
Fetching 'root.json' from director repository.
Shared data archive 'shared-data.tar.gz' successfully generated.
$ scp shared-data.tar.gz firstname.lastname@example.org:
$ ssh torizon@<your-device-ip-address>
apalis-imx8~$ sudo tar zxvf shared-data.tar.gz --directory /var/sota/import
Configure Device for Offline Updates
aktualizr-torizon offline updates are disabled in favor of traditional online/remote updates.
To enable offline updates, add a
toml file in the Aktualizr configuration directory
/etc/sota/conf.d/ with the following content:
enable_offline_updates = true
enable_online_updates = false
offline_updates_source = "<path to lockbox mountpoint>"
The first line is a boolean switch to enable offline updates. The second is a boolean switch to disable online updates.
The last is the path to the directory where your Lockbox will be mounted when plugged in (e.g.
The path might depend on the partition label set on your update medium, which is unrelated to offline updates. For a quick try, use the command
df -h to find out the directory name for your lockbox. In the long term, consider adding to your lockbox creation checklist setting the partition name of the update medium.
Once the configuration file is created restart the client for it to take effect:
# sudo systemctl restart aktualizr
To confirm whether Offline Updates were successfully enabled or not you can check the logs of the update client, like so:
# journalctl -f -u aktualizr*
If successful you should see
Offline Updates are enabled in the logs.
Once you have confirmed that everything is successfully configured you can capture changes with TorizonCore Builder, so these changes are persistently kept.
In versions of TorizonCore earlier than 5.7.0, the Offline Updates feature is not implemented. If your project requires Offline Updates then it is necessary to upgrade to at least TorizonCore 5.7.0.
Once your TorizonCore Builder project has both the provisioning data and the custom configuration, create a custom TorizonCore image ready for offline updates.
From a practical perspective, the lockbox is the USB stick or SD Card you will use for updates. Learn more in the First Steps with Secure Offline Updates - Terminology section.
Before proceeding, make sure that your OS (TorizonCore) and/or application (container images) updates are already available in the UI. Learn how in the Torizon Platform Services Web Interface article.
just because the update metadata is available in the Platform Services, it doesn't mean that your application is uploaded to our servers. In fact, container images cannot even be uploaded to our servers, they are kept in a container registry - in a public cloud (Docker Hub, Amazon AWS, Microsoft Azure) or a local registry, for example.
Defining a Lockbox
The first step to an Offline Update is defining the contents of the Lockbox in the Platform Services web UI, so that the signed installation instructions can be created.
Define your Lockbox like so:
- Login to your commercial-tier account on https://app.torizon.io/.
- Go to the
Lockboxestab in the left sidebar.
- Click on
- In the first menu, select the component(s) you want to update.
- In the second menu, select the specific package you want to include in your Lockbox. You need to select a package name and version for each component you selected in the first menu.
- In the third menu, review your selection. When you’ve confirmed it’s correct hit
- In the fourth and final menu, give your lockbox a fitting name. Write down this name, you’ll need it in the next section!
Downloading Your Lockbox
TorizonCore Builder can download any previously defined Lockbox by its name using the
platform lockbox command. If you haven't done it yet, download your account credentials (
credentials.zip file) to your PC.
With your Lockbox name, download it with the TorizonCore Builder tool:
torizoncore-builder platform lockbox --credentials credentials.zip LOCKBOX_NAME
Once done, your Lockbox will be available in a directory on your host machine - by default it is the
update directory. For more details on the exact usage of this command please see our commands manual.
As a final step, move or copy it to your update medium of choice.
Remember that the directory must match the directory name you've configured in aktualizr. By default, external storage media will be mounted at
/media/<volume_label>/, so it's important to either make sure that the volume label matches the aktualizr config, or set up an automount config that mounts at a predictable location.
Performing the Offline Update
At this point, you must have:
- An update medium (SD Card or UB stick) with a lockbox.
- A device running TorizonCore version 5.7.0 or higher, provisioned and configured for offline updates.
To start an update, perform the following steps:
- Install the custom TorizonCore image with Toradex Easy Installer.
- Once the installation ends, power on (or reboot) the board.
- Attach your update medium containing your Lockbox to the device. Pay attention to the following:
- The update medium needs to be attached after the system has started (specifically after the update client has started). If the update medium is attached prior to this then it will not trigger the update.
- Make sure the file path of your update medium is consistent with what you configured for the update client.
The update will start within seconds of your update medium being inserted. Once the update begins, the process will be similar to what happens in an online OTA update.
If you wish to follow the process you can view the logs of the update client:
# journalctl -f -u aktualizr*
Updating and Revoking Lockboxes
The most important thing to understand about secure offline updates with Torizon is the process of defining your lockbox. When you define the software that should go in a lockbox, you are creating trusted, signed metadata that tells your devices "trust this software". If you find a bug in version 1.7.4 of your software, and don't want that version to be installed anymore, you should make sure to revoke any lockboxes that contain version 1.7.4.
You can do that in one of two ways. The first is to simply modify the contents of the lockbox. When you modify the contents of a lockbox, you automatically revoke the old one. So if you had previously created a lockbox named
Prod_series_1.x_EMEA, you could modify that lockbox, removing version 1.7.4 and replacing it with version 1.7.5, with the security fix. You can then download the lockbox again with TorizonCore Builder, and distribute the updated software. As soon as a device has been updated with the newer Lockbox, it will no longer accept the revoked one, protecting you against downgrade attacks.
The other way to prevent a bad version of software from being installed is to revoke the entire lockbox via the Torizon Platform Web Interface. You might want to do this, for example, when you need to indicate that an entire major revision of your software shouldn't be installed anymore. Suppose you have lockboxes for
Prod_series_2.x_AMER. Every time you cut a new release of your software, either on the 1.x or 2.x branches, you update the lockboxes, but you've decided that the time has come to sunset the 1.x series. You can revoke
Prod_series_1.x_AMER, so that the next time any device gets an update, it will know that it should no longer install any lockbox in that series.
Secure Offline Updates have a limited feature set as of their initial release. We are collecting customer feedback and plan to make improvements:
- TorizonCore cannot be configured to perform Offline & Online updates at the same time. To switch the mode of operation, you must re-configure
aktualizr-torizonand restart the client.
- Offline Updates don't have a friendly logging/feedack mechanism. In comparison, Online OTA updates give feedback to the server about the status of an update. In order to see the status of an offline update, you will need to manually observe or parse the
aktualizr-torizonlogs as the update happens.
- Device Monitoring will normally not work while in offline-update mode. It will continue to work, however, if you had previously registered it with the platform (in online-update mode), and then later changed the configuration to offline-update mode. (As long as the device has internet access, of course.)
- If a device is online-provisioned but the client is still configured for offline-updates, then the device will not appear as “online" in our Torizon Platform Services web UI.
- Offline Updates have only been tested using USB keys and SD Cards. While you can specify other file paths as the location of the media, there may be issue depending on the filesystem in question. Notably using NFS or other network filesystems may have unexpected behaviors with regard to timeouts and disconnects that are not properly handled by Aktualizr. Due to this, we do not support storing offline updates on anything other than locally attached SD Cards or USB keys.
When operating the update client in offline update mode there are certain side effects to be aware of. They can either be ignored or worked around, please review them carefully and make sure to take them into account for your specific use case:
- Offline Updates: Update doesn't proceed after temporary update lock expires
- Offline Updates: Short-lived containers get pruned from the system and can't be re-fetched
- Offline Updates: Strange "INTERNAL_ERROR" log during an application update
- There may also be logs from the update client indicating curl-related errors. These can be safely ignored when in offline-update mode.