Offline Factory TUF Keys

FoundriesFactory® uses The Update Framework (TUF) for multi-level key management strategy to secure software updates. Part of this strategy utilizes roles to separate software update responsibilities. By restricting each role’s responsibilities or actions it is trusted to perform, the impact of a compromised role’s key is minimized.

Even so, if a key was compromised, TUF provides a mechanism to reliably revoke that key. The root role exists to delegate trust to all other top-level roles used in the system. TUF allows rotating its keys, for example, when they are compromised or their validity term expired. To rotate the key, the user must crypto-sign the new key with the current root key, and authorize this operation using their FoundriesFactory credentials. This builtin two-factor authorization of TUF keys management operations makes it hard to perform known attacks by obtaining only one of the secrets.

FoundriesFactory provides the ability to manage TUF keys via dedicated Fioctl™ commands.

Note

We recommend running the command fioctl keys tuf updates review from time to time. It will show you a list of our recommendations about improving your TUF usage, which evolve over time. For example, it can show you the up to date information about the Recommended Offline TUF Keys Schema.

How to Rotate Offline TUF Root Key

The TUF root key is the most important key in TUF. The owner of this key can sign TUF root.json which defines what keys and roles your Factory devices can trust.

The initial TUF root key for your Factory is generated by Foundries.io™, and it is stored online on our servers. The Factory admin must take that key offline by rotating it as soon as possible. Thereafter, that (offline) TUF root key must be rotated regularly (at the least yearly) to keep your Factory secure.

Note

In the past, we used to send the TUF keys tarball via email to Factory owners at Factory creation. That email can be ignored. Factory owners who received it and have not yet downloaded/rotated the initial keys, can proceed with the initial rotation as described in this section.

To take your TUF root key offline by performing its first time rotation, use this Fioctl command:

fioctl keys tuf rotate-offline-key --role=root --first-time \
  --keys=/absolute/path/to/root.keys.tgz

This will perform the following steps:

  • download the initial (online) TUF root key;
  • generate your first (offline) TUF root key;
  • generate and sign the new version of root.json by both keys;
  • and finally upload it to Foundries.io servers.

The above command only needs to be run once. Onwards, use a shorter command to rotate your (offline) TUF root key:

fioctl keys tuf rotate-offline-key --role=root --keys=/absolute/path/to/root.keys.tgz

Note

Here and below the root.keys.tgz file is the one created during the first offline TUF root key rotation.

When rotating the TUF root key, the newly generated key is added to the keys tarball (root.keys.tgz in examples). That file must never be lost. Otherwise, it will be impossible to make any future updates to the Factory TUF keys. That will lead to the inability to deliver new Over-the-Air (OTA) updates to your Factory devices. Therefore, after each TUF root key rotation, we recommend that you Backup Offline TUF Keys as described below.

How to Rotate Offline TUF Targets Key

TUF has the notion of a targets.json file which specifies what updates (Targets) are available to Factory devices. That file is automatically maintained by your Factory CI builds. At the end of each build it is updated, signed by (online) TUF targets key, and pushed to Foundries.io servers.

Before going to production and creating the first Wave and Production Targets, a Factory owner must generate the offline TUF targets key.

FoundriesFactory ensures that the Factory admins have exclusive control of updates to production devices. To achieve this, it requires production targets.json files to be signed by two TUF targets keys:

  • The Foundries.io owned online target signing key, stored on the server side.
  • The Factory admin owned offline target signing key, stored on their storage of choice.

To generate your first offline TUF targets key, or rotate an existing key, use the following Fioctl command:

fioctl keys tuf rotate-offline-key --role=targets \
  --keys=/absolute/path/to/root.keys.tgz --targets-keys=/absolute/path/to/targets.keys.tgz

This will perform the following steps:

  • generate a new targets key and store it in a file specified by --targets-keys;
  • generate a new version of root.json and sign it by the TUF root key (specified by --keys argument);
  • re-sign existing production targets using the newly generated key (if present);
  • and finally upload both new root.json and new production targets signatures to Foundries.io servers.

Note

Technically, that command generates two almost identical root.json files: the CI and production. The only difference is that the production root.json sets the targets signing threshold to 2, while the CI flavor of root.json keeps it at 1. That nuance allows you to require only 1 signature for your Factory CI targets but 2 signatures for production. This is convenient to quickly test your CI builds on test devices, while keeping production devices more secured.

Note

The --targets-keys argument in the above command is optional; if it is missing, the command saves the offline targets key in the same file as the offline root key. We recommend keeping these keys in two separate files. That approach makes it possible to distribute the targets key among a wider set of Factory admins, and allow them to sign production targets without exposing the TUF root key to the wider audience.

After each TUF targets key rotation we recommend that you Backup Offline TUF Keys as described below. If you lose the offline TUF targets key, a new key can be generated if you have your Factory offline TUF root key. However, losing this key may be inconvenient if more than one Factory admin can manage production targets.

How to View Offline TUF Keys

The Factory’s TUF metadata can be viewed using this Fioctl command:

# The normal "CI" root:
fioctl keys tuf show-root

# The production root. Note the target key role has:
#   "threshold" : 2
fioctl keys tuf show-root --prod

It prints the full root.json file to your console, where you can examine individual fields.

Private parts of the offline TUF keys for your Factory are only stored inside the key tarballs.

The initial contents of the offline TUF root keys tarball (after the first rotation), will look like below:

tufrepo
`-- keys
    |-- first-root.pub     # The initial (online) TUF root public key
    |-- first-root.sec     # The initial (online) TUF root private key
    |-- fioctl-root-<keyid>.sec  # Your first (offline) TUF root private key
    `-- fioctl-root-<keyid>.pub  # Your first (offline) TUF root public key

The most critical file here is fioctl-root-<keyid>.sec, e.g. fioctl-root-5d7397a7a9d62d4f89a39b77903831af12172abb8b9f483e7ad9638bacbc93b1.sec. The <keyid> part can be verified with the current root.json using this command:

$ fioctl keys tuf show-root | jq '.signed.roles.root.keyids[0]'
"5d7397a7a9d62d4f89a39b77903831af12172abb8b9f483e7ad9638bacbc93b1"

The TUF targets keys tarball have a similar structure. For example, after the first (offline) TUF targets key rotation, it will look like below:

tufrepo
`-- keys
    |-- fioctl-targets-<keyid>.pub
    `-- fioctl-targets-<keyid>.sec

Similarly, the <keyid> part can be verified using this command:

$ fioctl keys tuf show-root | jq '.signed.roles.targets.keyids[1]'
"cb58f6b83e1e16276c64b19aef7fb07afe3227818f8511ac3ceb288965afdb65"

See the section How to Backup Offline TUF Keys below, how the internal structure of these tarballs can be used.

How to Backup Offline TUF Keys

There are 3 recommended ways for backing up your Factory TUF keys:

  • Copy the entire tarball, e.g. cp <tarball> <path to backup storage media>.
  • Extract and copy a plain text file of the Factory’s active root private key.
  • Print out the Factory’s active root private key.

We recommend placing 2–3 copies of these backups in safes in different geographical locations. The last two options require understanding the offline keys file format. See the section How to View Offline TUF Keys to understand the keys tarball internal structure.

Expert Mode

The Fioctl keys tuf updates command set allows you to implement various TUF key workflows based on your requirements:

  • Generate the TUF root versus the TUF targets keys on separate machines by different people.
  • Make granular changes to your Factory TUF keys, e.g. rotate select online keys on Foundries.io servers.
  • Add more than 1 offline TUF signing key for the TUF root or the production TUF targets role.
  • Set a signature threshold for the TUF root or the production TUF targets role.

Please, run the command fioctl keys tuf updates --help to view the examples.

One command of interest is fioctl keys tuf updates review. It analyses your Factory’s existing TUF root on the server, and prints a list of recommendations for improving it. Over time our engineers will add more items to that list as we develop new security features.

How to Add More Than 1 Offline TUF Keys

Usually, you need to add more than 1 offline TUF signing key for your TUF roles in one of these use cases:

  • Improving the TUF root key redundancy, so that a single key loss does not leed to the loss of control over your Factory updates.
  • Improving the production TUF targets authorship transparency, so that every release engineer has their own key, not shared with others.

In order to implement any workflow involving TUF roles with multiple offline signing keys, one should use the fioctl keys tuf updates subcommands. They require making transactional changes distributed across several machines, thus they cannot be executed using shortcut fioctl keys tuf subcommands.

In order to add a new offline signing key to your TUF root, you would start with the below command:

$ fioctl keys tuf updates init -m 'Your TUF root changes summary'
A new transaction to update TUF root keys started.
Your transaction ID is ELNOADKR .
Please, keep it secret and only share with participants of the transaction.
Only the user who initiated the transaction can make changes to it without the transaction ID.
Other users are required to supply this transaction ID for all commands except review and cancel.

This command initiates a new transaction to modify the TUF root, without committing it yet. An admin initiating the transaction should capture the transaction ID (ELNOADKR above). It will be used as a 2-factor authorization of further changes to the TUF root in the same transaction by other admins.

More often than not, a new key needs to be added for a person not yet owning any offline TUF signing key.

Note

From the security perspective, that person needs to generate and add their cryptographic key on their own. It is a bad habit if the admin generates a cryptographic key on behalf of another user, and then shares it with that user. Such action would mean that two users have access to the same cryptografic key, violating basic security principles.

So, an admin initiating the transaction, should share the transaction ID from above with the user who will add a new key. There are many ways to share it, either by in-person talk, or encrypted peer-to-peer communication mediums. It is safe if the transaction ID is leaked after the transaction finishes, as it is only temporal.

Having the transaction ID, a user who needs to add a new key would run the below command (e.g. for the TUF root role):

$ fioctl keys tuf updates add-offline-key --role=targets --keys path/to/tuf-targets-keys.tgz --txid ELNOADKR

This command generates a new offline signing key for the TUF targets, and adds it to the TUF root. These changes are not committed yet, they are only staged for commit within the scope of the transaction.

Note

It is not secure to keep several cryptographic keys for the same entity in one place. Thus, Fioctl denies adding a new offline key into a file which already contains such key for the same Factory. It still allows to keep previous (already inactive) keys in the same file as a backup.

Now that the user added their key, an admin who owns the offline TUF root signing key, needs to sign these changes. If it is the same admin who initiated the transaction, they can run the below command:

$ fioctl keys tuf updates sign --keys path/to/tuf-root-keys.tgz

If it is a different admin, they would also need to supply the transaction ID to that command.

Once all the desired changes have been done, an admin can apply them (commit the transaction) using the below command:

$ fioctl keys tuf updates apply

Note

Before applying the TUF root updates, it is a good habit to review them using fioctl keys tuf updates review.

At any moment before applying the changes, and admin can cancel the transaction by the below command:

$ fioctl keys tuf updates cancel

Any user with admin rights can cancel the TUF root updates transaction, not only the one who initiated it.

How to Increase the TUF Signature Threshold

Requiring more than 1 offline signature for any TUF root changes greatly improves the TUF root role security. In some use cases you might also require more than 1 offline signature for production TUF targets.

For that, you would start a new transaction (as above), and set the signature threshold using the below commands:

$ fioctl keys tuf updates init -m 'Your TUF root changes summary'
$ fioctl keys tuf updates set-threshold 2 --role=<role>

It is not allowed to set the signature threshold to a higher value than the number of keys for a given TUF role. Thus, normally, you would run the workflow How to Add More Than 1 Offline TUF Keys before setting the threshold. These two operations can also be combined into one TUF root updates transaction.

When you increase the signature threshold for the production TUF targets, you also need to sign existing production targets by additional offline signing key. This can be done within the same transaction using the below command:

$ fioctl keys tuf updates sign-prod-target --keys path/to/tuf-targets-keys.tgz

Once you are satisfied with the changes, you can apply them using fioctl keys tuf updates apply.