Products' release process

Projects referenced from the platform manifest file are likely to be Yocto Project’s layers. Layers are git repositories which contain metadata for Yocto Project. Among the metadata are the so called recipes. Recipes in the Yocto Project’s jargon are files with instructions about how to build software.

Recipes build software from their source code. Recipes contain a reference to the software source code location (an URI). If the URI points to a VCS repository, the recipe also specifies a reference to a revision in the repository (e.g., a git commit hash).

It’s not uncommon to have in your company’s layers recipes to describe how to build software developed by your own company. For example, your product is a kiosk system and one of the software shipped with it is user interface (KUI), which is developed by your company. KUI is likely to be in its own git repository and constantly being updated.

So, while in the development phase, you’ll probably want to use the tip of your projects' repositories (e.g., tip of master branch). On the other hand, for releases, you want revisions at a point the projects have been properly tested.

This approach can be tricky to handle manually. Updating recipes' reference to software’s VCS every time they change is not ideal, since it would demand a lot of manual work and would pollute the layers' history.

O.S. Systems provides some tools to automate that. They will be described in the next sections.

ossystems-tag-projects is a tool to tag the whole set of projects used by a product. It assumes the existence of a platform manifest repository which references all the projects required by a product. Note that this platform manifest file is not the same as the one to describe the product. This one is just a convenience format to group all the git repositories for the product, independently of Yocto Project-related repositories. It would put together, for example, all the software components showed in the The problem section’s diagram.

ossystems-tag-projects assumes the projects platform clone is updated. So, before running it, you probably want to explicitly run:

in the projects platform directory.

ossystems-tag-projects will check for each project, if they have changes after the latest tag. If they have been changed, ossystems-tag-projects will ask you if it should bump the tag to create a new version.

Here’s an example of ossystems-tag-projects running on one of O.S. Systems' internal projects:

In the example above, ossystems-tag-projects detected the projects liboslicense and liboshwcollect have changes after the latest tag, so it asks you whether you want to bump them. If you answer no, nothing will be changed.

If you answer yes, ossystems-tag-projects will tag and push tags for the projects it detected that changed. Next it will ask you whether you want to generate SRCREV files in the conf/srcrevs project directory:

If you answer yes, it’ll show you all the projects' tags and their commit hashes that will be written to the conf/srcrevs/srcrevs.inc file.

The srcrevs.inc file is used by a Yocto Project’s class (ossystems-srcrev-handler.bbclass) in the meta-ossystems-base layer to set the project revisions to the stable versions (at release time) or to the repository tip (at development time).

You can always use ossystems-tag-projects with the --dry-run argument to check what it would do without actually changing things.

Instead of using --all, you can also specify a list of individual projects separated by space on the command line. ossystems-tag-projects will only check the provided projects in this case.

You may have noticed in the examples that ossystems-tag-projects reads a configuration file if it finds one in conf/tools (tools.conf).

With this configuration file you can specify:

Here’s an example of configuration file:

Those options can also be specified on the command line (see the documentation for --version-to-start-from and --version-mask in the ossystems-tag-projects' help message):

ossystems-release is the tool that actually tags a new product version. It tags the product’s platform manifest file. This tool is supposed to be used after you have tagged all the projects that are part of the product (see the documentation for the ossystems-tag-projects tool).

ossystems-tools should be run from the Yocto Projects root directory (the directory where you source setup-environment from).

Here’s an example usage session:

Like with the projects platform manifest, it’s always recommended to update the product platform manifest copy.

Answering yes here will make ossystems-release fill up the platform manifest file with the revision of all projects referenced from it, and tag the platform manifest repository with the release tag. With this procedure, we establish a release state, to witch in the future we can return, in case it is necessary.

The release tag determined based on the current tag and the version mask. The version mask can be specified on the command line or in the conf/tools/tools.conf configuration file (just like for ossystems-tag-projects).

On the example output, you can see the Skipping hook before-create-tag message. ossystems-release may run some hook scripts at some points. Hook scripts are looked for into the directory pointed by the release-hooks-dir configuration option in the configuration file (the default value is <platform sources dir>/ossystems-release-scripts) or via the --release-hooks-dir command line option. Scripts should be named after the hook name.

ossystems-release calls each hook for each project, giving them the project tag, the project name and the project branch as arguments on the command line.

One of the hooks is the before-create-tag hook. That hook allows you to run an arbitrary script right before the new tag is created. Other hooks available are:

Errors in hook scripts (except after-push-tag) will cause ossystems-release to abort. At the time of this writing, ossystems-release has no means to roll back what has been previously done.

Next, ossystems-release will ask if you want to see the changes in the platform manifest:

It’s always recommended to look at the differences before actually checking the changes in. If you agree with the changes, you can answer yes to the next question, then ossystems-release will commit changes and ask if you want to run the after-create-tag hook.

The new tag must be checked in before the build starts because your product probably needs the new tag at build time (for example, to automatically get the tag from the repository at build time to build up an About screen for your project).

The after-create-tag hook is the script which actually starts the build process.

If the after-create-tag hook finishes successfully (i.e., the build process has been successfully finished), ossystems-release will ask you if you want to revert back to development mode.

Let’s describe this step with more details. When ossystems-release showed you the differences in the platform manifest file and asked you whether you wanted to commit the changes, it checked in a commit which set the revisions for each project it references. So, at that point, your platform manifest file was now referencing projects' git objects for the current version. Once your build is finished and your new version is released, you’ll want to set the project revisions in the platform manifest file to their original revisions (usually the branch name for each project).

Now that everything is done, the only step left is actually pushing the changes and tag to the remote repository.

ossystems-diff-releases shows the difference between two releases. The differences can be shown in the git log format or in the git diff format (you can use the --git-command command line option to specify which format you want — the default is git log format).

If you don’t specify projects, ossystems-diff-releases will show the differences for all projects.

Here’s the help message for ossystems-diff-release: