This document describes an strategy to manage platforms based on the
Yocto Project. Basically, we’ll describe how to use the
repo tool and manifest files to
manage multiple git projects as if they were a single one and manage
platform specific settings in an integrated way.
Platform manifest files
Platform manifest files are XML files that describe the git
repositories your projects use. They are used by the
repo tool to
manage the git repositories required by your projects. Roughly
speaking, platform manifest files are a way to turn multiple git
repositories into a single project that can be managed by a tool on
top of git.
Platform manifest files are usually stored under their own git repository, which can be cloned by repo (using git behind the scenes). Here’s an example (from http://code.ossystems.com.br/#/admin/projects/ossystems-yocto-platform):
<?xml version="1.0" encoding="UTF-8"?> <manifest> <remote fetch="http://code.ossystems.com.br" name="code" review="code.ossystems.com.br"/> <remote fetch="git://git.yoctoproject.org" name="yocto"/> <remote fetch="git://github.com/Freescale" name="fslc"/> <remote fetch="git://git.openembedded.org" name="oe"/> <default revision="invalidRevision" sync-j="4"/> <project remote="yocto" revision="master" name="poky" path="sources/poky"/> <project remote="oe" revision="master" name="meta-openembedded" path="sources/meta-openembedded"/> <project remote="code" revision="master" name="ossystems-yocto-base-scripts" path="sources/base"> <copyfile dest="setup-environment" src="setup-environment"/> </project> <!-- O.S. Systems Yocto BSP --> <project remote="code" revision="master" name="meta-ossystems-base" path="sources/meta-ossystems-base"/> <project remote="code" revision="master" name="meta-ossystems" path="sources/meta-ossystems"/> <project remote="code" revision="master" name="meta-fsl-arm" path="sources/meta-fsl-arm"/> <project remote="code" revision="master" name="meta-fsl-arm-extra" path="sources/meta-fsl-arm-extra"/> <project remote="code" revision="master" name="meta-fsl-demos" path="sources/meta-fsl-demos"/> <!-- conf --> <project remote="code" revision="master" name="ossystems-yocto-platform" path="sources/conf/repo"/> <project remote="code" revision="master" name="ossystems-yocto-config" path="sources/conf/tools"/> </manifest>
Let’s take a closer look at the things specified in that XML document.
<?xml version="1.0" encoding="UTF-8"?> <manifest> … </manifest>
block defines a manifest. In that block we’ll define projects and remote locations for projects' source code repositories.
<remote> tag specifies a location for repositories. Let’s see
<remote fetch="git://git.yoctoproject.org" name="yocto"/>
This line specifies a remote named yocto, whose URL is
git://git.yoctoproject.org. name will be used by the
to specify projects' complete URLs.
Note that remotes can have a review attribute, which can be used the integrate your projects to, for example, the Gerrit code review tool. In those cases, all your changes to the source code repositories under repo control will be submitted to the code review tool before hitting the actual source repository.
As an example of the
<project> tag, let’s take a closer look at the
<project remote="yocto" revision="master" name="poky" path="sources/poky"/>
That line specifies a project named poky which uses the yocto
remote. It means that repo will use that project to fetch a project
named poky from the URL specified for the yocto remote. That would
repo will fetch the
sources from that location using git and save them under the
sources/poky directory. Note that
<project> tags have a
revision attribute, which can be very useful for specifying the
exact commit you want your code to be at.
Also note an interesting trick we use to add the manifest repository itself to the list of projects:
<project remote="code" revision="master" name="ossystems-yocto-platform" path="sources/conf/repo"/>
With that line, the repository which holds the manifest file will also
be fetched and saved under
sources/conf/repo. So, if you ever need
to change the manifest file, you can do it from the copy you have
repo is the command line tool
that uses the platform manifest files to aggregate all projects in a
single tree. Let’s see how to use it using the
ossystems-yocto-platform project as a practical example.
Assuming you have
repo installed (if you don’t, see
run the following commands:
$ mkdir ossystems-yocto-platform $ cd ossystems-yocto-platform $ repo init -u http://code.ossystems.com.br/ossystems-yocto-platform
Those commands will initialize the repo metadata under the directory
you created (
ossystems-yocto-platform). The projects' source code
repositories are still not there. To fetch/update them, run
$ repo sync
That command tells repo to fetch the code repositories for all the
<project> entries in the platform manifest file. The projects'
source code repositories will be fetched to the sources directory.
Under the sources directory, you’ll find clones of all the projects
you specified in the manifest file. That means you can use git as you
would do if they where individual projects. repo provides some
interesting features to ease the management of multiple projects. For
example, if you need to search for some string in all projects, you
repo grep <pattern> instead of running
git grep <pattern>
on each git repository. The same for
repo diff, instead of manually
git diff over all the repositories.
To change repositories, you can use
git commit as you would do in a
regular git workflow. To submit changes, you can use
repo will submit the changes to the right locations, as
they are specified in the manifest file.
Managing platform versions
Since the platform manifest file lives in its own git repository, and projects specified in the manifest file have a revision attribute, we have all we need to version the whole platform. To do that, all we need is filling all projects' revision attribute with the hashes of the commits corresponding to the version we want to tag. After setting up the revisions for projects, we commit the manifest file and set a tag on the repository. By doing that, we have tagged the whole platform — it is now possible to check out the full state of a version at any time.
O.S. Systems provides an improved setup script which allows users to
configure hooks that can be run in a per layer basis. That script
can be found in the
setup-environment will look for Python scripts under
<layer>/conf/setup-environment.d, load and run them. Hooks are
Python functions that receive no argument. Hooks are run at two
stages: before running Poky’s
oe-init-build-env (which actually
conf/bblayers.conf) and after Poky’s
oe-init-build-env (so that you can overwrite things in
Here’s an example of hook script:
def __my_hook_before_init(): import os import sys try: os.environ['MACHINE'] except: sys.stderr.write("ERROR: You must set 'MACHINE' before setting up the environment.\n") sys.exit(1) def __my_hook_after_init(): import os append_var('INHERIT', 'buildhistory') set_var('PACKAGE_CLASSES', 'package_ipk') PLATFORM_ROOT_DIR = os.environ['PLATFORM_ROOT_DIR'] append_layers([ os.path.join(PLATFORM_ROOT_DIR, 'sources', p) for p in ['meta-browser', 'meta-chicken', 'meta-java'] ] ) run_before_init(__before_init_my_hook) run_after_init(__after_init_my_hook)
The first definition (
my_hook_before_init) specifies what is to be
run before Poky’s
oe-init-build-env. We register that hook with
run_before_init(before_init_my_hook). That hook simply checks if
MACHINE environment variable has been set in the environment.
The second definition (
__my_hook_after_init) specifies what is to be
run after Poky’s
oe-init-build-env. At this point, both
conf/bblayers.conf have been created, so we
can use the `setup-environment’s Python API to edit those files. At
the example you can see some functions of the `setup-environment’s
Python API. Here’s a more detailed documentation for them:
set_var(<var>, <val> [, op=<operator>])
<val>, using the
opcan be any of the assignment operators supported by BitBake.
BBLAYERSvariable in the
<layers>is a Python list of strings. Each string must be the path to a layer directory.
append_layerscan be useful when your layer depends on another layer(s). In that case you can use append_layers to automatically add the layers you layers depend on to
conf/bblayers.conf(note that specifying what layers have to be fetched must be done in the platform manifest file).
End-user-license agreement API
Hook scripts also have the ability to prompt users for EULA (End-user
license agreement) acceptance right at the moment the
setup-environment script is executed, so all EULAs can be accepted (or
not!) before the build process starts.
There is no standard for specifying EULA acceptance and EULA text location, so layers may use different variables to indicate EULAs acceptance/rejection and may place the text anywhere in the layer directory layout.
The interface for specifying EULAs acceptance provided by
O.S. Systems' setup-environment script is quite flexible in that
regard. Below you can see an example for the
eulas.accept['meta-fsl-arm/EULA'] = 'ACCEPT_FSL_EULA = "1"'
eulas.accept is a Python dictionary whose key is path to the EULA
file (including the layer directory), and its value is the EULA
acceptance expression to be added to
local.conf in case the user
accepts the EULA.