Usage

Library

Labgrid can be used directly as a Python library, without the infrastructure provided by the pytest plugin.

Creating and Configuring Targets

The labgrid library provides two ways to configure targets with resources and drivers: either create the Target directly or use Environment to load a configuration file.

Targets

At the lower level, a Target can be created directly:

>>> from labgrid import Target
>>> t = Target('example')

Next, the required Resources can be created:

>>> from labgrid.resource import RawSerialPort
>>> rsp = RawSerialPort(t, name=None, port='/dev/ttyUSB0')

Note

Since we support multiple drivers of the same type, resources and drivers have a required name attribute. If you don’t require support for this functionality set the name to None.

Then, a Driver needs to be created on the Target:

>>> from labgrid.driver import SerialDriver
>>> sd = SerialDriver(t, name=None)

As the SerialDriver declares a binding to a SerialPort, the target binds it to the resource created above:

>>> sd.port
RawSerialPort(target=Target(name='example', env=None), name=None, state=<BindingState.bound: 1>, avail=True, port='/dev/ttyUSB0', speed=115200)
>>> sd.port is rsp
True

Before the driver can be used, it needs to be activated:

>>> t.activate(sd)
>>> sd.write(b'test')

Active drivers can be accessed by class (any Driver or Protocol) using some syntactic sugar:

>>> target = Target('main')
>>> console = FakeConsoleDriver(target, 'console')
>>> target.activate(console)
>>> target[FakeConsoleDriver]
FakeConsoleDriver(target=Target(name='main', …), name='console', …)
>>> target[FakeConsoleDriver, 'console']
FakeConsoleDriver(target=Target(name='main', …), name='console', …)

Environments

In practice, is is often useful to separate the Target configuration from the code which needs to control the board (such as a test case or installation script). For this use-case, labgrid can construct targets from a configuration file in YAML format:

targets:
  example:
    resources:
      RawSerialPort:
        port: '/dev/ttyUSB0'
    drivers:
      SerialDriver: {}

To parse this configuration file, use the Environment class:

>>> from labgrid import Environment
>>> env = Environment('example-env.yaml')

Using Environment.get_target, the configured Targets can be retrieved by name. Without an argument, get_target would default to ‘main’:

>>> t = env.get_target('example')

To access the target’s console, the correct driver object can be found by using Target.get_driver:

>>> from labgrid.protocol import ConsoleProtocol
>>> cp = t.get_driver(ConsoleProtocol)
>>> cp
SerialDriver(target=Target(name='example', env=Environment(config_file='example.yaml')), name=None, state=<BindingState.active: 2>, txdelay=0.0)
>>> cp.write(b'test')

When using the get_driver method, the driver is automatically activated. The driver activation will also wait for unavailable resources when needed.

For more information on the environment configuration files and the usage of multiple drivers, see Environment Configuration.

pytest Plugin

Labgrid includes a pytest plugin to simplify writing tests which involve embedded boards. The plugin is configured by providing an environment config file (via the –lg-env pytest option, or the LG_ENV environment variable) and automatically creates the targets described in the environment.

Two pytest fixtures are provided:

env (session scope)
Used to access the Environment object created from the configuration file. This is mostly used for defining custom fixtures at the test suite level.
target (session scope)
Used to access the ‘main’ Target defined in the configuration file.

Command-Line Options

The pytest plugin also supports the verbosity argument of pytest:

  • -vv: activates the step reporting feature, showing function parameters and/or results
  • -vvv: activates debug logging

This allows debugging during the writing of tests and inspection during test runs.

Other labgrid-related pytest plugin options are:

--lg-env=LG_ENV (was --env-config=ENV_CONFIG)
Specify a labgrid environment config file. This is equivalent to labgrid-client’s -c/--config.
--lg-coordinator=CROSSBAR_URL
Specify labgrid coordinator websocket URL. Defaults to ws://127.0.0.1:20408/ws. This is equivalent to labgrid-client’s -x/--crossbar.
--lg-log=[path to logfiles]
Path to store console log file. If option is specified without path the current working directory is used.
--lg-colored-steps
Enables the ColoredStepReporter. Different events have different colors. The more colorful, the more important. In order to make less important output “blend into the background” different color schemes are available. See LG_COLOR_SCHEME.

pytest --help shows these options in a separate labgrid section.

Environment Variables

LG_ENV

Behaves like LG_ENV for labgrid-client.

LG_COLOR_SCHEME

Influences the color scheme used for the Colored Step Reporter. dark (default) is meant for dark terminal background. light is optimized for light terminal background. Takes effect only when used with --lg-colored-steps.

Simple Example

As a minimal example, we have a target connected via a USB serial converter (‘/dev/ttyUSB0’) and booted to the Linux shell. The following environment config file (shell-example.yaml) describes how to access this board:

targets:
  main:
    resources:
      RawSerialPort:
        port: '/dev/ttyUSB0'
    drivers:
      SerialDriver: {}
      ShellDriver:
        prompt: 'root@\w+:[^ ]+ '
        login_prompt: ' login: '
        username: 'root'

We then add the following test in a file called test_example.py:

from labgrid.protocol import CommandProtocol

def test_echo(target):
    command = target.get_driver(CommandProtocol)
    result = command.run_check('echo OK')
    assert 'OK' in result

To run this test, we simply execute pytest in the same directory with the environment config:

$ pytest --lg-env shell-example.yaml --verbose
============================= test session starts ==============================
platform linux -- Python 3.5.3, pytest-3.0.6, py-1.4.32, pluggy-0.4.0
…
collected 1 items

test_example.py::test_echo PASSED
=========================== 1 passed in 0.51 seconds ===========================

pytest has automatically found the test case and executed it on the target.

Custom Fixture Example

When writing many test cases which use the same driver, we can get rid of some common code by wrapping the CommandProtocol in a fixture. As pytest always executes the conftest.py file in the test suite directory, we can define additional fixtures there:

import pytest

from labgrid.protocol import CommandProtocol

@pytest.fixture(scope='session')
def command(target):
    return target.get_driver(CommandProtocol)

With this fixture, we can simplify the test_example.py file to:

def test_echo(command):
    result = command.run_check('echo OK')
    assert 'OK' in result

Strategy Fixture Example

When using a Strategy to transition the target between states, it is useful to define a function scope fixture per state in conftest.py:

import pytest

from labgrid.protocol import CommandProtocol
from labgrid.strategy import BareboxStrategy

@pytest.fixture(scope='session')
def strategy(target):
    try:
        return target.get_driver(BareboxStrategy)
    except NoDriverFoundError:
        pytest.skip("strategy not found")

@pytest.fixture(scope='function')
def switch_off(target, strategy, capsys):
    with capsys.disabled():
        strategy.transition('off')

@pytest.fixture(scope='function')
def bootloader_command(target, strategy, capsys):
    with capsys.disabled():
        strategy.transition('barebox')
    return target.get_active_driver(CommandProtocol)

@pytest.fixture(scope='function')
def shell_command(target, strategy, capsys):
    with capsys.disabled():
        strategy.transition('shell')
    return target.get_active_driver(CommandProtocol)

Note

The capsys.disabled() context manager is only needed when using the ManualPowerDriver, as it will not be able to access the console otherwise. See the corresponding pytest documentation for details.

With the fixtures defined above, switching between bootloader and Linux shells is easy:

def test_barebox_initial(bootloader_command):
    stdout = bootloader_command.run_check('version')
    assert 'barebox' in '\n'.join(stdout)

def test_shell(shell_command):
    stdout = shell_command.run_check('cat /proc/version')
    assert 'Linux' in stdout[0]

def test_barebox_after_reboot(bootloader_command):
    bootloader_command.run_check('true')

Note

The bootloader_command and shell_command fixtures use Target.get_active_driver to get the currently active CommandProtocol driver (either BareboxDriver or ShellDriver). Activation and deactivation of drivers is handled by the BareboxStrategy in this example.

The Strategy needs additional drivers to control the target. Adapt the following environment config file (strategy-example.yaml) to your setup:

targets:
  main:
    resources:
      RawSerialPort:
        port: '/dev/ttyUSB0'
    drivers:
      ManualPowerDriver:
        name: 'example-board'
      SerialDriver: {}
      BareboxDriver:
        prompt: 'barebox@[^:]+:[^ ]+ '
      ShellDriver:
        prompt: 'root@\w+:[^ ]+ '
        login_prompt: ' login: '
        username: 'root'
      BareboxStrategy: {}

For this example, you should get a report similar to this:

$ pytest --lg-env strategy-example.yaml -v
============================= test session starts ==============================
platform linux -- Python 3.5.3, pytest-3.0.6, py-1.4.32, pluggy-0.4.0
…
collected 3 items

test_strategy.py::test_barebox_initial
main: CYCLE the target example-board and press enter
PASSED
test_strategy.py::test_shell PASSED
test_strategy.py::test_barebox_after_reboot
main: CYCLE the target example-board and press enter
PASSED

========================== 3 passed in 29.77 seconds ===========================

Feature Flags

Labgrid includes support for feature flags on a global and target scope. They will be concatenated and compared to a pytest mark on the test to decide whether the test can run with the available features.:

import pytest

@pytest.mark.lg_feature("camera")
def test_camera(target):
   [...]

together with an example environment configuration:

targets:
  main:
    features:
      - camera
    resources: {}
    drivers: {}

would run the above test, however the following configuration would skip the test because of the missing feature:

targets:
  main:
    features:
      - console
    resources: {}
    drivers: {}

This is also reported in the pytest execution as a skipped test with the reason being the missing feature.

For tests with multiple required features, pass them as a list to pytest::

import pytest

@pytest.mark.lg_feature(["camera", "console"])
def test_camera(target):
   [...]

Features do not have to be set per target, they can also be set via the global features key:

features:
  - camera
targets:
  main:
    features:
      - console
    resources: {}
    drivers: {}

This yaml would combine both the global and the target features.

Test Reports

pytest-html

With the pytest-html plugin, the test results can be converted directly to a single-page HTML report:

$ pip install pytest-html
$ pytest --lg-env shell-example.yaml --html=report.html

JUnit XML

JUnit XML reports can be generated directly by pytest and are especially useful for use in CI systems such as Jenkins with the JUnit Plugin.

They can also be converted to other formats, such as HTML with junit2html tool:

$ pip install junit2html
$ pytest --lg-env shell-example.yaml --junit-xml=report.xml
$ junit2html report.xml

Labgrid adds additional xml properties to a test run, these are:

  • ENV_CONFIG: Name of the configuration file
  • TARGETS: List of target names
  • TARGET_{NAME}_REMOTE: optional, if the target uses a RemotePlace resource, its name is recorded here
  • PATH_{NAME}: optional, labgrid records the name and path
  • PATH_{NAME}_GIT_COMMIT: optional, labgrid tries to record git sha1 values for every path
  • IMAGE_{NAME}: optional, labgrid records the name and path to the image
  • IMAGE_{NAME}_GIT_COMMIT: optional, labgrid tries to record git sha1 values for every image

Command-Line

Labgrid contains some command line tools which are used for remote access to resources. See labgrid-client, labgrid-device-config and labgrid-exporter for more information.

USB stick emulation

Labgrid makes it posible to use a target as an emulated USB stick, allowing upload, modification, plug and unplug events. To use a target as an emulated USB stick, several requirements have to be met:

  • OTG support on one of the device USB ports
  • losetup from util-linux
  • mount from util-linux
  • A kernel build with CONFIG_USB_GADGETFS=m
  • A network connection to the target to use the SSHDriver for file uploads

To use USB stick emulation, import USBStick from labgrid.external and bind it to the desired target:

from labgrid.external import USBStick

stick = USBStick(target, '/home/')

The above code block creates the stick and uses the /home directory to store the device images. USBStick images can now be uploaded using the upload_image method. Once an image is selected, files can be uploaded and retrived using the put_file and get_file methods. The plug_in and plug_out functions plug the emulated USB stick in and out.

hawkBit management API

Labgrid provides an interface to the hawkbit management API. This allows a labgrid test to create targets, rollouts and manage deployments.

from labgrid.external import HawkbitTestClient

client = HawkbitTestClient('local', '8080', 'admin', 'admin')

The above code connects to a running hawkbit instance on the local computer and uses the default credentials to log in. The HawkbitTestClient provides various helper functions to add targets, define distribution sets and assign targets.