Usage

Remote Access

As described in Remote Resources and Places, one of labgrid’s main features is granting access to boards connected to other hosts transparent for the client. To get started with remote access, take a look at Setting Up the Distributed Infrastructure.

Place Scheduling

When sharing places between developers or with CI jobs, it soon becomes necessary to manage who can access which places. Developers often just need any place which has one of a group of identical devices, while CI jobs should wait until the necessary place is free instead of failing.

To support these use-cases, the coordinator has support for reserving places by using a tag filter and an optional priority. First, the places have to be tagged with the relevant key-value pairs:

$ labgrid-client -p board-1 set-tags board=imx6-foo
$ labgrid-client -p board-2 set-tags board=imx6-foo
$ labgrid-client -p board-3 set-tags board=imx8m-bar
$ labgrid-client -v places
Place 'board-1':
  tags: bar=baz, board=imx6-foo, jlu=2, rcz=1
  matches:
    rl-test/Testport1/NetworkSerialPort
…
Place 'board-2':
  tags: board=imx6-foo
  matches:
    rl-test/Testport2/NetworkSerialPort
…
Place 'board-3':
  tags: board=imx8m-bar
  matches:
    rl-test/Testport3/NetworkSerialPort
…

Now, if you want to access any imx6-foo board, you could find that all are already in use by someone else:

$ labgrid-client who
User     Host     Place    Changed
rcz      dude     board-1  2019-08-06 12:14:38.446201
jenkins  worker1  board-2  2019-08-06 12:52:44.762131

In this case, you can create a reservation. You can specify any custom tags as part of the filter, as well as name=<place-name> to select only a specific place (even if it has no custom tags).

$ labgrid-client reserve board=imx6-foo
Reservation 'SP37P5OQRU':
  owner: rettich/jlu
  token: SP37P5OQRU
  state: waiting
  filters:
    main: board=imx6-foo
  created: 2019-08-06 12:56:49.779982
  timeout: 2019-08-06 12:57:49.779983

As soon as any matching place becomes free, the reservation state will change from waiting to allocated. Then, you can use the reservation token prefixed by + to refer to the allocated place for locking and usage. While a place is allocated for a reservation, only the owner of the reservation can lock that place.

$ labgrid-client wait SP37P5OQRU
owner: rettich/jlu
token: SP37P5OQRU
state: waiting
filters:
  main: board=imx6-foo
created: 2019-08-06 12:56:49.779982
timeout: 2019-08-06 12:58:14.900621
…
owner: rettich/jlu
token: SP37P5OQRU
state: allocated
filters:
  main: board=imx6-foo
allocations:
  main: board-2
created: 2019-08-06 12:56:49.779982
timeout: 2019-08-06 12:58:46.145851
$ labgrid-client -p +SP37P5OQRU lock
acquired place board-2
$ labgrid-client reservations
Reservation 'SP37P5OQRU':
  owner: rettich/jlu
  token: SP37P5OQRU
  state: acquired
  filters:
    main: board=imx6-foo
  allocations:
    main: board-2
  created: 2019-08-06 12:56:49.779982
  timeout: 2019-08-06 12:59:11.840780
$ labgrid-client -p +SP37P5OQRU console

When using reservation in a CI job or to save some typing, the labgrid-client reserve command supports a --shell command to print code for evaluating in the shell. This sets the LG_TOKEN environment variable, which is then automatically used by wait and expanded via -p +.

$ eval `labgrid-client reserve --shell board=imx6-foo`
$ echo $LG_TOKEN
ZDMZJZNLBF
$ labgrid-client wait
owner: rettich/jlu
token: ZDMZJZNLBF
state: waiting
filters:
  main: board=imx6-foo
created: 2019-08-06 13:05:30.987072
timeout: 2019-08-06 13:06:44.629736
…
owner: rettich/jlu
token: ZDMZJZNLBF
state: allocated
filters:
  main: board=imx6-foo
allocations:
  main: board-1
created: 2019-08-06 13:05:30.987072
timeout: 2019-08-06 13:06:56.196684
$ labgrid-client -p + lock
acquired place board-1
$ labgrid-client -p + show
Place 'board-1':
  tags: bar=baz, board=imx6-foo, jlu=2, rcz=1
  matches:
    rettich/Testport1/NetworkSerialPort
  acquired: rettich/jlu
  acquired resources:
  created: 2019-07-29 16:11:52.006269
  changed: 2019-08-06 13:06:09.667682
  reservation: ZDMZJZNLBF

Finally, to avoid calling the wait command explicitly, you can add --wait to the reserve command, so it waits until the reservation is allocated before returning.

A reservation will time out after a short time, if it is neither refreshed nor used by locked places.

Library

labgrid can be used directly as a Python library, without the infrastructure provided by the pytest plugin.

The labgrid library provides two ways to configure targets with resources and drivers: either create the Target directly or use Environment to load a configuration file.

Note

On exit of your script/application, labgrid will call cleanup() on the targets using the python atexit module.

Targets

Note

In most cases it is easier to use a complete environment from a YAML file instead of manually creating and activating objects. Nevertheless, we explain this in the following to clarify the underlying concepts, and how to work with targets on a lower level, e.g. in strategies.

At the lower level, a Target can be created directly:

>>> from labgrid import Target
>>> t = Target('example')

Next, any required Resource objects can be created, which each represent a piece of hardware to be used with labgrid:

>>> from labgrid.resource import RawSerialPort
>>> rsp = RawSerialPort(t, name=None, port='/dev/ttyUSB0')

Note

Since we support multiple drivers of the same type, resources and drivers have a required name attribute. If you don’t use multiple drivers of the same type, you can set the name to None.

Further on, a Driver encapsulates logic how to work with resources. Drivers need to be created on the Target:

>>> from labgrid.driver import SerialDriver
>>> sd = SerialDriver(t, name=None)

As the SerialDriver declares a binding to a SerialPort, the target binds it to the resource object created above:

>>> sd.port
RawSerialPort(target=Target(name='example', env=None), name=None, state=<BindingState.bound: 1>, avail=True, port='/dev/ttyUSB0', speed=115200)
>>> sd.port is rsp
True

Driver Activation

Before a bound driver can be used, it needs to be activated. During activation, the driver makes sure that all hardware represented by the resources it is bound to can be used, and, if necessary, it acquires the underlying hardware on the OS level. For example, activating a SerialDriver makes sure that the hardware represented by its bound RawSerialPort object (e.g. something like /dev/ttyUSB0) is available, and that it can only be used labgrid and not by other applications while the SerialDriver is activated.

If we use a car analogy here, binding is the process of screwing the car parts together, and activation is igniting the engine.

After activation, we can use the driver to do our work:

>>> t.activate(sd)
>>> sd.write(b'test')
4

If an underlying hardware resource is not available (or not available after a certain timeout, depending on the driver), the activation step will raise an exception, e.g.:

>>> t.activate(sd)
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/serial/serialposix.py", line 288, in open
    self.fd = os.open(self.portstr, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
FileNotFoundError: [Errno 2] No such file or directory: '/dev/ttyUSB0'

Active drivers can be accessed by class (any Driver or Protocol) using some syntactic sugar:

>>> from labgrid import Target
>>> from labgrid.driver.fake import FakeConsoleDriver
>>> target = Target('main')
>>> console = FakeConsoleDriver(target, 'console')
>>> target.activate(console)
>>> target[FakeConsoleDriver]
FakeConsoleDriver(target=Target(name='main', env=None), name='console', state=<BindingState.active: 2>, txdelay=0.0)
>>> target[FakeConsoleDriver, 'console']
FakeConsoleDriver(target=Target(name='main', env=None), name='console', state=<BindingState.active: 2>, txdelay=0.0)

Driver Deactivation

Driver deactivation works in a similar manner:

>>> target.deactivate(console)
[FakeConsoleDriver(target=Target(name='main', env=None), name='console', state=<BindingState.bound: 1>, txdelay=0.0)]

Drivers need to be deactivated in the following cases:

Some drivers have internal logic depending on the state of the target. For example, the ShellDriver remembers whether it has already logged in to the shell. If the target reboots, e.g. through a hardware watchdog timeout, a power cycle, or by issuing a reboot command on the shell, the ShellDriver’s internal state becomes outdated, and the ShellDriver needs to be deactivated and re-activated.
One of the driver’s bound resources is required by another driver which is to be activated. For example, the ShellDriver and the BareboxDriver both require access to a SerialPort resource. If both drivers are bound to the same resource object, labgrid will automatically deactivate the BareboxDriver when activating the ShellDriver.

Target Cleanup

After you are done with the target, optionally call the cleanup method on your target. While labgrid registers an atexit handler to cleanup targets, this has the advantage that exceptions can be handled by your application:

>>> try:
...     target.cleanup()
... except Exception as e:
...     pass  # your code here

Environments

In practice, it is often useful to separate the Target configuration from the code which needs to control the board (such as a test case or installation script). For this use-case, labgrid can construct targets from a configuration file in YAML format:

targets:
  example:
    resources:
      RawSerialPort:
        port: '/dev/ttyUSB0'
    drivers:
      SerialDriver: {}

To parse this configuration file, use the Environment class:

>>> from labgrid import Environment
>>> env = Environment('example-env.yaml')

Using Environment.get_target, the configured Targets can be retrieved by name. Without an argument, get_target would default to ‘main’:

>>> t = env.get_target('example')

To access the target’s console, the correct driver object can be found by using Target.get_driver:

>>> cp = t.get_driver('ConsoleProtocol')
>>> cp
SerialDriver(target=Target(name='example', env=Environment(config_file='example-env.yaml')), name=None, state=<BindingState.active: 2>, txdelay=0.0, timeout=3.0)
>>> cp.write(b'test')
4

When using the get_driver method, the driver is automatically activated. The driver activation will also wait for unavailable resources when needed.

For more information on the environment configuration files and the usage of multiple drivers, see Environment Configuration.

pytest Plugin

labgrid includes a pytest plugin to simplify writing tests which involve embedded boards. The plugin is configured by providing an environment config file (via the –lg-env pytest option, or the LG_ENV environment variable) and automatically creates the targets described in the environment.

These pytest fixtures are provided:

env (session scope): Used to access the Environment object created from the configuration file. This is mostly used for defining custom fixtures at the test suite level.
target (session scope): Used to access the ‘main’ Target defined in the configuration file.
strategy (session scope): Used to access the Strategy configured in the ‘main’ Target.

Command-Line Options

The pytest plugin also supports the verbosity argument of pytest:

-vv: activates the step reporting feature, showing function parameters and/or results
-vvv: activates debug logging

This allows debugging during the writing of tests and inspection during test runs.

Other labgrid-related pytest plugin options are:

--lg-env=LG_ENV (was --env-config=ENV_CONFIG): Specify a labgrid environment config file. This is equivalent to labgrid-client’s -c/--config.
--lg-coordinator=CROSSBAR_URL: Specify labgrid coordinator websocket URL. Defaults to ws://127.0.0.1:20408/ws. This is equivalent to labgrid-client’s -x/--crossbar.
--lg-log=[path to logfiles]: Path to store console log file. If option is specified without path the current working directory is used.
--lg-colored-steps: Enables the ColoredStepReporter. Different events have different colors. The more colorful, the more important. In order to make less important output “blend into the background” different color schemes are available. See LG_COLOR_SCHEME.
--lg-initial-state=STATE_NAME: Sets the Strategy’s initial state. This is useful during development if the board is known to be in a defined state already. The Strategy used must implement the force() method. See the shipped ShellStrategy for an example.

pytest --help shows these options in a separate labgrid section.

Environment Variables

LG_ENV

Behaves like LG_ENV for labgrid-client.

LG_COLOR_SCHEME

Influences the color scheme used for the Colored Step Reporter. dark is meant for dark terminal background. light is optimized for light terminal background. dark-256color and light-256color are respective variants for terminals that support 256 colors. By default, dark or dark-256color (depending on the terminal) are used.

Takes effect only when used with --lg-colored-steps.

LG_PROXY

Specifies a SSH proxy host to be used for port forwards to access the coordinator. Network resources made available by the exporter will prefer their own proxy, and only fallback to LG_PROXY.

Simple Example

As a minimal example, we have a target connected via a USB serial converter (‘/dev/ttyUSB0’) and booted to the Linux shell. The following environment config file (shell-example.yaml) describes how to access this board:

targets:
  main:
    resources:
      RawSerialPort:
        port: '/dev/ttyUSB0'
    drivers:
      SerialDriver: {}
      ShellDriver:
        prompt: 'root@\w+:[^ ]+ '
        login_prompt: ' login: '
        username: 'root'

We then add the following test in a file called test_example.py:

def test_echo(target):
    command = target.get_driver('CommandProtocol')
    result = command.run_check('echo OK')
    assert 'OK' in result

To run this test, we simply execute pytest in the same directory with the environment config:

$ pytest --lg-env shell-example.yaml --verbose
============================= test session starts ==============================
platform linux -- Python 3.5.3, pytest-3.0.6, py-1.4.32, pluggy-0.4.0
…
collected 1 items

test_example.py::test_echo PASSED
=========================== 1 passed in 0.51 seconds ===========================

pytest has automatically found the test case and executed it on the target.

Custom Fixture Example

When writing many test cases which use the same driver, we can get rid of some common code by wrapping the CommandProtocol in a fixture. As pytest always executes the conftest.py file in the test suite directory, we can define additional fixtures there:

import pytest

@pytest.fixture(scope='session')
def command(target):
    return target.get_driver('CommandProtocol')

With this fixture, we can simplify the test_example.py file to:

def test_echo(command):
    result = command.run_check('echo OK')
    assert 'OK' in result

Strategy Fixture Example

When using a Strategy to transition the target between states, it is useful to define a function scope fixture per state in conftest.py:

import pytest

@pytest.fixture(scope='function')
def switch_off(strategy, capsys):
    with capsys.disabled():
        strategy.transition('off')

@pytest.fixture(scope='function')
def bootloader_command(target, strategy, capsys):
    with capsys.disabled():
        strategy.transition('barebox')
    return target.get_active_driver('CommandProtocol')

@pytest.fixture(scope='function')
def shell_command(target, strategy, capsys):
    with capsys.disabled():
        strategy.transition('shell')
    return target.get_active_driver('CommandProtocol')

Note

The capsys.disabled() context manager is only needed when using the ManualPowerDriver, as it will not be able to access the console otherwise. See the corresponding pytest documentation for details.

With the fixtures defined above, switching between bootloader and Linux shells is easy:

def test_barebox_initial(bootloader_command):
    stdout = bootloader_command.run_check('version')
    assert 'barebox' in '\n'.join(stdout)

def test_shell(shell_command):
    stdout = shell_command.run_check('cat /proc/version')
    assert 'Linux' in stdout[0]

def test_barebox_after_reboot(bootloader_command):
    bootloader_command.run_check('true')

Note

The bootloader_command and shell_command fixtures use Target.get_active_driver to get the currently active CommandProtocol driver (either BareboxDriver or ShellDriver). Activation and deactivation of drivers is handled by the BareboxStrategy in this example.

The Strategy needs additional drivers to control the target. Adapt the following environment config file (strategy-example.yaml) to your setup:

targets:
  main:
    resources:
      RawSerialPort:
        port: '/dev/ttyUSB0'
    drivers:
      ManualPowerDriver:
        name: 'example-board'
      SerialDriver: {}
      BareboxDriver:
        prompt: 'barebox@[^:]+:[^ ]+ '
      ShellDriver:
        prompt: 'root@\w+:[^ ]+ '
        login_prompt: ' login: '
        username: 'root'
      BareboxStrategy: {}

For this example, you should get a report similar to this:

$ pytest --lg-env strategy-example.yaml -v --capture=no
============================= test session starts ==============================
platform linux -- Python 3.5.3, pytest-3.0.6, py-1.4.32, pluggy-0.4.0
…
collected 3 items

test_strategy.py::test_barebox_initial
main: CYCLE the target example-board and press enter
PASSED
test_strategy.py::test_shell PASSED
test_strategy.py::test_barebox_after_reboot
main: CYCLE the target example-board and press enter
PASSED

========================== 3 passed in 29.77 seconds ===========================

Feature Flags

labgrid includes support for feature flags on a global and target scope. Adding a @pytest.mark.lg_feature decorator to a test ensures it is only executed if the desired feature is available:

import pytest

@pytest.mark.lg_feature("camera")
def test_camera(target):
   pass

Here’s an example environment configuration:

targets:
  main:
    features:
      - camera
    resources: {}
    drivers: {}

This would run the above test, however the following configuration would skip the test because of the missing feature:

targets:
  main:
    features:
      - console
    resources: {}
    drivers: {}

pytest will record the missing feature as the skip reason.

For tests with multiple required features, pass them as a list to pytest:

import pytest

@pytest.mark.lg_feature(["camera", "console"])
def test_camera(target):
   pass

Features do not have to be set per target, they can also be set via the global features key:

features:
  - camera
targets:
  main:
    features:
      - console
    resources: {}
    drivers: {}

This YAML configuration would combine both the global and the target features.

Test Reports

pytest-html

With the pytest-html plugin, the test results can be converted directly to a single-page HTML report:

$ pip install pytest-html
$ pytest --lg-env shell-example.yaml --html=report.html

JUnit XML

JUnit XML reports can be generated directly by pytest and are especially useful for use in CI systems such as Jenkins with the JUnit Plugin.

They can also be converted to other formats, such as HTML with junit2html tool:

$ pip install junit2html
$ pytest --lg-env shell-example.yaml --junit-xml=report.xml
$ junit2html report.xml

labgrid adds additional xml properties to a test run, these are:

ENV_CONFIG: Name of the configuration file
TARGETS: List of target names
TARGET_{NAME}_REMOTE: optional, if the target uses a RemotePlace resource, its name is recorded here
PATH_{NAME}: optional, labgrid records the name and path
PATH_{NAME}_GIT_COMMIT: optional, labgrid tries to record git sha1 values for every path
IMAGE_{NAME}: optional, labgrid records the name and path to the image
IMAGE_{NAME}_GIT_COMMIT: optional, labgrid tries to record git sha1 values for every image

Command-Line

labgrid contains some command line tools which are used for remote access to resources. See labgrid-client, labgrid-device-config and labgrid-exporter for more information.

Advanced CLI features

This section of the manual describes advanced features that are supported by the labgrid client CLI.

Usage

Remote Access

Place Scheduling

Library

Targets

Driver Activation

Driver Deactivation

Target Cleanup

Environments

pytest Plugin

Command-Line Options

Environment Variables

LG_ENV

LG_COLOR_SCHEME

LG_PROXY

Simple Example

Custom Fixture Example

Strategy Fixture Example

Feature Flags

Test Reports

pytest-html

JUnit XML

Command-Line

Advanced CLI features

Sharing a place with a co-worker