Test a model with pytest

New in v0.11.0

syside.testing lets you write the rules a model must follow as pytest tests: every part definition is documented, no two parts share a name, the decomposition has no cycle. When a rule breaks, the failure message names the elements that broke it rather than reporting assert False. Use it to run modelling rules in CI alongside the code that depends on the model.

You need pytest installed in the same environment as Syside Automator; syside.testing imports it only when a fixture is defined, so scripts that use the assertion helpers alone do not need it.

Write the rules

The model below leaves one part definition undocumented on purpose:

package Vehicle {
  part def Engine {
    doc /* Converts fuel into rotation of the drive shaft. */
    part starter : Starter;
  }
  part def Starter {
    doc /* Electric motor that turns the engine over. */
  }
  part def Wheel {
    doc /* Load-bearing wheel with a pneumatic tyre. */
  }
  part def Seat;

  part car {
    part engine : Engine;
    part frontLeft : Wheel;
    part frontRight : Wheel;
    part driverSeat : Seat;
  }
}

The test file loads the model once for the whole session with model_fixture and states each rule with an assertion helper. The helpers take the elements to check, from syside.query or any other iterable, and a predicate:

"""The same rules as example_script.py, written as a pytest suite.

Run with `pytest test_model.py` from this directory. `model_fixture` loads
the model once for the whole session, and a rule that fails names the
offending elements in the assertion message.
"""

import pathlib

import syside
from syside import query, testing

EXAMPLE_DIR = pathlib.Path(__file__).parent

model = testing.model_fixture(EXAMPLE_DIR / "example_model.sysml")


def test_every_part_definition_is_documented(model: syside.Model) -> None:
    testing.assert_all(
        query.find_elements_of_type(model, syside.PartDefinition),
        lambda definition: query.description(definition) is not None,
        label="documented part definitions",
    )


def test_part_names_are_unique(model: syside.Model) -> None:
    testing.assert_foreach_unique(
        query.find_elements_of_type(model, syside.PartUsage),
        lambda part: part.name,
        label="part names",
    )


def is_user_defined(element: syside.Element) -> bool:
    """True for elements written in the model, false for library elements."""
    return element.document.document_tier is syside.DocumentTier.Project


def part_definitions_used_by(
    definition: syside.PartDefinition,
) -> list[syside.PartDefinition]:
    """The definitions of the parts that `definition` directly contains.

    Only parts written in the model count; every part definition inherits
    library parts from `Parts::Part`, which contains parts of its own type.
    """
    return [
        part_type
        for part in query.features_of(definition)
        if isinstance(part, syside.PartUsage) and is_user_defined(part)
        for part_type in query.types_of(part)
        if isinstance(part_type, syside.PartDefinition)
    ]


def test_no_part_definition_contains_itself(model: syside.Model) -> None:
    testing.assert_acyclic(
        query.find_elements_of_type(model, syside.PartDefinition),
        part_definitions_used_by,
    )

What lambda means here

Each helper needs a small function to apply to every element, and the tests write those functions inline with lambda. A lambda is an unnamed function written on one line: lambda part: part.name takes one argument, part, and returns part.name. It is the same as writing

def name_of(part: syside.PartUsage) -> str | None:
    return part.name

and passing name_of instead. Use a def whenever the rule needs more than one expression, or when a name would make the test easier to read; the helpers accept either. The Python tutorial’s section on lambda expressions covers the form in full.

Run pytest test_model.py in the directory holding both files. Two tests pass and one fails with the message:

AssertionError: assert_all documented part definitions: 1 of 3 items failed:
  - PartDefinition Vehicle::Seat

The third rule guards against a model that is legal SysML v2 and still describes something that cannot be built. Every part in this model is written without a multiplicity, which means exactly one: an Engine has one Starter. If someone later gave Starter a part typed Engine, every engine would need a starter that needs an engine, and no finite vehicle satisfies the model. Loading it reports nothing, because the specification allows a definition to contain its own type, and the pattern is legitimate when the multiplicity can be zero, as in a tree of subassemblies [0..*]. The damage appears downstream, in any script that expands the decomposition: a bill of materials generator or a mass roll-up follows Engine to Starter to Engine and never finishes.

A chain of parts leading back to where it started is a cycle in a graph, and that is what assert_acyclic looks for. It takes a set of starting points and a successors function saying which items each one leads to, then follows the links looking for a way back. In this test the starting points are the part definitions, and part_definitions_used_by leads from a definition to the definitions of the parts written inside it. Library parts are left out, since every part definition inherits from Parts::Part, which contains parts of its own type by design. The check passes here; had it failed, the message would show the cycle as a path:

AssertionError: assert_acyclic graph: cycle found:
  PartDefinition Vehicle::Engine → PartDefinition Vehicle::Starter → PartDefinition Vehicle::Engine

The label argument is what appears after the helper’s name; without it the message shows the predicate’s name. Elements are described by their class and qualified name through describe, and a describe= argument replaces that rendering.

The assertion helpers

Helper	Fails when
`assert_all`	any item fails the predicate; passes on no items
`assert_any`	no item passes the predicate, including when there are no items
`assert_none`	any item passes the predicate
`assert_implies`	an item satisfies `when` but not `then`
`assert_foreach_unique`	two items map to the same `target` value
`assert_acyclic`	following `successors` from the nodes finds a cycle; the message shows one
`assert_reachable`	a source cannot reach any target along `successors`

Each helper reads its input once and calls describe only for the elements that appear in the message, capped by limit (10 by default), so a rule over a large model costs no more than the loop it replaces.

Outside pytest

The helpers are plain functions raising AssertionError, so a script can call them without pytest and report the message itself. This script applies the same three rules to the same model:

import pathlib
from collections.abc import Callable

import syside
from syside import query, testing

EXAMPLE_DIR = pathlib.Path(__file__).parent
MODEL_FILE_PATH = EXAMPLE_DIR / "example_model.sysml"


def is_user_defined(element: syside.Element) -> bool:
    """True for elements written in the model, false for library elements."""
    return element.document.document_tier is syside.DocumentTier.Project


def part_definitions_used_by(
    definition: syside.PartDefinition,
) -> list[syside.PartDefinition]:
    """The definitions of the parts that `definition` directly contains.

    Only parts written in the model count; every part definition inherits
    library parts from `Parts::Part`, which contains parts of its own type.
    """
    return [
        part_type
        for part in query.features_of(definition)
        if isinstance(part, syside.PartUsage) and is_user_defined(part)
        for part_type in query.types_of(part)
        if isinstance(part_type, syside.PartDefinition)
    ]


def rules(model: syside.Model) -> dict[str, Callable[[], None]]:
    """The modelling rules, each as a check that raises on violation."""
    definitions = query.find_elements_of_type(model, syside.PartDefinition)
    parts = query.find_elements_of_type(model, syside.PartUsage)
    return {
        "every part definition is documented": lambda: testing.assert_all(
            definitions,
            lambda definition: query.description(definition) is not None,
            label="documented part definitions",
        ),
        "part names are unique": lambda: testing.assert_foreach_unique(
            parts, lambda part: part.name, label="part names"
        ),
        "no part definition contains itself": lambda: testing.assert_acyclic(
            definitions, part_definitions_used_by
        ),
    }


def main() -> None:
    (model, diagnostics) = syside.load_model([MODEL_FILE_PATH])
    assert not diagnostics.contains_errors(warnings_as_errors=True)

    # In a test suite each rule is a test function and pytest reports the
    # AssertionError; here the message is printed instead.
    for name, check in rules(model).items():
        try:
            check()
        except AssertionError as failure:
            print(f"FAIL  {name}\n{failure}")
        else:
            print(f"PASS  {name}")


if __name__ == "__main__":
    main()

It prints:

FAIL  every part definition is documented
assert_all documented part definitions: 1 of 4 items failed:
  - PartDefinition Vehicle::Seat
PASS  part names are unique
PASS  no part definition contains itself

Test a model with pytest

Write the rules

The assertion helpers

Outside pytest

Feedback

Legal

Company