Test a model with pytest
New in v0.11.0
testing lets you write the rules a model must follow as tests:
every part definition is documented, no two parts share a name, no part
definition contains itself. When a rule breaks, the failure message names the
elements that broke it rather than reporting assert False. Use it to run
modeling rules in CI alongside the code that depends on the model.
The page shows the rules as a pytest suite, which is how they are
usually run, but pytest is optional: the assertion helpers are plain functions
that raise AssertionError, and Without pytest shows the
same rules in an ordinary script. Only model_fixture
needs pytest installed.
The model and the tests
The model leaves one part definition undocumented on purpose. Save it as
example_model.sysml:
package Vehicle {
part def Engine {
doc /* Converts fuel into rotation of the drive shaft. */
part starter : Starter;
}
part def Starter {
doc /* Electric motor that turns the engine over. */
}
part def Wheel {
doc /* Load-bearing wheel with a pneumatic tire. */
}
part def Seat;
part def Car {
doc /* Four-wheeled passenger vehicle. */
part engine : Engine;
part frontLeft : Wheel;
part frontRight : Wheel;
part driverSeat : Seat;
}
}
The test file loads the model once for the whole session with
model_fixture and states each rule with
an assertion helper. A helper takes the elements to check, here from
syside.query, and a function to apply to each. Save
it beside the model as test_model.py:
"""The same rules as example_script.py, written as a pytest suite.
Run with `pytest test_model.py` from this directory. `model_fixture` loads
the model once for the whole session, and a rule that fails names the
offending elements in the assertion message.
"""
import pathlib
import syside
from syside import query, testing
EXAMPLE_DIR = pathlib.Path(__file__).parent
model = testing.model_fixture(EXAMPLE_DIR / "example_model.sysml")
def is_documented(definition: syside.PartDefinition) -> bool:
return query.description(definition) is not None
def test_every_part_definition_is_documented(model: syside.Model) -> None:
testing.assert_all(
query.find_elements_of_type(model, syside.PartDefinition),
is_documented,
label="documented part definitions",
)
def test_part_names_are_unique(model: syside.Model) -> None:
testing.assert_foreach_unique(
query.find_elements_of_type(model, syside.PartUsage),
lambda part: part.name,
label="part names",
)
def is_user_defined(element: syside.Element) -> bool:
"""True for elements written in the model, false for library elements."""
return element.document.document_tier is syside.DocumentTier.Project
def part_definitions_used_by(
definition: syside.PartDefinition,
) -> list[syside.PartDefinition]:
"""The definitions of the parts that `definition` directly contains.
Only parts written in the model count; every part definition inherits
library parts from `Parts::Part`, which contains parts of its own type.
"""
return [
part_type
for part in query.features_of(definition)
if isinstance(part, syside.PartUsage) and is_user_defined(part)
for part_type in query.declared_types_of(part)
if isinstance(part_type, syside.PartDefinition)
]
def test_no_part_definition_contains_itself(model: syside.Model) -> None:
testing.assert_acyclic(
query.find_elements_of_type(model, syside.PartDefinition),
part_definitions_used_by,
)
Run pytest test_model.py in the directory holding both files. The sections
below take the three rules in turn.
Every part definition is documented
is_documented returns whether description finds a doc
comment on the definition, and
assert_all requires it of every
definition. Seat has none, so this test fails with:
AssertionError: assert_all documented part definitions: 1 of 5 items failed:
- PartDefinition Vehicle::Seat
Read the message from the left: the helper that failed, the label you gave
it, how many of the checked items failed, and then the failing items, each as
its class and qualified name. Without a label, the message shows the name of
the function you passed instead. The list is capped at limit items, 10 by
default, so a rule that fails everywhere stays readable. To render elements
differently, pass your own describe= function; the default is
describe.
Part names are unique
assert_foreach_unique applies a
function to every element and fails if two elements produce the same value. Here
the function is lambda part: part.name, and it passes, since no two parts in
the model share a name.
What lambda means here
A lambda is an unnamed function written on one line: lambda part: part.name
takes one argument, part, and returns part.name. It is the same as writing
def name_of(part: syside.PartUsage) -> str | None:
return part.name
and passing name_of, which is how the first test passes is_documented. Use
a def whenever the rule needs more than one expression, or when a name would
make the test easier to read; the helpers accept either. The Python
tutorial’s section on lambda expressions covers the form in
full.
No part definition contains itself
This rule guards against a model that is legal SysML v2 and still describes
something that cannot be built. Every part in this model is written without a
multiplicity, which means exactly one: an Engine has one Starter. If someone
later gave Starter a part typed Engine, every engine would need a starter
that needs an engine, and no finite car satisfies the model. Loading it reports
nothing, because the specification allows a definition to contain its own type,
and the pattern is legitimate when the multiplicity can be zero, as in a tree
of subassemblies [0..*]. The damage appears in any script that expands the
decomposition: a bill of materials generator or a mass roll-up follows Engine
to Starter to Engine and never finishes.
A chain of parts leading back to where it started is a cycle in a graph, and
that is what assert_acyclic looks
for. It takes a set of starting points and a successors function saying which
items each one leads to, then follows the links looking for a way back. Here the
starting points are the part definitions, and part_definitions_used_by leads
from a definition to the definitions of the parts written inside it. Library
parts are left out, since every part definition inherits from Parts::Part,
which contains parts of its own type by design. The rule passes for this model;
had it failed, the message would show the cycle as a path:
AssertionError: assert_acyclic graph: cycle found:
PartDefinition Vehicle::Engine → PartDefinition Vehicle::Starter → PartDefinition Vehicle::Engine
The assertion helpers
Helper |
Fails when |
|---|---|
any item fails the predicate; passes on no items |
|
no item passes the predicate, including when there are no items |
|
any item passes the predicate |
|
an item satisfies |
|
two items map to the same |
|
following |
|
a source cannot reach any target along |
Each helper reads its input once and calls describe only for the elements that
appear in the message, so a rule over a large model costs no more than the loop
it replaces.
Without pytest
The helpers raise AssertionError like any assert, so a script can call them
without pytest and report the message itself. This script applies the same three
rules to the same model:
import pathlib
from collections.abc import Callable
import syside
from syside import query, testing
EXAMPLE_DIR = pathlib.Path(__file__).parent
MODEL_FILE_PATH = EXAMPLE_DIR / "example_model.sysml"
def is_user_defined(element: syside.Element) -> bool:
"""True for elements written in the model, false for library elements."""
return element.document.document_tier is syside.DocumentTier.Project
def part_definitions_used_by(
definition: syside.PartDefinition,
) -> list[syside.PartDefinition]:
"""The definitions of the parts that `definition` directly contains.
Only parts written in the model count; every part definition inherits
library parts from `Parts::Part`, which contains parts of its own type.
"""
return [
part_type
for part in query.features_of(definition)
if isinstance(part, syside.PartUsage) and is_user_defined(part)
for part_type in query.declared_types_of(part)
if isinstance(part_type, syside.PartDefinition)
]
def is_documented(definition: syside.PartDefinition) -> bool:
return query.description(definition) is not None
def rules(model: syside.Model) -> dict[str, Callable[[], None]]:
"""The modeling rules, each as a check that raises on violation."""
definitions = query.find_elements_of_type(model, syside.PartDefinition)
parts = query.find_elements_of_type(model, syside.PartUsage)
return {
"every part definition is documented": lambda: testing.assert_all(
definitions, is_documented, label="documented part definitions"
),
"part names are unique": lambda: testing.assert_foreach_unique(
parts, lambda part: part.name, label="part names"
),
"no part definition contains itself": lambda: testing.assert_acyclic(
definitions, part_definitions_used_by
),
}
def main() -> None:
(model, diagnostics) = syside.load_model([MODEL_FILE_PATH])
assert not diagnostics.contains_errors(warnings_as_errors=True)
# In a test suite each rule is a test function and pytest reports the
# AssertionError; here the message is printed instead.
for name, check in rules(model).items():
try:
check()
except AssertionError as failure:
print(f"FAIL {name}\n{failure}")
else:
print(f"PASS {name}")
if __name__ == "__main__":
main()
It prints:
FAIL every part definition is documented
assert_all documented part definitions: 1 of 5 items failed:
- PartDefinition Vehicle::Seat
PASS part names are unique
PASS no part definition contains itself