Report Generation
=================

This example shows how to generate professional documents directly from SysML v2 models.
It takes a set of SysML models and a markdown template as input, and produces
publication-quality documents in a variety of formats (e.g. PDF, HTML, and DOCX).

Diagrams are generated in two ways: using open source Python libraries (`Graphviz
<https://graphviz.org/>`__), and via :ref:`Modeler CLI <modeler_cli>`, producing the
same output as :ref:`Syside Modeler <modeler_home>`.

The template system is general-purpose. You can produce any kind of document from your
models. Here we walk through a **requirements report** as a concrete example, which
includes traceability matrix, dependency graphs, and revision history.

.. raw:: html

    <iframe width="720" height="405" src="https://www.youtube.com/embed/Wj8piQB6fh0" frameborder="0" allowfullscreen></iframe>

What You Get
------------

The generated document contains:

- A title page with project metadata and revision history
- Requirement tables with all attributes (status, description, justification, etc.)
- Parent/child traceability between requirements
- Generated dependency graphs (SVG) per requirement
- Generated summary diagrams (SVG) showing requirements grouped by packages
- A traceability matrix linking requirements to system components
- A changelog comparing the current version against the last tagged release

.. raw:: html

    <iframe src="../_static/embeddables/requirements.html" width="100%" height="500px"
    style="border: 1px solid #ccc; border-radius: 4px;"></iframe>

.. note::

    The generated documents are also included in the downloadable example zip.

The input is a SysML v2 model containing requirements with attributes, documentation,
and relationships. For example:

.. code-block:: sysml

    requirement 'STKR-EXEC-001' : ReqDefs::'Trading Firm Requirement' {
        doc Description
        /* System shall generate consistent risk-adjusted returns
         * while maintaining regulatory compliance.
         */

        doc Justification
        /* Primary business objective to deliver value to investors
         * while managing operational and regulatory risk.
         */

        doc SuccessCriteria
        /* Achieve minimum Sharpe ratio of 2.0 over rolling 12-month
         * periods with zero regulatory violations.
         */

        allocate 'Algorithmic Trading System Design'::'System Instances'
                ::'Algorithmic Trading Platform' to self;
    }

Child requirements are linked to their parents through derivation connections:

.. code-block:: sysml

    #derivation connection 'STKR-EXEC-002 Derivation' {
        end #original original_req references
            'Stakeholder Requirements'::'Executive Stakeholder Requirements'
            ::'STKR-EXEC-002';
        end #derive derived_req_1 references
            'Stakeholder Requirements'::'User Stakeholder Requirements'
            ::'Compliance Requirements'::'STKR-COMP-001';
    }

Setup
-----

Download
~~~~~~~~

Download and extract the example model and script into a folder.

.. button-link:: report_generation.zip
    :color: primary
    :shadow:

    Download Full Model (ZIP)

External Dependencies
~~~~~~~~~~~~~~~~~~~~~

This example requires three external tools:

- `WeasyPrint <https://doc.courtbouillon.org/weasyprint/stable/first_steps.html>`__ (PDF
  generation)
- `Graphviz <https://graphviz.org/download/>`__ (dependency graph rendering)
- `Pandoc <https://pandoc.org/installing.html>`__ (DOCX conversion)

.. tab:: :nerdfont:`` Windows

    Install `Graphviz <https://graphviz.org/download/>`__ and
    `Pandoc <https://pandoc.org/installing.html>`__ using their official installers, or
    via `Chocolatey <https://chocolatey.org/>`__:

    .. code-block:: shell
        :class: code-wrapped

        choco install graphviz pandoc

    WeasyPrint requires GTK3 runtime libraries (Pango, Cairo, etc.) to be installed.
    Download and install the `GTK3 Runtime
    <https://github.com/tschoonj/GTK-for-Windows-Runtime-Environment-Installer/releases>`__
    installer. After installation, ensure the GTK3 ``bin`` directory is on your system
    ``PATH``.

.. tab:: :nerdfont:`` macOS

    .. code-block:: shell
        :class: code-wrapped

        brew install weasyprint graphviz pandoc librsvg

.. tab:: :nerdfont:`` Linux

    .. code-block:: shell
        :class: code-wrapped

        sudo apt install weasyprint graphviz pandoc librsvg2-bin

    Package names may differ on non-Debian distributions. See the official
    documentation for each tool linked above.

Python Dependencies
~~~~~~~~~~~~~~~~~~~

This example requires additional Python packages. If you are using a virtual
environment, ensure that you install them into the same ``.venv`` where Syside Automator
is installed:

.. code-block:: shell

    pip install -r scripts/requirements.txt

This installs Syside Automator and the libraries used for template processing and
document conversion (Jinja2, WeasyPrint, pypandoc, Graphviz, etc.).

Running the Generator
~~~~~~~~~~~~~~~~~~~~~

Basic generation, producing PDF, HTML, and DOCX in the output directory:

.. code-block:: shell
    :class: code-wrapped

    python scripts/generate_docs.py --output ./output

With version tracking. This compares the current model against the latest git tag and
updates the revision history and changelog:

.. code-block:: shell
    :class: code-wrapped

    python scripts/generate_docs.py --output ./output --update-version

**CLI options:**

.. list-table::
    :header-rows: 1
    :widths: 30 70

    - - Option
      - Description
    - - ``--output <path>``
      - Output directory for generated documents (required).
    - - ``--project_path <path>``
      - Path to the directory containing SysML model files. Defaults to ``models/``.
    - - ``--update-version``
      - Compare requirements against the latest git tag, update
        :file:`metadata/versions.json`, and include a changelog in the generated
        document.

Project Structure
-----------------

::

    report_generation/
    ├── models/
    │   ├── syside_demo_requirements.md        # Jinja2 template (document structure)
    │   ├── syside_demo_requirements.sysml     # Requirements model
    │   ├── syside_demo_views.sysml            # SysML view definitions for diagrams
    │   ├── syside_demo_common_defs.sysml      # Requirement type definitions
    │   ├── syside_demo_design.sysml           # System design and components
    │   ├── syside_demo_stakeholders.sysml     # Stakeholder definitions
    │   ├── syside_demo_verification.sysml     # Verification test definitions
    │   └── syside_demo_vvr.sysml              # V&V requirements
    ├── scripts/
    │   └── generate_docs.py                   # Generation script
    ├── assets/
    │   ├── styles.css                         # PDF/HTML styling
    │   ├── template.docx                      # DOCX reference template
    │   └── logo_placeholder.png               # Company logo
    ├── metadata/
    │   └── versions.json                      # Revision history
    └── output/                                # Generated documents

The template file (``syside_demo_requirements.md``) lives alongside the SysML models in
``models/`` because it references model elements directly.

How It Works
------------

The generation pipeline has four steps:

1. **Model loading.** The script loads all ``*.sysml`` files from the project directory
   using Syside Automator and builds an in-memory model.
2. **Template processing.** The Jinja2 template (``syside_demo_requirements.md``) is
   rendered. Template expressions call functions exposed by the script to extract data
   from the loaded model: requirements, attributes, relationships, documentation.
3. **Document assembly.** The rendered markdown is converted to HTML. The Syside Modeler
   CLI renders SysML views into SVG diagrams via ``syside viz view`` and embeds them
   inline. A table of contents is generated from the headings. CSS styling is applied,
   including headers/footers with project metadata.
4. **Output conversion.** The styled HTML is converted to three formats: HTML (direct),
   PDF (via WeasyPrint), and DOCX (via Pandoc).

.. tip::

    The template is the main thing you will work with. The script provides the functions
    that the template calls. You should not need to modify the script unless you are
    adding entirely new data extraction logic.

The Template
------------

The template file (:file:`models/syside_demo_requirements.md`) is a markdown file with
three kinds of content: YAML frontmatter, static markdown, and Jinja2 expressions that
pull data from the model.

Frontmatter
~~~~~~~~~~~

The top of the template contains YAML metadata between ``---`` delimiters. These fields
are used to populate the title page, headers, footers, and revision tracking:

.. code-block:: yaml

    project_title: "Algorithmic Trading System"
    document_title: "Software Requirements Specification"
    author: "Sensmetry"
    project_reference: "DEMO-2025-001"
    document_version: "1.1"
    document_version_description: "Non-functional SysML changes."
    company_website: "www.example.com"
    company_email: "info@example.com"
    company_phone: "XXX XXX XXXX"

These values are passed to template functions like ``title()`` and are also used by the
script to generate PDF headers and footers. To customize the output for your project,
update these fields.

Static Content
~~~~~~~~~~~~~~

Regular markdown that appears in the output as-is. For example, the Introduction section
of the demo template is plain markdown:

.. code-block:: markdown

    ## Introduction

    ### Purpose

    This document serves as the Software Requirements Specification (SRS) for the
    Algorithmic Trading System. It outlines the comprehensive set of requirements
    that must be met by the system...

You can add, remove, or edit static sections freely. They are standard markdown, no
special syntax needed.

Dynamic Content
~~~~~~~~~~~~~~~

Jinja2 expressions (``{{ }}``) call template functions to extract and format data from
the loaded SysML model. Here is how the Requirements section of the demo template works:

.. code-block:: markdown

    ## Requirements

    {{
        repeat_for_each_item(
            data_source=get_children_with_attributes(
                root_package="Algorithmic Trading Requirements",
                metatype="RequirementUsage",
                ignore_abstract_metatypes=True,
                attributes={
                    "Name": "ElementName",
                    "Description": "Documentation",
                    "SuccessCriteria": "Documentation",
                    "Justification": "Documentation",
                    "Parents": "Req_Parents",
                    "Derivations": "Req_Derivations",
                    "Implemented": "Req_Implemented",
                    "verifyRequirement": "Req_Verified",
                    "Graph": "Req_DependencyGraph",
                    "Package": "OwningNamespace",
                    "Status": "AttributeUsage"
                },
            ),
            sections=[
                {"type": "h3", "value": "unique_headers_from_qualified_name(row[9])"},
                {"type": "h4", "value": "row[0]"},
                {"type": "h5", "value": "Requirement details"},
                {"type": "table", "value": "generic_table(
                    columns=[...],
                    rows=[row[10:11] + row[0:6] + row[8:9]],
                    flipRowsAndCols=True,
                    ...
                )"},
            ],
            page_break_between_rows=True
        )
    }}

This breaks down as:

1. ``get_children_with_attributes()`` extracts all ``RequirementUsage`` elements from
   the ``Algorithmic Trading Requirements`` package, along with their attributes. Each
   element becomes a row (a list of attribute values in the order defined by the
   ``attributes`` dictionary).
2. ``repeat_for_each_item()`` loops over those rows. For each requirement, it renders
   the ``sections`` list: headings, tables, and text. Inside section values, ``row[0]``
   refers to the first attribute (``Name``), ``row[1]`` to the second (``Description``),
   and so on.
3. ``generic_table()`` formats a subset of each row's attributes into an HTML table. The
   ``flipRowsAndCols=True`` flag transposes it so attribute names are row headers
   (useful for single-item detail views).

.. note::

    Row indices always correspond to the order of keys in the ``attributes`` dictionary.
    If you add, remove, or reorder attributes, the indices in ``row[N]`` references and
    row slicing (e.g., ``row[10:11] + row[0:6]``) must be updated to match.

The template also uses simpler expressions for standalone elements:

.. code-block:: markdown

    {{ title() }}             {# Title page from frontmatter #}
    {{ toc() }}               {# Table of contents #}
    {{ page_break() }}        {# Page break for PDF/DOCX #}
    {{ revision_history() }}  {# Version table from versions.json #}
    {{ changelog() }}         {# Diff against last git tag #}

To embed a view diagram, call ``render_view()`` with the qualified name of the view. For
example, the demo template adds a requirement hierarchy diagram under the Introduction
section:

.. code-block:: markdown

    {{
        render_view(
            view=
                "'Algorithmic Trading Views'"
                "::'Requirement Views'"
                "::'Executive Requirements'",
            width="50%"
        )
    }}

The view must be defined in a SysML model file (see
:file:`models/syside_demo_views.sysml`). The ``render_view()`` call invokes ``syside viz
view``, reads the resulting SVG, and embeds it inline scaled to input width.

Customization
-------------

Branding
~~~~~~~~

- **Company information**: update the frontmatter fields at the top of the template:

  .. code-block:: yaml

      author: "Your Company Name"
      company_website: "www.yourcompany.com"
      company_email: "info@yourcompany.com"
      company_phone: "+1 234 567 8900"

- **Logo**: replace :file:`assets/logo_placeholder.png` with your company logo.
- **PDF/HTML styling**: modify :file:`assets/styles.css` to change fonts, colors,
  spacing, and table styles.
- **DOCX styling**: modify :file:`assets/template.docx` to set Word document styles
  (paragraph formats, heading styles, page layout).

Adding Attributes
~~~~~~~~~~~~~~~~~

To add a new attribute (e.g., ``Priority``) to the generated requirements:

1. Add the attribute to your SysML model:

   .. code-block:: sysml

       requirement 'STKR-RISK-001' : ReqDefs::'Risk Management Requirement' {
           attribute Priority = "High";
           ...
       }

2. Add it to the ``attributes`` dictionary in the template's Requirements section:

   .. code-block:: markdown

       attributes={
           "Name": "ElementName",
           "Description": "Documentation",
           ...
           "Status": "AttributeUsage",
           "Priority": "AttributeUsage"
       },

3. Include it in the table columns and row slicing. The new attribute's index
   corresponds to its position in the ``attributes`` dictionary (in this case,
   ``row[11]``):

   .. code-block:: markdown

       columns=["Status", "Identifier", "Description", ..., "Priority"],
       rows=[row[10:11] + row[0:6] + row[8:9] + row[11:12]],

4. Regenerate the document.

Changing Model References
~~~~~~~~~~~~~~~~~~~~~~~~~

If your SysML model uses different package names, update the ``root_package`` parameter
in all template function calls. For example:

.. code-block:: markdown

    root_package="'Algorithmic Trading Requirements'"

becomes:

.. code-block:: markdown

    root_package="'Your Package Name'"

The ``metatype`` parameter controls which kind of elements are extracted. Common values:

.. list-table::
    :header-rows: 1
    :widths: 35 65

    - - Metatype
      - Extracts
    - - ``"RequirementUsage"``
      - Requirements
    - - ``"ItemUsage"``
      - Generic items (used for reference documents in the demo)
    - - ``"PartUsage"``
      - System components (used in the traceability matrix)
    - - ``"RequirementDefinition"``
      - Requirement type definitions

Tables and Layout
~~~~~~~~~~~~~~~~~

The ``generic_table()`` function accepts several layout parameters:

.. list-table::
    :header-rows: 1
    :widths: 25 75

    - - Parameter
      - Description
    - - ``widths``
      - Column width percentages, e.g., ``["20%", "25%", "55%"]``
    - - ``css_class``
      - CSS class for custom styling (e.g., ``"table-2d"`` for detail tables,
        ``"matrix-table"`` for traceability matrices)
    - - ``col_alignment``
      - Text alignment per column: ``"left"``, ``"right"``, or ``"center"``
    - - ``flipRowsAndCols``
      - Transpose the table so columns become rows. Useful for single-item detail views
        where attribute names are row headers.

Requirement Relationships
~~~~~~~~~~~~~~~~~~~~~~~~~

The template can extract several types of requirement relationships. These are specified
as attribute types in the ``attributes`` dictionary:

.. list-table::
    :header-rows: 1
    :widths: 30 70

    - - Attribute type
      - Description
    - - ``"Req_Parents"``
      - Parent requirements (requirements that derive this one)
    - - ``"Req_Derivations"``
      - Derived child requirements
    - - ``"Req_Implemented"``
      - Components allocated to implement this requirement
    - - ``"Req_Verified"``
      - Verification elements linked to this requirement
    - - ``"Req_DependencyGraph"``
      - Auto-generated SVG showing the derivation tree

These relationships are extracted from the SysML model structure: derivation
connections, allocation usages, and verification memberships respectively.

Adding and Removing Sections
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. tip::

    The template is a plain markdown file. You can freely add, remove, or reorder
    sections.

To add a new section, insert markdown and template expressions at the desired position.
To remove a section, delete it. The script processes whatever is in the template
top-to-bottom.

For example, to add a summary table of all requirements (without per-requirement detail
pages), you could add:

.. code-block:: markdown

    ## Requirements Summary

    {{
        generic_table(
            columns=["Status", "Name", "Description"],
            rows=get_children_with_attributes(
                root_package="Algorithmic Trading Requirements",
                metatype="RequirementUsage",
                ignore_abstract_metatypes=True,
                attributes={
                    "Name": "ElementName",
                    "": "Documentation",
                    "Status": "AttributeUsage"
                }
            ),
            widths=["15%", "25%", "60%"]
        )
    }}

Template Function Reference
---------------------------

All functions below are available inside ``{{ }}`` expressions in the template.

Document Structure Functions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``title()``
    Generates the title page HTML using frontmatter metadata (project title, document
    title, author, reference number, version, company info, and logo).

``toc()``
    Generates a table of contents from document headings (up to 4 levels deep). The
    placeholder is replaced after the full document is rendered, so it reflects all
    headings.

``page_break()``
    Inserts a page break. Rendered as a ``<div>`` with CSS ``page-break-after: always``,
    which applies to PDF and DOCX output.

``revision_history()``
    Generates a revision history table from :file:`metadata/versions.json`. Each entry
    shows version, date, description, and a summary of changes.

``changelog()``
    Compares the current model against the latest git tag and generates a detailed
    changelog showing added, modified, and deleted requirements with their attribute
    changes. Requires the ``--update-version`` CLI flag to be active.

Data Extraction Functions
~~~~~~~~~~~~~~~~~~~~~~~~~

``get_children_with_attributes(...)``
    Extracts elements from the SysML model and returns their attributes. Each element
    becomes a row (a list of strings), with values in the same order as the
    ``attributes`` dictionary.

    Parameters:

    .. list-table::
        :header-rows: 1
        :widths: 30 70

        - - Parameter
          - Description
        - - ``root_package``
          - Qualified name of the package to search, e.g., ``"Algorithmic Trading
            Requirements"``
        - - ``metatype``
          - Element type to extract (``"RequirementUsage"``, ``"ItemUsage"``,
            ``"PartUsage"``, ``"RequirementDefinition"``, etc.)
        - - ``ignore_abstract_metatypes``
          - If ``True``, skip abstract type definitions
        - - ``attributes``
          - Dictionary mapping display names to extraction types (see table below)

    Extraction types for the ``attributes`` dictionary:

    .. list-table::
        :header-rows: 1
        :widths: 25 75

        - - Type
          - Description
        - - ``"ElementName"``
          - The element's name (dictionary key is ignored)
        - - ``"Documentation"``
          - Documentation body. The key selects which ``doc`` block: use ``""`` for the
            default (unnamed) doc, or a name like ``"Description"`` to match ``doc
            Description /* ... */``
        - - ``"AttributeUsage"``
          - Value of a named attribute. The key specifies the attribute name (e.g.,
            ``"Status"``, ``"Prefix"``)
        - - ``"OwningNamespace"``
          - Qualified name of the parent namespace
        - - ``"Req_Parents"``
          - Parent requirements (via derivation connections)
        - - ``"Req_Derivations"``
          - Derived child requirements
        - - ``"Req_Implemented"``
          - Allocated components (via allocation usages)
        - - ``"Req_Verified"``
          - Verification elements (via verification memberships)
        - - ``"Req_DependencyGraph"``
          - SVG dependency graph of derivation relationships

``get_docs(...)``
    Extracts documentation elements from a package. Returns a list of ``[doc_name,
    doc_body]`` pairs. If ``doc_name`` is provided, only matching documentation blocks
    are returned.

Formatting Functions
~~~~~~~~~~~~~~~~~~~~

``generic_table(...)``
    Generates an HTML table.

    .. list-table::
        :header-rows: 1
        :widths: 25 75

        - - Parameter
          - Description
        - - ``columns``
          - Column header labels
        - - ``rows``
          - List of rows, each a list of cell values
        - - ``flipRowsAndCols``
          - Transpose the table (columns become rows). Default ``False``
        - - ``widths``
          - Column width percentages, e.g., ``["25%", "75%"]``
        - - ``css_class``
          - CSS class to apply (e.g., ``"table-2d"``, ``"matrix-table"``)
        - - ``col_alignment``
          - Per-column alignment: ``"left"``, ``"right"``, or ``"center"``

``headless_table(...)``
    Like ``generic_table`` but uses the first row of ``data`` as column headers.

``list_items(...)``
    Formats rows as a markdown list. Use ``{0}``, ``{1}``, etc. to reference columns by
    index (matching the order in the ``attributes`` dictionary).

    Example format string:

    .. code-block:: markdown

        "- **[{0}]({2})** - {1}"

    This produces a list of linked items.

``text(...)``
    Extracts specific rows from data and formats them. The ``rows`` parameter selects
    which rows to include by index. Useful for extracting a single value.

Iteration Functions
~~~~~~~~~~~~~~~~~~~

``repeat_for_each_item(...)``
    Loops over ``data_source`` and renders ``sections`` for each row. Each section is a
    dictionary with two keys:

    .. list-table::
        :header-rows: 1
        :widths: 20 80

        - - Key
          - Description
        - - ``"type"``
          - ``"h1"`` through ``"h6"`` for headings, ``"table"`` for tables, ``"body"``
            for text
        - - ``"value"``
          - A string that can reference ``row[0]``, ``row[1]``, etc. For tables, the
            value is a ``generic_table(...)`` call as a string, which is evaluated with
            the current row in scope.

    Set ``page_break_between_rows=True`` to insert page breaks between items.

Specialized Functions
~~~~~~~~~~~~~~~~~~~~~

``render_view(view, width=None)``
    Renders a SysML view to an SVG diagram and returns it as an HTML fragment.

    Invokes ``syside viz view <models_dir> -n <view>`` in a temporary directory, reads
    the resulting ``*.svg`` file, sanitizes it for inline embedding (strips XML/DOCTYPE
    headers, prefixes SVG IDs to avoid collisions), ensures a ``viewBox`` attribute is
    present, and returns the SVG. Inline styles (``max-width:{width}``, ``width:100%``,
    ``height:auto``) are applied so it scales to the content column in both PDF and HTML
    output. ``width`` defaults to ``100%``.

    The view must be defined in a SysML model file loaded by the script (e.g.,
    :file:`models/syside_demo_views.sysml`). See :ref:`modeler_diagram_views` for how to
    define views.

    .. list-table::
        :header-rows: 1
        :widths: 20 80

        - - Parameter
          - Description
        - - ``view``
          - Qualified name of the SysML view to render, e.g., ``"'Algorithmic Trading
            Views'::'Requirement Views'::'Stakeholder Requirements Derivation'"``
        - - ``width``
          - Maximum width of the rendered SVG (e.g., ``"600px"``). Defaults to
            ``"100%"``.

``traceability_matrix(...)``
    Builds a traceability matrix. Rows are elements from ``row_package`` (e.g.,
    requirements), columns are elements from ``col_package`` (e.g., system components).
    Cells are marked with ``X`` where an allocation relationship exists. The result is
    passed to ``headless_table()`` for rendering.

``unique_headers_from_qualified_name(...)``
    Extracts the last segment of a qualified namespace name and deduplicates it across
    calls. Useful for generating package-level section headings when iterating over
    requirements. A new heading is only emitted when the package changes.
