Creating Scenarios with YAML ============================ Scenarios in Curie are implemented as YAML but will include other peripheral files such as those for workload descriptions or additional scripts. Curie Scenario Structure ------------------------- Scenarios must have the following structure to be valid:: / |── test.yml |-- ├── ├── ... Scenario resource files may be referenced within each of the steps. For example, if a workload generator is used to create storage traffic in the scenario, the workload configuration files are placed in the subdirectory (alongside ``test.yml``), and are referenced in steps from the :mod:`curie.steps.workload` module. A typical scenario directory structure might look like this:: /custom_mixed_database_scenario ├── test.yml ├── dss.fio └── oltp.fio Scenarios included with X-Ray and examples on how to create scenarios can be found in the `X-Ray Scenarios Repository `_. Scenario Fields --------------- Required Fields ............... name ~~~~ The internally used name of the scenario. This must be unique. .. literalinclude:: examples/scenario/example_scenario_1/test.yml :caption: :ref:`example-scenario-1` :language: yaml :start-at: name: :end-at: name: :linenos: :lineno-match: display_name ~~~~~~~~~~~~ The externally visible name of the scenario. This must be unique. .. literalinclude:: examples/scenario/example_scenario_1/test.yml :caption: :ref:`example-scenario-1` :language: yaml :start-at: display_name: :end-at: display_name: :linenos: :lineno-match: summary ~~~~~~~ A one-line description of the scenario; A tagline. .. literalinclude:: examples/scenario/example_scenario_1/test.yml :caption: :ref:`example-scenario-1` :language: yaml :start-at: summary: :end-at: summary: :linenos: :lineno-match: description ~~~~~~~~~~~ A detailed description of the scenario. This should cover the goals of the scenario, the steps performed, and how to interpret the results. Today, this section is formatted in HTML. .. literalinclude:: examples/scenario/example_scenario_1/test.yml :caption: :ref:`example-scenario-1` :language: yaml :start-at: description: :end-before: tags: :linenos: :lineno-match: tags ~~~~ A list of strings that can be used when filtering search results. .. literalinclude:: examples/scenario/example_scenario_1/test.yml :caption: :ref:`example-scenario-1` :language: yaml :start-at: tags: :end-before: presets: :linenos: :lineno-match: estimated_runtime ~~~~~~~~~~~~~~~~~ The estimated duration of the scenario in seconds. .. literalinclude:: examples/scenario/example_scenario_1/test.yml :caption: :ref:`example-scenario-1` :language: yaml :start-at: estimated_runtime: :end-before: vms: :linenos: :lineno-match: vms ~~~ A list of VM group names, with configuration information for each. A VM group represents a group of VMs that are all identical, and perform operations at the same time. **Required** - **template** - The name of the VM image template to use. **Optional** - **target_role** - The ID of the target on which the VMs in the VM group are to be placed. This ID must be defined in the target_roles_ section. - **vcpus** - The number of vCPUs that should be allocated for each VM in the VM group. - **ram_mb** - The amount of RAM in megabytes that will be allocated for each VM in the VM group when deployed. - **data_disks** - A dictionary containing ``count`` and ``size`` of the data disks that should be created for each VM. - **nodes** - A string that describes the nodes in the cluster upon which the VMs should be deployed. This string is a comma-separated series of slices described using Python's built-in slicing behavior. - **count_per_node** - The number of VMs to deploy on each of the nodes described by ``nodes``. - **count_per_cluster** - The total number of VMs to deploy evenly across the nodes described by ``nodes``. .. note:: Either ``count_per_node`` or ``count_per_cluster`` must be defined. They can not be used together. .. literalinclude:: examples/scenario/example_scenario_1/test.yml :caption: :ref:`example-scenario-1` :language: yaml :start-at: vms: :end-before: workloads: :linenos: :lineno-match: workloads ~~~~~~~~~ A list of workload names, with configuration information for each. A workload is used to generate storage load on the cluster using a VM group. **Required** - **vm_group** - The name of the VM group to run the workload. This is defined in the vms_ section. - **config_file** - The name of the workload configuration file to run. This is usually a .fio file in the same directory as the scenario's YAML file. .. literalinclude:: examples/scenario/example_scenario_1/test.yml :caption: :ref:`example-scenario-1` :language: yaml :start-at: workloads: :end-before: results: :linenos: :lineno-match: results ~~~~~~~ A list of result names, with configuration information for each. Each result will show up as a plot in the X-Ray UI. **Required** - **workload_name** - The name of the workload generating the data for this set of results. This is defined in the workloads_ section. - **result_type** - The type of plot to render for this result. Valid choices are: - **iops** - Create a line plot showing the number of I/O operations per second over time. - **errors** - Create a line plot showing the number of cumulative I/O errors over time. - **active** - Create a line plot showing I/O activity (boolean) over time. - **latency** - Create a line plot showing latency of I/O operations over time. **Optional** - **result_hint** - Provides text to the interface on how to interpret the result. - **result_expected_value** - Provides an indication of the expected value of the result. Can be used to create a line indicating this value. - **result_value_bands** - A list of dictionaries with name, upper, and lower fields used to help create visual bands, for example a 95% prediction interval. - **aggregate** - Combine the results from all VMs associated with the workload into a single plot. Options include ``sum``, ``min``, ``mean``, ``median``, and ``max`` - **show_statistics** - Calculate summary statistics for the entire result. Produces ``min``, ``max``, ``mean``, ``median``, ``stddev``, ``var``, ``p90``, and ``p95`` as part of a result. .. literalinclude:: examples/scenario/example_scenario_1/test.yml :caption: :ref:`example-scenario-1` :language: yaml :start-at: results: :end-before: setup: :linenos: :lineno-match: setup ~~~~~ A list of step names, with parameters for each. These steps are performed during the scenario's setup phase. These are the steps that must occur before the run phase can be reached. Because these setup steps occur before the measurement period of the scenario begins, no results are rendered in the X-Ray UI during these steps. For a complete list of the steps available, see the :doc:`/apidoc/curie.steps`. .. note:: The ``test`` parameter of each step is passed in implicitly, and should not be included in the YAML. .. literalinclude:: examples/scenario/example_scenario_1/test.yml :caption: :ref:`example-scenario-1` :language: yaml :start-at: setup: :end-before: run: :linenos: :lineno-match: run ~~~ A list of step names, with parameters for each. These steps are performed during the scenario's run phase. Results are rendered in the X-Ray UI during these steps. For a complete list of the steps available, see the :doc:`/apidoc/curie.steps`. .. note:: The ``test`` parameter of each step is passed in implicitly, and should not be included in the YAML. .. literalinclude:: examples/scenario/example_scenario_1/test.yml :caption: :ref:`example-scenario-1` :language: yaml :start-at: run: :end-before: teardown: :linenos: :lineno-match: teardown ~~~~~~~~ Steps to take once the test is finished. .. literalinclude:: examples/scenario/example_scenario_1/test.yml :caption: :ref:`example-scenario-1` :language: yaml :start-at: teardown: :linenos: :lineno-match: Optional Fields ............... presets ~~~~~~~ An optional section defining common scenario variations. The presets_ section is not required, but a corresponding vars_ section must be present in order to use presets_. .. literalinclude:: examples/scenario/example_scenario_1/test.yml :caption: :ref:`example-scenario-1` :language: yaml :start-at: presets: :end-before: [vars] :linenos: :lineno-match: vars ~~~~ This optional section is for declaring the variables used throughout the scenario. You can declare public variables, which show up on the X-Ray UI, or private variables, which do not. To declare a private variable, prefix the variable name with an underscore. E.g ``_private_variable``. In addition to a mapping between a variable and a value, you can also define the constraints of a variable. For example, you may choose to specify a variable has a ``min`` and/or ``max`` value. The vars_ section is not required unless you define a presets_ section. .. literalinclude:: examples/scenario/example_scenario_1/test.yml :caption: :ref:`example-scenario-1` :language: yaml :start-after: [vars] :end-before: estimated_runtime: :linenos: :lineno-match: .. note:: In order to use a variable it must be declared in the vars_ section .. note:: When defining presets_ a corresponding vars_ section must be defined. target_roles ~~~~~~~~~~~~ An optional list of target roles can be defined when creating Scenarios which use multiple targets. The target role IDs may be passed to certain fields (e.g. vms_) or steps (e.g. :class:`curie.steps.nodes.PowerOff`) to specify which target to use. For example, this can be used to place VM groups on different targets. .. literalinclude:: examples/scenario/example_unplanned_site_failover/test.yml :caption: :ref:`example-unplanned-site-failover` :language: yaml :start-at: target_roles: :end-before: tags: :linenos: :lineno-match: Steps ----- For a complete list of the steps available, see the :doc:`apidoc/curie.steps`. .. note:: The ``test`` parameter of each step is passed in implicitly, and should not be included in the YAML. Examples -------- Examples can be found in the :doc:`examples` section.