StackStorm handbook¶
Reference installation¶
Core daemons¶
Stackstorm installation is using upstream package of StackStorm which are available on Package-cloud.
This installation also requires a repository for MongoDB, which is not included in latest repositories (for licensing reasons). A repository is provided at mongodb.org.
The StackStorm daemons requires some DBs and services to be reachable:
MongoDB as the main database. MongoDB URL is configured in
/etc/st2/st2.conf, section[database]and this requires configuring a database and the related user on the MongoDB instance.RabbitMQ as the message queue for communication between daemons. RabbitMQ should be configured with a dedicated vhost with a dedicated user. RabbitMQ URL is configured in
/etc/st2/st2.conf, section[messaging]Optionnaly, Redis as coordination system between daemons. Authentication should be configured on the Redis instance. Redis URL is configured in
/etc/st2/st2.conf, section[coordination]
Reference installation drops Mistral support because it’s usage is deprecated with StackStorm and introduces some more complexity during installation.
This reference installation is not higly-available but it may become so.
Site customization¶
Some site customization is required by the fact that all StackStorm deployments are made offline. And most officialy documented StackStorm process requires an internet access.
PyPi Repo¶
To begin with, a local pypi repository is required in order to distribute dependencies to StackStorm core or packs. To do so, a simple folder where python packages (tar.gz or pyton wheels) are downloaded.
In order to use this repo, the following pip environment variables can be used in st2 commands:
PIP_NO_INDEX=yes
PIP_FIND_LINKS="file:///volspoms2/stackstorm/py_repo"
Custom runners¶
All our cluster administration teams uses milkcheck and clustershell as administration tools, and most procedures are implemented with either one.
To be able to used both of them, StackStorm requires some glue code that represents a runner in StackStorm.
Clustershell and Milkcheck runner are already implemented and may be deployed as python packages.
To install them, launch a pip installation in StackStorm’s python virtual environment.
/opt/stackstorm/st2/bin/pip install -U /volspoms2/stackstorm/runners/milkcheck_runner/
/opt/stackstorm/st2/bin/pip install -U /volspoms2/stackstorm/runners/clush_runner/
Packs installation¶
Pack is the meat of our automation procedures, it contains all the workflow implementation and must be generic enough to be used elsewhere. As such, development process must follow StackStorm’s best practices and devellopment must be made outside of StackStorm runtime.
To create a new pack, first create a git repo with git init and drop in a file called pack.yaml which contains some metadata about the pack.
---
name : clush
description : Generic clush actions
version: 1.8.1
python_versions:
- "2"
- "3"
author : John Doe
email : john.doe@noreply.fr
You can then add all your actions, rules, etc… Implementation of such a pack may be documented later on.
To install a pack, StackStorm requires a Git URL containing all the information (Location, ref, …) about what the pack install. The format of this URL is specified in pip documentation. Don’t forget to add required pip environment variables for offline deployments.
Pack requirement may be described in a requirement.txt (for installation inside StackStorm’s virtual environment) or manually managed on the StackStorm host (for system-wide installation). In the latter, installation must be handled using hosts’s puppet agent.
# st2 run --tail packs.install packs=file:///volspoms2/stackstorm/packs/clush/ env='{"PIP_NO_INDEX": "yes", "PIP_FIND_LINKS": "file:///volspoms2/stackstorm/py_repo"}'
# st2 run --tail packs.install packs=file:///volspoms2/stackstorm/packs/diskless/ env='{"PIP_NO_INDEX": "yes", "PIP_FIND_LINKS": "file:///volspoms2/stackstorm/py_repo"}'
# st2 run --tail packs.install packs=file:///volspoms2/stackstorm/packs/slurm/ env='{"PIP_NO_INDEX": "yes", "PIP_FIND_LINKS": "file:///volspoms2/stackstorm/py_repo"}'
[...]
If packs requires some configuration, it should be done using st2 pack config.
Remember, no pack should be modified directly in /opt/stackstorm/packs. Instead, develop inside a Git repo and then deploy in StackStorm. If testing is required, please refer to the pack testing documentation. Pack tests can be launched using st2-run-pack-tests:
# /opt/stackstorm/st2/bin/st2-run-pack-tests -p /volspoms2/stackstorm/packs/clush/