After using SaltStack for around 3 years, it has also been almost a year since became part of SaltStack formulas maintainers. I'd like to write my thoughts about some practices and tips I learned to write high quality Salt formulas.
So these tips go as you write a formula, which could be different from one to another. But generally speaking that how is it. And if you need to review the actual result of these tips, Apache Flume formula is one of best formulas I created. Also Fluent Bit NG formula (but WIP).
1. Read official best practices!
Well, there is already a page for formulas best practices. So it's good to take a look on it. Especially if you are new to SaltStack.
However, this post has some more particular points.
2. Use proper vars structure.
Many times I created a variable like home_dir. After that I need to make another dir, let's say conf_dir. Then I realize it's better to have a key is called dirs and put all directories under it!
So to avoid ended with many related but separated vars, and rewrite many parts of the structure, it's better to use the right structure from the beginning even with a single value. Because it will end like at every time.
From:
foo: home_dir: /home/foo conf_dir: /etc/foo bar_dir: /bar
To:
foo:
dirs:
home: /home/foo
conf: /etc/foo
bar: /bar
3. Use YAML for defaults.
For some reason I found that old formulas put default variables inside map.jinja! Which requires extra unnecessary effort to deal with syntax! (a lot of quotes and brackets).
It's better to put default variables in defaults.yaml and keep map.jinja clean and just for mapping.
## defaults.yaml
Defaults:
dirs:
home: /home/foo
install:
pkg: foo
user: bar
group: baz
## map.jinja
{% import_yaml 'foo/defaults.yaml' as defaultmap %}
# Defaults.
{% set defaults = salt['grains.filter_by'](
defaultmap,
base='Defaults',
) %}
4. Split defaults per grain.
Also for long time map.jinja was used for grains defaults (the variables that change from system to another). But since now importing YAML is much easier and cleaner, it's better to keep those vars modular too.
So create YAML file for each grain like os_family, os, etc.
## osfamilymap.yaml
Debian:
install:
pkg: apache2
Red Hat:
install:
pkg: httpd
## map.jinja
{% import_yaml 'foo/osfamilymap.yaml' as osfamilymap %}
{% import_yaml 'foo/osmap.yaml' as osmap %}
[...]
# Make a list of tuples that have grains name and grain values,
# to make it easy to merge them with common defaults.
{% set vars_map = [
("os_family", osfamilymap),
("os", osmap)
] %}
# Here we update `defaults` var with grain vars.
{% for map_name, map_value in vars_map %}
{% do salt['defaults.merge'](defaults,
salt['grains.filter_by'](
map_value,
grain=map_name,
) | default({}, True)
) %}
{% endfor %}
5. map.jinja is for mapping!
As shown previously, no more static variable should be inside map.jinja. I believe that map file should only be used for "mapping" logic! If we need to add dynamic vars, we can do that inside map file.
## map.jinja
# Update dynamic vars.
{% do salt['defaults.merge'](foo, {
'service': {
'provider': salt['grains.get']('init')
},
}) %}
That means also the sls files (formula states) shouldn't have anything rather than SaltStack states unless it's really really needed (i.e. don't set Jinja vars inside the sls files).
6. Avoid redundancy.
Always try to use all options available to avoid redundancy. For example when you work with templates, you could have common parts in the templates files. Jinja macros could help a lot to have a better structure and less redundancy.
## macros.jinja
{%- macro format_key_value(conf_dict) -%}
{% for key, value in conf_dict.items() %}
{{ key }} = {{ value }}
{% endfor %}
{%- endmacro -%}
And you can import that macro and use it many times in the formula states/templates.
## templates/foo.conf.jinja
{%- import "foo/macros.jinja" as macros -%}
{{ macros.format_key_value(cfg) }}
But remember, it's better to avoid using macros too much! At the end Jinja is just a templating engine and has many limitations. And it's much better if you can do the same thing using Salt modules.
7. Modularize formula states.
Many times I write a small formula I put everything in init.sls. After sometime, I do extend the formula and start splitting it to smaller sls files and rewrite a lot of stuff.
Mostly it will end like that every time. So seriously, do it from the beginning!
Use init.sls file to include other sls files:
include: - .install - .config - .service
8. Utilize all SaltStack abilities.
SaltStack provides a lot of capabilities on many levels. Always use all options available and reduce all custom and indirect work.
For example, there are many internal variables available like sls and slspath which make life easier during wiring formulas and even when you need to edit it.
#
# During the run of this formula, it will be clear to which file does
# this state belong. So if the formula is called `foo` and the sls file
# is called `config.sls` then `sls` var will be `foo.config`.
{{ sls }}~create_foo_conf_file:
file.managed:
- name: /etc/foo.conf
#
# Also for a reason or another if you changed formula dir name,
# no need to rename it inside the formula itself, where `slspath`
# will always refer to formula dir name.
- source: salt://{{ slspath }}/templates/foo.conf.jinja
[...]
Another nice example is SaltStack custom Jinja filters. Which are not part of original Jinja filters. Here is an example for "regex_match" custom filter.
{% if 'Linux' | regex_match('.*?n.x', ignorecase=True) %}
Yay! Nix system!
{% endif %}
So before writing your own stuff, check first all what SaltStack provides like states, modules, grains, mine, reactor, Salt Cloud. Also SaltStack has intense documentation.
But also remember if you are going to share the formula publicly, always keep in mind the backward compatibility! For example, avoid to use a module or a feature is just available for latest SaltStack version. It'd be much better to use another way to do it even with a bit of effort.
9. Automate formula tests.
Well, as I mentioned before in another post about testing configuration management:
Since Infrastructure as Code (IaC) is just a "code" at the end, so it also needs to be tested to make sure everything is working as expected and ready to be applied with confidence!
If you want to share formula publicly, "testing" it is a main key to make it much much easier! KitchenCI is commonly used to do that kind of tests. Simply it provides unittest for IaC.
KitchenCI works as an abstract layer where it supports many configuration management tools to make sure everything is in place. It even supports many versifiers like InSpec and Serverspec!
With CI platforms like Travis or Drone, it's really easy to test every PR and make sure the new changes will work as expected.
10. Push to upstream.
During working with SaltStack for 3 years, I faced many challenges to make high quality formulas. I created custom modules, macros, workaround, and many other stuff. But the best part when I have a problem/need and find someone else already pushed the solution/component to upstream or shared it publicly.
That was the same for me, I enhanced modules and states I used to use, created new ones and pushed them to upstream. I believe that could be helpful for other people. And that's the best part about Open Source :-)
Take a look on SaltStack contributing guide, and SaltStack Formulas central repositories. You contributions are much appropriated even small ones :-)
Conclusion.
SaltStack has a great ecosystem and numerous capabilities, however its learning curve not so high! So after some time when you use to SaltStack, you will start to focus on creating high quality IaC using it.
You can also see all of these practices in Apache Flume formula and Fluent Bit NG formula (but last one is still in progress).
