When everything is built SCAP Security Guide (or SSG) is a bunch of SCAP files – source datastream, XCCDF, OVAL, OCIL, CPE dictionary and other files. But these files are huge and hard to work on. So developers of SSG split everything up and use a rather complex build system to merge it into bigger files. This helps prevent git conflicts and other nasty problems. The issue is that it also gets harder to figure out what to change if we want to affect the final built file.

In this blog post I will cover where various parts of the XCCDF (also part of the source datastream) come from. We will cover benchmark and rule metadata – title, description, rationale, identifiers and rule remediations – both bash and ansible. So after reading this blog post you will be able to contribute any of those.

Cloning the repository and git flow basics

Go to https://github.com/OpenSCAP/scap-security-guide and click the “Fork” button. This will create your own copy of the upstream repository so that you can make changes to it and suggest the upstream to adopt them using pull requests. After you have your own copy of scap-security-guide, clone it using git.

git clone git@github.com:mpreisler/scap-security-guide.git

Replace the username with your own. At this point I recommend keeping the “origin” remote pointing to your fork and setting up an “upstream” remote so that you can easily pull latest changes other developers integrated.

cd scap-security-guide
git remote add upstream https://github.com/OpenSCAP/scap-security-guide.git

Let’s say I want to work on an amazing new feature. First I’d go to the master branch and make sure it’s in sync with upstream.

git checkout master
git pull upstream master --ff-only
git push origin master # push latest upstream to your fork not necessary but helps if you develop on multiple machines
git checkout -b new_feature
# do all the changes
git push origin new_feature

Now go to https://github.com/OpenSCAP/scap-security-guide and click “New pull request”, use the new_feature branch.

If you there are conflicts you need to resolve them. Here is how:

# we are in the new_feature branch
git checkout master
git pull upstream master --ff-only
git push origin master # push latest upstream to your fork not necessary but helps if you develop on multiple machines
# go back to new_feature branch
git checkout new_feature
git rebase -i master # this is the most important step, base our new_feature branch on latest upstream master instead of the original commit where we branched
git push --force origin new_feature # push force the rebased branch to our fork
# the pull request will update automatically

Build process overview

There are many intermediate steps before the final source datastream is built. Let’s only focus on XCCDF in this blog post, I will cover OVAL and other files in the future.

Separate files -> shorthand XML -> XCCDF 1.1 -> XCCDF 1.2 -> source datastream

When contributing we only change the separate source files. Changing anything in the “output” directory is futile, the changes will be overwritten.

After you have made the changes run:

make -j 4

You can run this command either from the root directory of the git repository, or you can go to the product’s directory and run it there. Running it from root dir will build all products, running it from the product’s directory will only build that product.

To test your changes go to RHEL/7/output and use ssg-rhel7-ds.xml for evaluation / testing.

Benchmark title, description, intro guidance

Let’s walk through the XCCDF file from the beginning to the end.

The Benchmark is the root element and its data come first in the XCCDF. Since the introductory text is mostly the same for various OSes it is shared between multiple products.

To change the title, description, front-matter, rear-matter go to shared/xccdf/shared_guide.xml.

If you want to change the introductory text and disclaimers go to shared/xccdf/intro and choose either shared_intro_app.xml or shared_intro_os.xml depending on the type of the product you want to affect. OS affects RHEL6, 7, … App affects JRE, Chromium, … The contents of the files should be pretty self-explanatory, it is the XCCDF format without namespaces and a few other formalities that are added automatically during the build.

Rule metadata

It gets a bit more complicated with rules. Some are shared and some aren’t so first we need to figure out where the rule we need to change is coming from. I will use RHEL7 and the ensure_gpgcheck_repo_metadata rule ID as an example.

First we need to figure out which group the rule belongs to. You can do this using vim or another text editor but it’s much simpler to use SCAP Workbench.

scap-workbench ssg-rhel7-xccdf.xml

Choose any profile and click Customize. Use the search box to search for the rule ID. We can see that the parent XCCDF Group is “updating”, its parent group is “software”, its parent group is “system” and that is a top level group. So here is how the hierarchy goes:

system/software/updating/ensure_gpgcheck_repo_metadata

Now let’s go to RHEL/7/input and open guide.xslt. We will see a line like this:

 <xsl:apply-templates select="document(concat($SHARED_RP, '/xccdf/system/system.xml'))" />

This tells us that the entire system group is shared. Let’s go to shared/xccdf/system. In that directory we see a “software” subdirectory and inside it is finally the “updating.xml” that represents the “updating” Group. After we open it we finally see where Rule title, description, identifiers and other metadata are coming from.

When changing these, keep in mind that they are using in other products, not just the one you are testing.

Remediations

The situation was simple with Benchmark, a little more complex with Rule and with remediations, you guessed it, it’s going to get even more complicated 🙂

Remediations can be “static”, typically specific to just one rule and product or they can be generated from templates and the template then applies to multiple rules and sometimes even multiple products.

Let’s keep using our ensure_redhat_gpgkey_installed example from RHEL7. We can see that in the XCCDF there is a bash remediation in the <fix> element. So where is this coming from? Answering that is quite difficult and even though you can deduce it from the build system I recommend using “find” or “grep” to do it because that’s going to be simpler most of the times.

$ find . | grep ensure_redhat
./shared/templates/static/bash/ensure_redhat_gpgkey_installed.sh
./shared/oval/ensure_redhat_gpgkey_installed.xml

And if we go to ./shared/templates/static/bash/ensure_redhat_gpgkey_installed.sh and look at the file it is indeed the source of the remediation. This bash remediation file is just a normal bash snippet with one exception: the # platform line. Depending on its value it is or isn’t included in various products. This one says multi_platform_rhel which means it will get included in all the versions of RHEL. Check out the “shared/modules/map_product_module.py” file for all the possible values.

In this example the remediation is not templated even though it is in the “templates” directory. That is very confusing and we most likely will change that in the future.

Different example – Ansible remediations

The rule we just looked at doesn’t have an Ansible remediation yet. Let us look at another example to explore how ansible remediations are included. I picked the package_aide_installed rule from RHEL7.

Using the same tricks we will find:

shared/templates/output/bash/package_aide_installed.sh
shared/templates/output/ansible/package_aide_installed.yml

Changing those files will temporarily change the final built XCCDF and SDS but that will not persist and those files are not tracked by git. So where do they come from?

They are generated using shared/templates/create_package_installed.py which uses csv/packages_installed.csv and template_ANSIBLE_package_installed and template_BASH_package_installed.

If we want to alter the remediation the file we need to modify depends on the type of change. If the change applies to all package installed remediations we should change the template_* files. If we need to specialize this remediation we need to remove aide from the csv file and create a new remediation in shared/templates/static/{ansible,bash}. If we need to start building a new remediation for a new package we add that package to the csv file and run the build system from scratch.

How are these remediation files used?

Templates are used to build the final remediation snippet, these snippets are then combined using shared/utils/combine-remediations.xml into a huge remediation XML file. This file is used to insert them into the XCCDF.

(contributed by Zbynek Moravec) The prioritization of the various folders is as follows – left = highest priority:

product static > product template > shared static > shared template

Conclusion

I hope this blog post shed some light on the arcane magic of the SCAP Security Guide build system. Let me know in the comment section if something wasn’t clear and what you want to read about in part 2.

Check out the upstream Contribution guide in the meantime: https://github.com/OpenSCAP/scap-security-guide/wiki/Contributing