At GitLab, we believe that everyone can contribute. We also use automated testing extensively to make contributing to GitLab easier. Using automated testing is a great way to improve confidence when someone needs to change the code, which actually is the case in the majority of contributions to software projects.
But how do we ensure that our test suite covers enough to aid the confidence in changing behavior of the software, and what can we do to keep on improving it?
What is code coverage?
Using the code coverage metric is a technique that helps to improve the test suite, development process, and the software itself.
Tools used to measure the code coverage percentage usually extend the test harness environment and make it possible to map the application execution process back to the source code while automated tests are being executed. With that approach, you can not only learn how much of your code is covered by tests, but it is also possible to find out what exact parts of the codebase are not covered well enough.
Some coverage analysis tools also make it possible to generate code coverage reports in HTML format that you can then view in your browser. It makes it much easier to inspect the areas of code that are missing tests and are likely to need some improvements as well.
You can take a look at the Ruby code coverage report for GitLab that is hosted on GitLab Pages.
How to generate a code coverage report
There are a lot of code coverage tools available for many different languages, and you will need to find the most appropriate option for your particular needs. At GitLab, with projects using Ruby, we often use SimpleCov.
You will need to check the documentation for your tool of choice to learn how to generate the code coverage report. Once you are able to do this locally, check out the rest of this tutorial to learn how to publish the report with GitLab Pages!
For the sake of this example, we will assume that you are using Ruby with RSpec and SimpleCov.
How to configure your tools
Configuring SimpleCov can be as simple as extending your spec_helper.rb
with:
require 'simplecov'
SimpleCov.start
When you run the rspec
command, you will notice the code coverage report being
generated when tests are completed. The RSpec example below comes from a very simple
code that contains a single test for the single class that is there:
spec/dog_spec.rb
describe Dog do
it 'barks' do
expect(subject.bark).to eq 'Woof, woof!'
end
end
dog.rb
class Dog
def bark
'Woof, woof!'
end
end
And the RSpec test harness output is:
Dog
barks
Finished in 0.00058 seconds (files took 0.08804 seconds to load)
1 example, 0 failures
Coverage report generated for RSpec to /tmp/coverage_example/coverage. 6 / 6 LOC (100.0%) covered.
At the end of the output, you can see that code coverage report was generated
to the coverage/
directory whose contents look like:
$ ls coverage/
assets/ index.html
Yes! This is an HTML code coverage report that we can publish with GitLab Pages!
GitLab CI configuration
Take a look at our documentation
to learn more about how to use .gitlab-ci.yml
.
The GitLab CI configuration can be defined in .gitlab-ci.yml
file. Let's go
through the configuration that is necessary to publish coverage report with
GitLab Pages.