Automated testing for H5P

Submitted by thomasmars on Wed, 06/08/2016 - 18:16

In this article we will focus on using Jasmine as our testing framework, and Karma as our test-runner, which will run our testing framework inside the browser.

We will progress through the following steps:

If you just want to skip to the application of the tests on our example content feel free to skip to the example.

Getting Jasmine and Karma up and running

The first thing we need to do is grab karma and jasmine-core

npm install -D karma jasmine-core

This will allow us to initialize our project with karma as the test runner:

karma init

Here we will select Jasmine as our testing framework and keep the default settings for the rest. This will create a configuration file for Karma called karma.conf.js. If we try to run Karma now it should run, but we have not told it what it should test so it will return Execute 0 of 0:

karma start

In order to get our tests running we have to tell karma where it can find them, and obviously write them.

Configuration

Common convention for Jasmine is to put your tests inside a spec folder, so we will create the spec folder at the base of our library.

Inside our spec folder we will create a separate test for each script that we want to test. In our case that will be the source-files:

The naming convention for tests is to put a spec.js at the end of the file-name of the source-files you are creating tests for, so in our case we will create the test-files:

Now, when we open up our karma configuration file we can tell karma that our test files exists in the spec folder. It is allowed to use wildcards:

files: [
  'spec/*.spec.js'
],

We can test that karma works by creating a dummy test inside css-challenge.spec.js:

describe('css-challenge', function () {
  it('should have one test', function () {
    expect(true).toBeTruthy();
  });
});

Now we get a Executed 1 of 1 SUCCESS when running

karma start

Great! Now let’s try to require our development bundle script inside our spec in order to test our source-code:

var cssChallenge = require(‘../dev.js);

When running karma start you will get a require is not defined statement. We are trying to use require in a native browser setting, which is not supported.

In order to require them together with their dependencies we can use webpack once again.

Using Webpack with Karma

The first thing we need is to grab karma-webpack:

npm install -D karma-webpack

Then we need to add it into our karma configuration, by supplying it as a preprocessor to our test files:

preprocessors: {
  'spec/*spec.js': ['webpack']

},

This time when running karma start your browsers should pop up and run smoothly.

Writing an integration test

Now that we have our development environment up, we can write tests for the usual user stories. For this particular library we will write a test that check whether our css target changes style when we type in our answer to the css-challenge.

Since the main API entry point of our content type is user input, we will simulate some input on a given element, our input element, then check that a different element, the view target element, is moved.

Beginning with the input we first find the elements:

const input = document.getElementsByClassName('h5p-css-challenge-input')[0];
const output = document.getElementsByClassName('h5p-css-challenge-target')[0];

We then set the text of the input field to a valid style:

input.textContent = 'margin:auto;';

We then create, and fire off the keyup event on the input field, in order for it to send the new value to our output element:

const event = new Event('keyup');
input.dispatchEvent(event);

At this point the event should have been caught, and the our output element should have changed, so we will check if this has actually happened:

expect(output.style.margin).toMatch(‘auto’);

Which expects the first argument to match the second one. We will also throw in a test to check that our output is not updated with invalid values:

input.textContent = ‘invalid:rule;’;
expect(output.style.margin).toMatch(‘rule;’);

That’s it for our integration test for now, you can experiment with different expectations and different integrations depending on what is most important for your content type. Your final test should look something like this:

var cssChallenge = require('../dev');

describe('css-challenge', () => {
  describe('input', () => {
    beforeEach(() => {
      this.input = document.getElementsByClassName('h5p-css-challenge-input')[0];
      this.output = document.getElementsByClassName('h5p-css-challenge-target')[0];
      this.event = new Event('keyup');
    });

    it('should change style of target element in view', () => {
      this.input.textContent = 'margin:auto;';
      this.input.dispatchEvent(this.event);
      expect(this.output.style.margin).toMatch('auto');
    });

    it('should not change style to invalid values', () => {
      this.input.textContent = 'invalid:rule;';
      this.input.dispatchEvent(this.event);
      expect(this.output.style.invalid).not.toMatch('rule');
    });
  });
});

Now we have tried to use our dev bundle to create an integration test, which had all the necessary bundling we needed in order for our tests to work. We have set it up to test one of our content.json that we use for development.

This is one way to test, that should scale well when changing the inner workings of our library, and make sure that the library still communicates well throughout the different parts.

There are other tests we can run to test the intricacies within one script. This is called unit testing, and can not be tested with our current setup, since all your code is mashed together. Thus, in order to unit test a single script we need to separate our code completely and only test units within the script. This is done by shaving everything that our source-file communicates with away using stubs or mocks.

Writing a Unit Test

This time we will attack our library at the function level. We will look at view.js which has three parts to it:

  • The constructor
  • appendTo function
  • setTargetStyle function

We will focus on the setTargetStyle function and how we can separate it from all the other logic, in order to test the smallest unit of code that we have alone.

We will start by requiring our view.js from our spec view.spec.js:

const View = require(‘../scripts/view’);

Now, in order to test setTargetStyle we have to initialize View which takes two parameters existingRulesString and answeRulesString.

const view = new View(‘targetStyle’, ‘goalStyle’);

We will then set the target style of our view using the function we are testing:

view.setTargetStyle(‘userInputStyle’);

At this point we have run all the lines required to make our function do work, and be testable, but how will we test it ? We have to look at the function to see what it actually does:

/**
 * Add given style to existing styles for target element.
 *
 * @param {string} style Style as string
 * @returns {View}
 */
this.setTargetStyle = function (style) {
  targetElement.setAttribute('style', existingRulesString + style);
  return this;
};

It sets the attribute of target element. However we do not care what the DOM does with the targetElement, that is the DOMs responsibility, so in order to test this we will just check that setAttribute is called with the correct parameters. However in order to check the correct DOM element we need to attach the View to the DOM, find our element and create a spy for the element we are testing:

view.appendTo(document.body);
const targetElement = document.getElementsByClassName('h5p-css-challenge-target')[0];
const domSpy = spyOn(targetElement, 'setAttribute');

Here we are moving into a grey zone of how much is actually supposed to happen inside a unit test. This is a choice the developer must be aware of when writing unit tests that deal with the DOM, because multiple components are naturally intertwined.

At this point we have our spy on the DOM element, so we can easily determine if setAttribute was called with the correct parameters, which according to our function should be existingRulesString and a given style:

expect(domSpy).toHaveBeenCalledWith('style', 'targetStyleuserDefinedStyle');

The final test should look something like this:

const View = require('../scripts/view');

describe('View', () => {
  it('should set target style', () => {
    const view = new View('targetStyle', 'goalStyle');
    view.appendTo(document.body);
    const targetElement = document.getElementsByClassName('h5p-css-challenge-target')[0];
    const domSpy = spyOn(targetElement, 'setAttribute');
    view.setTargetStyle('userDefinedStyle');
    expect(domSpy).toHaveBeenCalledWith('style', 'targetStyleuserDefinedStyle');
  });
});

If you want to check out the rest of the tests you can have a look at github. For the next article we will attack setting up continuous integration with H5P libraries.

Hope you have enjoyed this introduction to testing in H5P libraries. If you have any questions or comments, please leave it down below