Getting Started with Testing in Rails

This guide gives an introduction to built-in mechanisms in Rails for testing your application.

After reading this guide, you will know:

• Rails testing terminology.
• How to write unit tests for your applications models and controllers
• How to write system tests for your application.

This guide is a shorter version of the offical Guide to Testing Rails Applications.

Slides - use arrow keys to navigate, esc to return to page view, f for fullscreen

1 Why Write Tests?

Rails makes it super easy to write your tests. It starts by producing skeleton test code while you are creating your models and controllers.

By running your Rails tests after every change in the code you can ensure that you did not break anything.

Rails tests can also simulate browser requests without or with javascript and thus you can test your application's response without having to test it through your browser.

2 Setup

2.1 Folders and Files

Rails creates a test directory for you. If you list the contents of this directory you will find two files that hold the configuration for your tests:

• test_helper.rb global test configuration
• application_system_test_case.rb configure a browser for system test

and several directories:

The models, controllers, mailers, channels, and helpers directory hold tests for (surprise) models, controllers, mailers, channels and view helpers respectively. These tests that are focussed on one single class are also called unit tests.

The integration directory hold tests that exercise the whole Rails stack, from HTTP Request through routes, controllers, down to models and back up to the view. The only thing left out is the client side of your app: Javascript cannot be tested here.

The system directory is meant to hold tests that test the whole system by accessing the app with a browser, including running the javascript.

Fixtures are a way of organizing test data; they reside in the fixtures directory.

2.2 The Test Environment

By default, every Rails application has three environments: development, test, and production.

Each environment has a configuration file in config/environments/. For the test environment the file is config/environments/test.rb.

In the Gemfile you can add gems that are only used in one environment by putting them in a section like so:

# Gemfile
# use gem capybara only in test environment:
group :test do
  gem 'capybara', '~> 2.13'
end

2.3 Write one Test

When you use a generator it will also create basic tests and fixtures for you:

$ bin/rails generate model article title:string body:text
...
create  app/models/article.rb
create  test/models/article_test.rb
create  test/fixtures/articles.yml
...

require 'test_helper'

class ArticleTest < ActiveSupport::TestCase
  # test "the truth" do
  #   assert true
  # end
end

A line by line examination of this file will help get you oriented to Rails testing code and terminology.

require 'test_helper'

By requiring the file test_helper.rb the default configuration to run our tests is loaded. You will include this with all the tests you write.

class ArticleTest < ActiveSupport::TestCase

The ArticleTest class defines a test case because it inherits from ActiveSupport::TestCase, which in turn inherits from Minitest::Test.

Inside this class you will define the tests, either by giving them a method name beginning with test_ (case sensitive) or by using this syntax:

test "the truth" do
  assert true
end

Which is approximately the same as writing this:

def test_the_truth
  assert true
end

Next, let's look at our first assertion:

assert true

An assertion is a line of code that evaluates an expression for expected results. assert true is always satisfied, so this test always passes.

In a real test an assertion can check many things:

• does this value equal that value?
• is this value nil?
• does this line of code throw an exception?
• is the user's password longer than 5 characters?

Every test must contain at least one assertion, with no restriction as to how many assertions are allowed. Only when all the assertions are successful will the test pass.

3 Test Driven Development

Here an example of a useful test, written with Test-Driven Development (TDD).

Test Driven Development means:

1. Write a test - it fails - "RED"
2. Write some code to make the test pass - "GREEN"
3. Improve your code, but make sure the test still passes - "REFACTOR"

For this example I want to add a validation to my article class to forbid very short or missing titles.

3.1 Red

I start by writing this test:

test "article title needs to be at least 3 characters long" do
  a = Article.new(title: 'x')
  assert_not article.save
end

To run all the tests for your project use rails test.

If you run the test above the result might look like this:

$ rails test
Run options: --seed 33837

# Running:

........F

Finished in 0.492655s, 18.2684 runs/s, 30.4473 assertions/s.

  1) Failure:
ArticleTest#article_should_not_save_article_without_title [test/models/article_test.rb:10]:
Expected true to be nil or false

9 runs, 15 assertions, 1 failures, 0 errors, 0 skips

In the output, F denotes a failure. You can see the corresponding trace shown under 1) along with the name of the failing test. The next few lines contain the stack trace followed by a message that mentions the actual value and the expected value by the assertion. The default assertion messages provide just enough information to help pinpoint the error.

3.2 Green

Now to get this test to pass we can add a model level validation for the title field.

class Article < ApplicationRecord
  validates :title, length: { minimum: 3 }
end

Now the test should pass. Verify this by actually running the test!

$ rails test
Run options: --seed 8625

# Running:

........

Finished in 0.498780s, 16.0391 runs/s, 28.0685 assertions/s.

8 runs, 14 assertions, 0 failures, 0 errors, 0 skips

Every dot stands for one test that ran through sucessfully.

4 Unit Tests

4.1 What an error in you test looks like

An error is different from a failing test. An error is a problem in your test code, not your application code. To see how an error gets reported, here's a test containing an error:

test "should report error" do
  # some_undefined_variable is not defined elsewhere in the test case
  some_undefined_variable
  assert true
end

Now you can see even more output in the console from running the tests:

$ rails test
.....E...

Finished tests in 0.030974s, 32.2851 tests/s, 0.0000 assertions/s.

  1) Error:
test_should_report_error(ArticleTest):
NameError: undefined local variable or method `some_undefined_variable' for #<ArticleTest:0x007fe32e24afe0>
    test/models/article_test.rb:10:in `block in <class:ArticleTest>'

10 tests, 10 assertions, 0 failures, 1 errors, 0 skips

Notice the 'E' in the output. It denotes a test with an error.

When a test fails you are presented with the corresponding backtrace. By default Rails filters that backtrace and will only print lines relevant to your application. Read the backtrace!

4.1.1 How to catch exceptions from your application code

If you want to ensure that an exception is raised by your application code you can use assert_raises like so:

test "MyClass no longer implements @@counter, raises error" do
  assert_raises(NameError) do
    MyClass.counter
  end
end

4.2 Available Assertions

You have seen some assertions above.

Here's an extract of the assertions you can use with Minitest, the default testing library used by Rails. The [msg] parameter is an optional string message that is only displayed if the test fails. It is available in all assertions, but only shown in the first one here. For most assertions there is a simple negation assert and assert_not, assert_equal and assert_no_equal, and so on.

The above are a subset of assertions that minitest supports. For an exhaustive & more up-to-date list, please check Minitest API documentation, specifically Minitest::Assertions.

Because of the modular nature of the testing framework, it is possible to create your own assertions. In fact, that's exactly what Rails does. It includes some specialized assertions to make your life easier.

4.3 Better Asserstion

better assertion with ramcrest

4.4 Testing a Model

Models are easy to test separately because they do not depend on other code (most of the time). In Rails the tests for a model X are found in test/models/x_test.rb.

All the examples shown so far are from model tests. Below you can see what the a complete model test could look like. There is a setup section that will be executed before each test.

# file test/models/course_test.rb
require 'test_helper'

class CourseTest < ActiveSupport::TestCase
  setup do
    @course = courses(:one)
  end

  test "should not save new course without title" do
    course = Course.new
    assert_not course.save, "Saved new course without a title"
  end
  test "cannot save existing course after removing title" do
    @course.title = ''
    assert_not @course.save, "Saved existing course without a title"
  end
  test "Course no longer implements .counter, raises error" do
    assert_raises(NameError) do
      @course.counter
    end
  end
end

So is the model test a unit test? It only tests one unit of source code that you have written. It also exercises ActiveRecord and the test database. So you could argue that it is more than just a unit test. But for now it is a near a unit test as we can get. We will look at more advanced testing later one. There you will learn how to build tests that only test one unit of code.

4.5 Testing a Controller

Controller tests exercise several parts of the rails stack: routing, the controller, models, the test database. But they do try to keep views out of the mix.

To activate controller test add to your Gemfile:

group :development, :test do
  gem 'rails-controller-testing'
end

Testing the controller without testing the view at the same time is quite tricky. You should test for things such as:

• was the web request successful?
• was the user redirected to the right page?
• was the user successfully authenticated?
• was the correct object sent to the view?

In a controller test you can use the methods get, post, and so on. These will be handled by rails routing as usual, and end up calling an action in the controller with certain parameters.

In this example the create action of the article controller is called by sending a post request to articles_url. The params hash is set up with key article[title] and value some title:

post articles_url, params: { article: { title: 'some title' } }

This example just shows params, you can also set:

• headers a hash of HTTP Request headers
• env a hash of environment variables
• xhr true or false to make this an AJAX request or not
• as to request a content type, for example as: :json
• cookies to set cookies included in the request

After the request you get the response and three hashes:

• session
• flash
• cookies

Rails adds some custom assertions for controllers. You can see them at work in the tests created by scaffold:

assert_response checks the status code of the HTTP response generated by the controller:

# test/controller/articles_controller_test.rb
test "should get new" do
  get :new
  assert_response :success
end

assert_difference checks a value before and after a block of code is run, to make sure that the value changed by one. This is often used when creating models:

test "should create article" do
  assert_difference('Article.count') do
    post :create, params: { article: { title: 'some title' } }
  end
end

assert_redirect makes sure the controller returns a HTTP redirect header to the appropriate url:

test "should destroy article" do
  assert_difference('Article.count', -1) do
    delete :destroy, params: { id: 1 }
  end

  assert_redirected_to articles_path
end

5 Test Data

Just about every Rails application interacts heavily with a database and, as a result, your tests will need a database to interact with as well. To write efficient tests, you'll need to understand how to set up this database and populate it with sample data.

The database for each environment is configured in config/database.yml.

A dedicated test database allows you to set up and interact with test data in isolation. This way your tests can mangle test data with confidence, without worrying about the data in the development or production databases.

5.1 Maintaining the test database schema

In order to run your tests, your test database will need to have the current structure. The test helper checks whether your test database has any pending migrations. If so, it will try to load db/schema.rb into the test database. If migrations are still pending, an error will be raised. Usually this indicates that your schema is not fully migrated. Running the migrations against the development database (rails db:migrate) will bring the schema up to date.

5.2 Test Data with Fixtures

For good tests, you'll need to give some thought to setting up test data. In Rails, the most simple way of doing this is by defining and customizing fixtures. Fixtures are database independent and written in YAML. There is one file per model.

You'll find fixtures under your test/fixtures directory. When you run rails generate model to create a new model, Rails automatically creates fixture stubs in this directory.

5.2.1 YAML

YAML-formatted fixtures are a human-friendly way to describe your sample data. These types of fixtures have the .yml file extension (as in users.yml).

Here's a sample YAML fixture file:

# I am a YAML comment
david:
  name: David Heinemeier Hansson
  birthday: 1979-10-15
  profession: systems development

steve:
  name: Steve Ross Kellock
  birthday: 1974-09-27
  profession: guy with keyboard

Each fixture is given a name followed by an indented list of colon-separated key/value pairs. Records are typically separated by a blank line. You can place comments in a fixture file by using the # character in the first column.

If there are associations between models, you can simply refer to the fixture in a related model using its name. Here's an example with a belongs_to/has_many association:

# In fixtures/categories.yml
about:
  name: About This Site

# In fixtures/articles.yml
one:
  title: Welcome to Rails!
  body: Hello world!
  category: about

The category key of the one article found in fixtures/articles.yml has a value of about. This tells Rails to load the category about found in fixtures/categories.yml.

5.2.2 Using fixtures in Tests

The data defined by the fixture files will be available in your tests as Active Record objects. For example:

# in test/models/user_test.rb
# the User object for the fixture named david
users(:david)

# turn the property for david called id
users(:david).id

# one can access methods available on the User class
users(:david).partner.email

To get multiple fixtures at once, you can pass in a list of fixture names. For example:

# this will return an array containing the fixtures david and steve
users(:david, :steve)

6 System Tests

System test are used to test that the various parts of your application interact correctly to implement features. In a system test a real browser is used to test your app, including the client side javascript.

Rails comes with built in system tests. These are stored in the folder test/system/

These tests take a lot more time to run than the unit test discussed earlier. They are not included if you run rails test, you have to start them separately with rails test:system.

We will use the gems capybara and selenium-webdriver. they will enable us to sue headless browser firefox or chrome for system tests.

# Gemfile
group :test do
  gem 'capybara'
  gem 'selenium-webdriver'
end

# test/application_system_test_case.rb
require 'test_helper'

class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
  driven_by :selenium, using: :headless_chrome, screen_size: [1400, 1400]
end

There is a generator to create a test skeleton for you.

$ rails generate test_unit:system add_a_star_to_a_user
      create  test/system/add_a_star_to_a_user_test.rb

Here's what a freshly-generated system test looks like:

require "application_system_test_case"

class AddAStarToAUsersTest < ApplicationSystemTestCase
  # test "visiting the index" do
  #   visit add_a_star_to_a_users_url
  #
  #   assert_selector "h1", text: "AddAStarToAUser"
  # end
end

visit is capybaras method for making a HTTP request just as the browser would.

assert_selector is one of the assertions implemented by capybara. this assertion will wait for the page to load, and then it will look for a h1 tag containing the text.

Generally speaking System tests are black box tests: we only interact with the app through the web browser, and have no "inside knowledge" about the app.

In Rails system test you do have access to the full application including the database.

6.1 Some helper methods

click_link('id-of-link')
click_link('Link Text')
find('#navigation').click_link('Home')  # only look for link inside #navigation

click_button('Save')
find("#overlay").find_button('Send').click

fill_in('First Name', with: 'John')   # fill in a text input field
choose('A Radio Button')
check('A Checkbox')
uncheck('A Checkbox')
select('Option', :from => 'Select Box')
attach_file('Image',  Rails.root + 'test/fixtures/files/example.png') # file upload

all('a').each { |a| ... a[:href] ... }  # loop over all the found nodes

within("li#employee") do    # the code in the block will only use the dom inside li#employee
  fill_in 'Name', :with => 'Jimmy'
  ...
end
page.execute_script 'window.scrollBy(0,10000)' # run javascript to simulate user behaviour

6.2 Some assertions

assert_content('foo')
assert_text('bar')
assert_selector('table tr')
assert_button('save')
assert_checked_field('newsletter')
assert_link('more')

6.3 debugging

As with all tests, you can read the log file log/test.log to see how you test is doing.

System tests offer two more options:

take_screenshot  # and give it an automatic file name
save_screenshot('tmp/screenshots/book_event_step1.png')
save_and_open_page

You can save a screenshot of the (invisible) browser window at any time. If the test fails a screenshot will also be saved automatically.

And you can save the current state of the webpage as a html file and open it in the browser. The links to css and javascript will not work, but you will be able to use browser tools to inspect the DOM.

6.4 Testing Javascript

Testing with an headless browser makes it possible to test JavaScript behaviour.

Many assertions supplied by capybara wait for the page to load / the JavaScript to run before checking the assertion.

7 Further Reading

• better assertion with ramcrest
• A Guide to Testing Rails Applications - also contains an introduction to testing routes, mailers, helpers, jobs and more in depth information on testing controllers
• Fixtures API documentation