Software Testing
by Nicolas McCurdy
What is software testing?
- The process of verifying software to ensure it works as expected
-
Can be manual or automated
- Whenever it is possible, automated testing is prefered because of its repeatability and ease of use.
- In more complex scenarios, some things may only be easy to test manually.
Why should I test?
- To prove that your code satisfies requirements
- To catch bugs faster, before software is out in the field
- To have an unambiguous way of determining if a unit of code still works after it is modified (regression testing)
-
To add flexibility and efficiency to your project's development
- Continuous integration is a great example.
Testing Methods
- There are many testing methods that define what aspects of a system should be tested and how.
- It is common for specific software projects to use multiple testing methods to cover different kinds of defects (bugs, performance issues, security vulnerabilities, etc.).
- Many test frameworks and other testing tools are available for most popular testing methods.
Unit Testing
- The process of testing individual units of code separately from each other
- By running unit tests, it is easier to isolate what units of code are to blame for bugs.
- Unit tests are considered "white box" tests because they involve looking at the internals of an application's code and runtime.
What do we need to know to write a unit test?
- The unit we are testing (usually a function or method)
- The test cases we will create to prove the unit functions as expected
-
Inputs
- Equivalance classes: groups of inputs that should result in similar behaviors
- Boundary cases: values on the edges of these classes
- Expected outputs and side effects
Unit Testing Tips
- Try to test with as many equivalance classes of inputs as possible.
- Start by focusing on tests for individual methods and functions.
- Share common setup between unit tests.
Test Case Example
What should our test cases be?
def palindrome?(word)
word == word.reverse
end
Test cases
- When the word is empty, it should be a palindrome
- When the word has one character, it should be a palindrome
- When the word has multiple characters, we should ensure the right words are marked as palindromes
- When the word is null, it should not be a palindrome
Unit Testing with Minitest
require 'minitest/autorun'
class TestPalindrome < Minitest::Test
def test_palindrome
assert_true palindrome? ''
assert_true palindrome? 'c'
assert_false palindrome? 'car'
assert_true palindrome? 'racecar'
assert_false palindrome? nil
end
end
Minitest Results
$ ruby test_palindrome.rb
Run options: --seed 65310
# Running:
E
Finished in 0.001653s, 604.7796 runs/s, 2419.1183 assertions/s.
1) Error:
TestPalindrome#test_palindrome:
NoMethodError: undefined method `reverse' for nil:NilClass
/Users/nick/testing-talk/palindrome.rb:2:in `palindrome?'
test_palindrome.rb:10:in `test_palindrome'
1 runs, 4 assertions, 0 failures, 1 errors, 0 skips
Unit Testing with RSpec
describe 'palindrome?' do
context 'given an empty string' do
it 'returns true' do
expect(palindrome? '').to be true
end
end
context 'given a string with only one character' do
it 'returns true' do
expect(palindrome? 'c').to be true
end
end
context 'given a longer non-palindrome' do
it 'returns true' do
expect(palindrome? 'car').to be false
end
end
context 'given a longer palindrome' do
it 'returns true' do
expect(palindrome? 'racecar').to be true
end
end
context 'given a null value' do
it 'returns false' do
expect(palindrome? nil).to be false
end
end
end
Integration Testing
- Multiple software modules are tested together to verify their communications with each other.
- Spies can be used to wrap interfaces and inspect what they are called with.
- Integration tests do not need to know the internals of the individual modules.
Acceptance Testing
- The process of testing a program's features to ensure that they meet customer requirements
- Acceptance tests give us a way to prove that software meets needs and works from a user's perspective.
- Acceptance tests are considered "black box" tests because they simulate user actions without requiring access to an application's internals (however, they can still be automated).
Smoke Testing
- The process of performing quick checks to see if there are any basic problems with the code that would prevent it from running.
- Smoke testing is generally not enough for testing an entire application, though it can be useful at earlier phases of the testing process.
Examples
- Check if there are any compiler errors
- Check if the application launches
- Perform a request on an important API endpoint and assert that it returns a 200 (success)
Mock Objects
"Fake" objects that replace the real implementations of code
- Very useful for dealing with slow, unreliable, and hard to test code that you aren't testing directly.
- Can make tests more flexible, but can also increase the amount of complexity in test setup.
- Especially useful for services and HTTP requests.
- Example: Tests for a weather application that relies on a third party API.
Testing and Process
- Depending on a project's process model and personal development practices, testing may happen after, during, or even before development.
- Waiting to test until after all development is done (waterfall) can cause difficulties with integrating work and verifying if systems themselves or their dependencies are broken.
Test Driven Development
The process of writing test code before implementation code.
Plan what you are going to implement.
Write tests (usually unit tests or acceptance tests) for what you will implement.
Run the tests (before implementing).
Write the implementation.
Run tests and ensure that the implementation's tests pass.
Rinse and repeat until all tests are passing.
Behavioral Driven Development
Similar to test driven development, but involving more process and abstraction. Tests (usually acceptance tests) are written to verify specific user stories as they are implemented.
BDD Tools
Test frameworks that focus on heirarchal specifications (like RSpec) or executable user stories (like Cucumber) are designed for and frequently used for BDD.
Software Testing
by Nicolas McCurdy