What Is Software Testing? | Software Testing, Part 1

What is software testing? Is it a distraction from creating new features, a drag on productivity, or maybe even an unnecessary step since it's all working fine on your local machine?

Well, imagine this-- you're an engineer who has spent years designing and building the control software for a next-generation rocket. On launch day, everything goes according to plan until, mere seconds into its flight, the rocket veers off course and explodes.

[EXPLOSION]

The cause? A simple case of integer overflow. This is exactly what happened to the maiden flight of the Ariane 5 rocket, and it is far from the only example of catastrophic software bugs-- radiation therapy machines dispensing fatal, miscalculated doses; trading algorithms losing millions of dollars due to obsolete code; even a Mars satellite lost on orbital insertion due to incorrect unit conversions. These are all real-world results caused by bugs that could have been identified through software testing.

Behind the headlines, this directly impacts your daily job. You want to write code that works and not just code that works today, but remains reliable over time. You don't want to come back and fix it later or pass those problems along to other people. Important decisions will depend on your software. If you care that it works, you need to test it.

In this video, we will describe what software testing is, what are its goals, and share an example. If you've ever written code, you've tested it, even if you didn't realize it or think of it that way. If you write a simple script, what's the first thing you do with it? Run it and see if the answer looks right. That's a test. If the answer doesn't look right, or the script errors, the test fails.

This is an example of a manual test. And for much of software history, that's all there was. However, this requires testers to-- remember to do the test, know how to perform it, and be able to assess if the result is correct. If the person who knows all that leaves the project or the company, that knowledge just left with them. Modern software testing has moved away from manual testing towards an automated, systematic process of evaluating a software application to identify bugs, errors, and missing requirements.

Instead of remembering a bunch of manual steps, we write down the expected behavior of our software and the steps to test that behavior, based on our requirements. So we can automatically compare the actual behavior with the expected behavior. When the actual and expected behavior don't match, we've got a problem. That's the core idea of software testing.

And now that it's all written down, anyone can test the software. As you're building your tests, there are several goals to keep in mind. The first goal is to find bugs early. Bugs are any time software does not perform according to its requirements. This includes-- wrong answers, missing functionality, or failure to execute tasks as expected. These are also known as functional defects.

Organizations like NASA have published studies showing the cost to fix a bug rises exponentially throughout a project's lifecycle. Bugs that make it all the way to deployment can cause financial loss, damage to a company's reputation, or worse.

A second goal is to ensure the reliability and performance of your code under all conditions. It's common to run code many times under typical conditions during the process of developing it. But can you say the same for all the edge cases? Performance issues such as slow response times or inefficient resource utilization can cause as many problems as wrong answers.

A third goal is to build user trust and satisfaction. Have you ever been afraid to touch an old piece of software out of fear you might break something? Well-tested software is more likely to be used, trusted, and recommended by users. This is especially true for safety-critical and tightly regulated industries, such as aerospace, automotive, medical, and finance.

Now that we've answered the question, what is software testing? and you have a better understanding of its goals, let's look at a simple example. We'll illustrate the example with MATLAB, but the idea is the same in any environment. We're using a type of software test called a unit test, which tests a small piece of code in isolation.

An example of a code unit is our Fibonacci function here, which generates the first end numbers of the Fibonacci sequence. If we were to manually test this, we might run it at the command line, using example inputs where we the expected output. Using the MATLAB unit testing framework, we can instead write a repeatable test that checks if the function returns the expected result by writing down the example input and output.

MATLAB can automatically generate a template for us where we just need to fill in the same example input and output as in our manual test. The template will then run the function and verify the actual and expected answers are equal. Now that our test is written down, we can easily run it. And we'll see that it fails. That's because we have a bug in our code. Did you notice that earlier when we ran our manual test? Once we fixed the bug, we can rerun our test and see that it passes.

Now that we have this test written down, we don't have to remember any of the steps or answers to test this code. As we continue to develop our project, we can grow into more advanced ways of writing tests. And we should expand on this list of tests to include all expected use cases. For example, you may have tested the Fibonacci function with positive integers. But what if a user tries a fractional or negative input? Whatever behavior you expect from your function, even if you expect it to error, you need to test that it actually happens.

Testing alone can't determine all use cases. We need to understand the application and its requirements to determine that. A best practice is to write code and tests simultaneously, rather than saving tests for the end of a project.

Hopefully, this video gave you a good understanding of software testing and its benefits. If you're ready to learn more about software testing, check out the rest of this video series. Or if you'd like to learn how to write unit tests in MATLAB, check out our "Getting Started" video linked below. Thanks for watching, and we'll see you in the next one.