Black-Box vs. White-Box Testing

There are lots of methods of app testing, so I will only mention a couple of essentials: syntax testing compiler, code review performed by the development team, or tests run by the client on the ready product. Thousands of books and studies about good and bad practices in testing have been written, so one text is not enough to present this topic in a comprehensive manner. In this article, I’m going to characterize the differences between black-box and white-box testing.

Testing: then vs. now

In the course of my programming career, I’ve encountered numerous ways of code testing. The most popular methods are manual tests run by the programmers themselves, but there are also integration, unit, or functional tests.

In the first years of my programming adventure, testing generally consisted in verifying the functionality of the code by its author. If everything worked according to plan, the code would be committed to the repository. In the second iteration, before releasing the app or its new functionalities, the development team would test the app manually and, if necessary, fix the detected errors. They were just random application tests, with special emphasis placed on new functions.

Looking back, I can now see how wrong and risky that solution was. Why, though?

Correct testing of an extended project

The more complex the project is, the higher the risk of an error. When there are no automated tests, the risk grows: by adding new functionality, you might damage another. Two years ago, I was effectively convinced of the real value of automated and manual tests. I was working on a complicated API for a mobile app. The project had no tight deadline, so I decided to cover each REST API endpoint with test scenarios.

When I was building new functionalities, I wanted to cover both positive variants and all erroneous scenarios, even those that one could call corner cases, which – as I believed then – should never have occurred.

So, the app was covered by automated tests in 90%; the testing included unit tests, functional tests, and integration tests with external APIs. I considered any case that could turn up while connecting to external API, such as:

• server responds with the HTTP codes 5xx,
• server responds with the 4xx errors.

This way, I wrote various tests to check a series of typical errors that could occur.

After a few months, the project grew in scale. The addition of new functionalities or even minor code refactoring could have a significant impact on other elements of the system. The test revealed immediately if one component of the app stopped working.

You might be also interested in the article:

Tech

5 Git commands that will make your work smarter

Over the course of time, the time of delivering new functionalities became much longer, because code patches and test updates were pretty time-consuming. However, thanks to the large number of automated tests that I had written, I was quite sure that another API upgrade on the production server wasn’t going to stop various elements from functioning.

These days, all programmers can make use of a number of app testing methods. Each of them has its own uses, but today I would like to focus on black-box and white-box testing.

Black-Box testing

Why is it called black-box testing? It’s because the tester is testing a project which in a way resembles a black box on the outside – we know how it looks, but we don’t know what’s inside it.

The secret content is usually the code, architecture, or tools that have been used to implement a given app. The person using black-box testing for the app doesn’t know how the tested element has been created. The tester doesn’t need to be aware of how the solutions have been implemented. What matters for the tester is whether the functionality follows the documentation.

Black-box testing usually consists of functional tests where the tester enters the input parameters and checks if the app functions correctly and if it handles errors in the right way.

The drawback of black-box testing is that even though you can discover if a particular element functions correctly or not, you are often unable to detect what’s caused the error if the cause is hidden in the code. Because the testing person doesn’t know the implementation details, he or she may have no access to server logs. It may happen that the architecture of a fragment of a code reports erroneous actions, for example in the form of a stack trace generated to the logs.

This kind of diagnosis requires the intervention of the programmer responsible for the implementation of the functionality in question.

The role of the tester in the testing method described above boils down to the preparation of appropriate test scenarios – positive and negative, respectively for each – and of proper input parameters as well as the expected responses and behaviors of the tested system. Black-box testing is used, for instance, in tests conducted by Quality Assurance teams. They are manual tests run by specialized testers involved in the project.

Black-Box testing pros:

• the testing person doesn’t have to know the application code,
• the testing person is focused only on how the application functions,
• the testing person doesn’t need to be an expert software developer,
• they are high-level tests.

Black-Box testing cons:

• the limited chance of verifying the source of the bug,
• no opportunity to test individual code methods,
• the paths to be covered by tests are frequently restricted by the app’s GUI,
• they are high-level tests.

In the comparison above, I’ve said that the fact that black-box testing is high-level testing may be both an advantage and a disadvantage. Depending on your requirements, this can actually be both. That’s a positive aspect when the system is stable, no more errors are detected, and the tests are done before the final release, or when the flow is being tested. It can be a drawback, however, when this is the first release of a fresh functionality and many errors occur.

White-Box testing

Unlike in the case of black-box testing, where the tester has no knowledge of how a particular functionality has been implemented, in white-box testing the testing person knows its interior, or more specifically: knows and understands the code of the program.

White-box testing can be done by testers, but it’s more often done by the developers on their own. Testing takes the internal code structure into account. The strong point of this testing method is that you are able to test particular fragments of code and not full functionalities, like in black-box testing. What’s more, such tests allow you to reveal the exact place and cause of the flaw, which is very difficult to achieve in black-box testing.

Sometimes, to run the tests, you introduce a code to the tested program to test it more precisely, e.g. instructions such as assert, try/catch, log, etc. Tests of this type can be manual or automated.

What are the results of white-box testing? It often happens that, apart from the error detection as such, or a precise indication of the cause of the bug, it lets you diagnose and potentially optimize specific fragments of the app code. It’s easier to automate than black-box testing. The market offers a number of tools to automate such tests, whatever technology you use: they can be PHP, Java, C# tests.

These tests may require a huge amount of workload, though. Even more so, when it comes to code analysis and detecting the cause of the error. After reporting the error, the programmer must find the incorrect fragments of code, e.g. unhandled conditions like if, switch, etc.

White-Box testing pros:

• the possibility of testing particular fragments of the code,
• the tests make it possible to detect the cause of the error,
• easy test automation,
• it enables optimization of particular fragments of the code.

White-Box testing cons:

• it requires knowledge about the tested code itself,
• test preparation can be time-consuming.

Black-box testing vs white-box testing: summary

I hope my presentation of the characteristic features of black-box and white-box testing has been clear to you. Black-box testing can be successfully performed by people who don’t know the basics of programming or of application implementation. However, this type of testing sometimes fails to deliver all the necessary information, such as what’s caused the error in the code. In such cases, you can use white-box testing, which requires a higher level of technical knowledge and the developer’s involvement; it’s time-consuming, too. And yet, the time you spend on implementing it is not wasted. Writing automated test brings numerous benefits at later stages of development, when the app is extended and testing each single path manually would take up too much time.

This is why it makes no sense to try and state whether one of these testing methods is better or worse. To my mind, they complement each other, and the best way is to use both methods to eliminate as many errors in the app as possible. Employing both kinds of testing will certainly have a positive impact on every software product.