In software development testing is vital. At Devoxx Belgium 2015 several experts formulated useful tips for the creation and use of such tests:
#1 Use graphs to monitor behavior
Lorin Hochstein, Senior Software Engineer at Netflix showed a graph with the typical use of the services of Netflix: most users watch videos and series in the evening, while there is less activity during the day and at night. If the graph does not follow this expected pattern on a specific day, there might be a problem within the system. Graphs might even help finding the cause of a problem. Tony Printezis, Staff Software Engineer at Twitter can, for example, detect problems with the collection of dead object by recognizing certain patterns in the behavior of a system.
#2 Define a SLA
Another way to determine whether a system is working, is to define a Service Level Agreement (SLA). Software engineer Christopher Batey states that you should create one for yourself if your business does not define one. After all, how can you check whether the connection timeout is acceptable when you have not defined any standards for this?
#3 Test regularly and thoroughly
Batey adds that you should also test for highly unlikely but possible situations that might have a high impact on your system, such as an invalid HTTP or a malformed response body. The developers should create alternative scenarios for each of those situations, so the application can fail gracefully, with minimal impact on the user. At Netflix, there might, for example, be a situation in which users on an entire continent need to be redirected to another service point. According to Hochstein, such difficult operations, which potentially have a high impact on the reliability of the system, are being tested regularly, even monthly.
#4 Test in sandboxes as well as in the production environment
Netflix uses the term ‘Chaos Engineering’ for doing continuous experiments on the production system. According to Hochstein, such tests are vital, since it is impossible to hypothesize all possible scenarios in a test environment. Batey, on the other hand, states that there are no scenario’s which cannot be tested in a test environment. However, most business will probably combine these approaches: new features will not be released before they have been tested in a sandbox, while their behavior will also be monitored in the ‘live’ environment.
#5 Use ‘Canary testing’
In order to limit the impact of live tests, Twitter uses ‘Canary testing’ when releasing new features. This means that new features will first be deployed and tested on a small part of the system. Only after a few weeks, they will be made available to the entire system. This way problems can be detected soon, without affecting the entire service.
In any case, it is a vital part of the development process to thoroughly test software. In this respect I agree with Batey, who stated: “If you don’t have a test, you don’t have a feature; it is more like a rumor: it might be there today, but it might be gone tomorrow”.