Testing is not an effort that should be done solely by a lone tester or a group of testers. It is something that should be done on different levels by the entire development team. Unfortunately, in a lot of projects, even Agile projects, the tester is (still) the last person to see the product when the developers are ‘done’. In a project we’ve been working on for the last 10 months, we try to make testing a team effort, and it seems we are succeeding.
When I joined this project, the first thing I noticed was the front-end and back-end developer were writing functional automated tests and the tester was writing code. At that point, the team consisted of a tester, an architect, two back-end developers, two front-end developers and the product owner. With a lot to test and a quickly approaching deadline, I was added as an extra tester. After working on the project for about a month I sat down with the other tester and discussed the layers of testing in the project. We discussed the layers we had covered and the layers we wanted to add.
After discussing the different layers we deemed necessary for the project, I put this in the testing pyramid, so we would have a clear vision to present to the team and stakeholders. The paper I used was squared, the layers didn’t fit inside the pyramid, and so I drew a cake instead. This cake has been on our wall since.
Our testing starts at the bottom layer of the cake. ESLint checks if our code is conform the set linting settings. During development the unit tests are created, testing the smallest portions of the code we produce. The second layer is about the components we build, which are tested separately as well. Then comes the third layer, in which we test the integration of one component with other parts of the system. When a new feature is done, the code will be peer-reviewed and altered when necessary before merging.
For our end-2-end testing, the fourth layer of the cake, we will write some automated tests that check if the system’s different components behave as expected in a specific user flow. This can be a negative flow or a positive flow and has the needed test data accompanying this. The icing on the cake, or the flame of the candle, is exploratory testing the new feature.
We will perform a security scan on a regular basis to warn us about any security issues during the development. A penetration test is part of the mandatory tests that are performed on our system due to the nature of the application.
To improve our testing, we want to add performance, load and scalability tests. Furthermore, we constantly have to keep checking and updating the tests that we already have. That means checking if we still test the right things at the right time and adding or removing tests if needed.
Like a cake, each layer of our testing should be complementary to the other layers. For example, in our component tests we check if the rendering of a component is done with the expected elements. While in our end-2-end tests we check if the expected elements on a specific page in the user flow contain the expected values based on the ‘real’ data we provide the application. In our unit tests we thoroughly test calculations made for a very important net calculator, while in our end-2-end tests we perform a login with different data and check if the expected net amount is displayed on the expected page.
Making sure these layers are complementary, means you have to discuss testing from an early point in the development process. Discuss with the developers which unit tests they will make, and give them ideas to add to these. Discuss what is already tested in a different layer to decide if an automated end-2-end test is needed or to make sure they check the right stuff. This is something that we try to do for every story but don’t manage to yet.
Icing on the cake
Of course, there are ways to improve our way of working and layers we still have to add to get more value in testing. Like mentioned earlier, we want to add some tests to check if our application can handle a specific load. KPI’s describe our required performance, which will have to be checked to make sure we can handle that performance. So far, these tests weren’t a priority because of the cloud architecture of the application. This architecture does make it necessary to think about scalability and test that as well.
As explained, it is needed to discuss what to test for every story and we don’t always do that yet. We need to be more active in discussing the tests we will write for each layer of a feature. When that really becomes part of our routine, we will be able to add even more value to our customer, the icing on the cake.