An old friend told me that they did not do automatic testing at her company, or any kind of testing for that matter, because in terms of money they are better off if they do ad hoc testing and bug fixing one week prior to the delivery date. Then I got into an interesting discussion where the topic was that the customer does not pay for tests, she pays for a working software. I was really surprised at the attitude of companies towards testing. So I had to find out what experts think, and at the end I found the following quote on stackoverflow from Kent Beck (more on the topic is available on hacker news):
I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence (I suspect this level of confidence is high compared to industry standards, but that could just be hubris). If I don’t typically make a kind of mistake (like setting the wrong variables in a constructor), I don’t test for it. I do tend to make sense of test errors, so I’m extra careful when I have logic with complicated conditionals. When coding on a team, I modify my strategy to carefully test code that we, collectively, tend to get wrong.
This was almost a game changer for me, but then I remembered why I liked [automated] test cases in the first place. They give me confidence that my change does not modify existing functionality, and I don’t have to do manual tests repetitively. So I’m still good, and I learnt a new thing: when I talk to people about testing or work for somebody else, then I have to consider the time (money) factor, but in my own projects I’ll write as many automated tests as I can.
I have to admit that testing, especially using automatic test cases, costs a lot. This looks like an extra expense in short time, but saves a lot of trouble in the long run. Unfortunately, not every organisation is mature enough to realize this, or can afford to spend expensive coding time on things that have no value to the customer. Nevertheless, I know that deep down they want to do testing, so in this post I’m going to share several methods that can be used to find areas which are worth testing, so that companies do not have to spend more time on testing than what is absolutely necessary.