I know this is a sticky question for any testing professional. Metrics and measures form a very important part of managing and governing projects well. This is especially true with testing projects as the need to measure and communicate the quality is paramount. Metrics provide a platform/framework for teams to be able to analyze and understand their capabilities and find ways to improve. At the same point it is important for projects to find out what should be measured, what are the metrics that make sense for my project.
One of the things we have evangelized at Mindtree is for test teams to define clearly keeping their customer in mind, what their testing objectives/scope/goals are. To help teams come up with right metrics or measures for their projects, we have launched the TMAD model, what it stands for is the “Testing Metrics Analysis and Decision” making framework.
This simple to use framework contains few excel sheets. The first sheet contains an exhaustive definition of metrics. What is interesting is that it not only contains the definition, but also –
– Goal that the metric helps to meet
– Questions one should ask if the metric is below/above the baseline
– What kind of behavioral change could it lead to among the team members
– Relevance of this metric in the type of project you are executing
Another important message that we often hear and very true is to never look at one metric as a standalone. To make sense of a metric and get the right picture, you might need to look at few other metrics. E.g. A defect trend metric might need to be correlated with a defect ageing metric to make more sense. Hence, the second sheet of the TMAD model helps the testers to see the influence of one metric on the other metric. This would give the tester a view of what metrics are related and how they need to be viewed. The last sheet of the model talks about important attributes, such as product quality, and the metrics that we need to look at to understand it.
This model has evolved over a period of time and now we are able to identify the right metrics and apply it across various projects. In my next blog, let me take you through a real case study with some practical examples.
Meanwhile, do let me know your thoughts on how you went about identifying the right metrics and measures for a given project/engagement. Challenges you faced in the process and how you have overcome them.