Metrics in Performance Testing

A question very often asked by SQAs to the performance testing team is - what are the metrics collected for Performance Testing. How many times are we tired of answering - it's difficult to define the metrics exclusively for Performance Testing.

Let's make an effort in identifying the metrics for performance testing. Apart from the general "Effort & Schedule Variation", is there any useful metrics that could be collated exclusively for performance testing?

It's worth mentioning the quote I heard from one of the recent training sessions on Statistical Process Control - 'Don't collect metrics just for the sake of measurement, do it only if you think that that measurement will be helpful to you in some way in improvisation'. How true.

Bye for now.

Perf. Testing Metrics... It’s a subjective & relative term that cannot be defined unless & otherwise the nature of the project & type of the project is known. What appears to be critical metrics for one project is less important metrics for other.

To the max metrics that are of directional nature may bring in more benefit rather than the typical test metrics.

Some of them I could think are:

Number of iterations executed till the system achieved a 50% & 100% of the target

- This may indicate about the efforts (unplanned)
- Stability of the product & time taken for it
- Improvement brought in by the tuning & rework on coding

Cheers,
M;~}

I partly agree with you. However the term metric necessarily means that it is not project specific ( if any process/quality consultant are listening, kindly correct me).

I believe in your reply you meant the 'number of runs', as iterations in performance testing context would mean something different, especially for the LoadRunner lovers. That surely could be a metric 'number of runs/times the same test was executed to achieve the performance goal' (Traditionally we plan just 3 runs during our project planning stage).

Let's try and collate the list.

How about 'number of defects identified'. In Performance testing context, each performance bottleneck identified could be treated as 'defects'. Isn't it?

There could also be a 'defects vs solution ratio' as well. i.e. number of performance bottlenecks identified vs number of recommendations made. This could be a key metric to decide the overall efficiency of the performance testing team. What do you say?