Always frustrating when an organisation knows it has got better but can’t back this up with improved performance metrics. This happens for one of three reasons:
1. Performance metrics are avoided as being too hard. The organisation may instead collect, but not use, what is most easily collectable. Often output from tools.
2. A sophisticated metrics repository is considered prerequisite to collecting performance metrics. This achieves the same result as (1), quite possibly wasting £500k on the way.
3. Performance metrics are deferred to late in the improvement journey, by which time much of the improvement has happened. The excellent Steve Haighway from BT has talked about this phenomenon.
Any project delivery organisation investing in process improvement that is NOT collecting performance metrics should rectify this before July is out! Why July? Well this is so urgent you need to start now and you can get it up and running within one month.
The following are a great starting point:
Project predictability: % variance from original scheduled delivery date and budget.
What about when the delivery date or budget change? OK. If the reason is the customer (that means the customer funds the change not just we say “it’s the customer’s fault”), then the budget and schedule are rebaselined. If the customer is not funding the change then the actual delivery date and cost at completion are compared against the original delivery date and budget.
What about organisations with internal customers who always pay for the change regardless of why it happened? Again, that’s not a barrier. Several organisations in this situation have adopted most basic rules for classifying changes as customer initiated or self-inflicted by the project delivery organisation.
Customer satisfaction: Using the “quality is meeting and exceeding customer expectations” viewpoint, this is a great complement to project predictability. Do collect the data from the real customer: the senior executive who understands the business objectives for the delivery and pays the bill. (This is not a user survey).
Projects stopped: The first two metrics are about “doing a project right”. This one gets thinking going about “doing the right projects”. Organisations with external customers should choose not to bid for uneconomic work. Organisations with internal customers should refuse to start projects if the requirements are so poorly understood that delivering something that meets customer expectation is a matter of chance. If every bid becomes a project or every project is passed through governance then there may well be a problem. Start monitoring this now and have a look.
Of course there are many more equally meaningful and perhaps more useful metrics to choose from. Remember I’m speaking with those organisations doing process improvement but not collecting performance metrics and their task is to remedy this by the end of July.
July 5, 2010 at 20:52
Collecting this is so important! So many people regret it after the fact!
July 7, 2010 at 16:07
I would be interested to learn more about your experience in quantifying customer satisfaction. You suggest that this can be done without resorting to a user survey, what other techniques have you employed and what success have you had with them?
July 21, 2010 at 15:13
“This is not a user survey” related to the person surveyed rather than the tool used. The customer perception needs to be collected from the senior executives who sponsor the project rather than the end users who are on the receiving end. Because the sponsors know the business objectives for the project i.e. they may be OK with the system not being too user friendly if it was cheap and delivered fast. Some organisations I know do also survey end users; I think that’s a “nice to have” for the organisations just starting with performance metrics.
January 4, 2011 at 11:09
[...] The busiest day of the year was July 23rd with 489 views. The most popular post that day was We know we got better but we can’t prove it. [...]