Is the system live, is it stable? What are the risk areas of the new system? When will it be working properly? Dreaded questions when managing complex systems. A multi-factor success matrix can help an organization visualize key elements of a system and make informed decisions… or perhaps just answer the question, have we gone live yet?
Successful Project Management Questions
How will we know if the project was successful is question three of four key project management questions. For those unfamiliar with this Phrankism, the other questions are:
- Why are we doing this (Project, Task, Activity)?
- How will we know when we are finished?
- How will we know if we were successful?
- Who answers the first three questions?
Multi-Factor Status
In a recent Enterprise Resource Planning project I am familiar with, declaring success was challenging. Large parts of the system worked but some critical components were running slow or failing. It was difficult to definitively answer the question: is the system live and stable and if so, can the resources assigned to the project be released to perform other activities.
In this project, no one element was causing an overall system failure but the sub-performing parts were making a successful Go-Live difficult to declare. It was easy for the executive level to get lost in the nuances of interfaces, the forest of defects and the blur between upgrade and operational issues. As a result, a multi-factor ‘score-card’ was developed so leadership could have a critical number, the following is an anonymized version:
| ACTIVITY | RATING 0-1 | Weight | Value | COMMENTS |
| STABILITY | 0.7 | 3 | 3.5 | System is responsive albeit slower than pre-upgrade, improvements expected with other activities. |
| DEFECTS | 0.3 | 1 | 0.3 | Five severity-1 and 13 severity-2; 35 total defects (> severity 2) |
| INTERFACES | 0.5 | 2 | 1.0 | Department X and Y interfaces not fully operational. |
| TUNING | 0.5 | 2 | 1.0 | Updating queries to better utilize upgraded database. |
| RESOURCES | 0.5 | 2 | 1.0 | Technical resources required; transition of functional resources. |
| 2.5 | 10 | 6.8 | 68% operational capacity |
On a Scale of 1-5…
The rubric uses a simple 0-1 rating scheme against five criteria. In the example given above, the system was rated at 2.5/5.0 or a minimally viable system.
- 0-2: Critical Failure to minimally viable system; success not possible (weighted: <2).
- 2-3: Minimally viable system, full success declaration not possible (weighted: 2 to 5).
- 3-4: Success with future remediation and work required (weighted: 5 to 8).
- 4-5: Success with a clean closure of the project (weighted: 8+).
A Weighty Decision
A weighting can also be added if the above factors are not all equally important. In this case, new defects were the least important to the organization. System stability the most important and the others falling in between.
While there is a false sense precision the more factors that are added (such as weighting), in the above example, it is worth it. The weighting could help the organization understand that while there are challenges, the system is mostly functioning.
Risk Bringing Your Own Factors
Organizations need to adjust the rubric to their own unique circumstances. Other projects may include things such as revenue generation, customer satisfaction, support for logistics, regulatory compliance, and a myriad of system specific considerations. Determining the factors in advance can help to mitigate project and system risks.
Not Earth Shattering But Potentially Useful
A weighted rubric is not new but the application to assess project success is perhaps useful. The rubric is not restricted to technology, a launch of a new product line, a fundraising campaign for a non-profit or a change to government policy may all benefit from such a success rubric.
What are your thoughts, could you use a success rubric on your next project to answer dreaded questions?
Organizational Biology 