The purpose of validating benefits is to ensure that real value has been delivered from projects post-implementation as well as increase value from future projects when shortfalls have been identified in present projects. Although the business case promises to deliver a certain amount of value (in most cases), few companies take the time to ensure that real value gets delivered post-implementation. It requires real organizational discipline and rigor to do it well. The following three questions capture the full scope of benefits realization:
1. What benefits are we expecting in the future?
2. What benefits have we realized?
3. How do we ensure we realize full benefits on future projects?
Question one starts with the business case, specifically with cost-benefit analysis. There are no benefits to measure in the future if they don’t get captured before the project is fully initiated. Business case development is an important capability in portfolio management. Reasonably mature organizations will develop the capabilities of measuring anticipated benefits, often with the help of a senior business analyst. Unfortunately, some organizations are jaded when it comes to cost-benefit analysis and may not do it because of bad experiences in the past and the lack of accountability in the process. The purpose of tracking benefits is to help the organization ensure it gets the value it is expecting and to take action when the value is not realized.
Question two focuses on measuring the benefits after a project has closed down. The person accountable for delivering the project benefits (e.g. project sponsor) should use the same metrics documented in the business case to help track and validate the project benefits. In a mature organization, the effort to track benefits will resemble the effort at the start of the project to document expected benefits. The problem many companies face is that a gap often exists between the stated benefits and the actual benefits. This leads to question three.
Apparently, tracking benefits should be the end of the validation process. However, question three addresses the problem of gaps between anticipated and actual benefits. The scope of validating benefits is complete once gaps are addressed and resolved. This requires that the company take the time to analyze why the benefit gaps exist and to capture lessons learned that will be used by future project teams to do a better job of realizing benefits. This step completes the portfolio lifecycle of selecting the right projects, optimizing portfolio value, protecting portfolio value, and delivering portfolio value.
Virtually all of the benefit to a company comes after a project has been delivered. In order to increase the benefits realized from its projects, the organization should track the benefits that are actually received. According to Gaylord Wahl of Point B, such post-implementation evaluations can help improve business case development, which, in turn, increases the odds that the project portfolio will contain more winning projects going forward.
From a very pragmatic point of view, getting the right data at the right time is at the heart of good project portfolio management. If the right data is not available for decision makers to use, the issue will be mediocre results at best. Portfolio management is about selecting the right projects, optimizing the portfolio to deliver maximum benefit, protecting portfolio value to ensure that that value is delivered, and improving portfolio value by maturing organizational processes. At every step, data is required. The quality and quantity of data correlates to portfolio maturity. Some less mature organizations will collect insufficient data which leads to sub-optimal decisions. Other organizations may try to collect too much data before they are ready to utilize it and can do more harm than good by burning out employees with burdensome processes. Mature organizations will have the discipline and rigor to collect the right amount of quality data.
Therefore, understanding the data needed upfront is a success factor for portfolio management. There are several types of portfolio data:
- Strategic data
- Resource data
- Schedule data (forecasts and actuals)
- Performance data
- Financial data (estimates and actuals)
- Time tracking
- Request data
Senior management bears the responsibility for identifying the right data to be used in the portfolio management process. In addition, senior leadership needs to drive the accountability for collecting the right data. Without active engagement and feedback from senior management, data quality can suffer.
Organizational processes are very important for ensuring that the right data is collected. Selecting the right projects requires that good data is collected about each candidate project. Such data must be relevant to the senior management team that makes portfolio decisions. Data that is not used for decision making or information sharing is considered a waste. Collecting data comes at a cost, and organizations need to put the right processes in place in order to collect good data. From this angle, portfolio management processes are about collecting a sufficient amount of the right data. Without good standards and processes, important portfolio data will be collected inconsistently resulting in confusion and possible error.
Portfolio tools have a very important place in the portfolio management ecosystem, but only after leadership has identified what is required and lean processes have been created to facilitate data collection. Portfolio systems store and transform project and portfolio data for general consumption (aka reporting and analytics). For less mature organizations with fewer data requirements, simple portfolio systems such as Excel and Sharepoint can be used in the portfolio process. Maturing organizations should select portfolio software that meets the needs of its data requirements.
Lastly, senior leadership needs to use the data in the system for making better portfolio decisions. Strong portfolio systems will generate the reports and analytics necessary to support better investment decisions. Good data is the fuel that makes the portfolio engine run! Without good ‘fuel’, senior management will be unable to drive the organization toward its strategic goals. The data perspective of portfolio management begins and ends with senior leadership.
I am attending my first Gartner conference and have included some of the highlights from day 1 below:
It’s about risk. Don’t apply process overhead to low risk efforts.
Today’s status has something for pleasing everyone, but the full truth for no one.
Don’t ask everyone to ‘run’ when walking is ok.
Business cases may be ‘adorable’ but emotions still drive executive decision making.
Take a holistic approach–focus on outcomes first, process second.
Power PMO (Matt Light – Gartner )
Defective business cases lead to defective portfolios.
“Only when you see value are you able to tell what is waste and then start to get rid of it”
Portfolio value at the beginning of the project lifecycle is not approving wasteful projects
Portfolio value at the middle of the project lifecycle is cancelling or fixing poorly performing projects
Portfolio value at the end of the project lifecycle is reviewing the project benefits and results
PPM Maturity Workshop (Donna Fitzgerald – Gartner )
Don’t throw process at a level 1 organziation
Slow down just enough for level 2 organizations
“Real” portfolio management begins at level 3.
No individual heroic effort will get you to level 4. The enterprise must go with you.
Level 5 (if achieved) only lasts a few years and will fail after key senior leaders leave.
PPM Governance (Robert Tawry – Gartner )
Either get your process well established first OR buy a tool that is easily configurable.
You can optimize resources for speed or efficiency. If you optimize for speed, contributors should only work on 1 project. If you optimized for efficiency, you should do 3 or less projects simultaneously.
In a previous blog post, I commented on doing portfolio management ‘by hand’ to learn the processes before adopting a robust portfolio tool. In a recent discussion on LinkedIn, one consultant commented that this most successful PPM software implementations occurred when companies took a phased approach to ease in the new solution. The first phase involved simpler tools to allow the organization to become familiar with portfolio management, followed by the full implementation with the advanced PPM capabilities.
After reading this I felt that there was a lot of wisdom in such an approach. Firstly, it gives the organization time to develop their own portfolio processes without the burden of learning a new tool upfront. Secondly, it allows stakeholders (ie. Project managers, steering team, etc) to understand portfolio processes and be comfortable using them. Third, based on the early experience with project portfolio management, the organization will better understand their own requirements for a full fledged tool.
In a great article on IT Project Portfolio Management, Jonathan Feldman asks organizations to consider the problem they are trying to solve and start with high level data rather than getting too detailed. “If you know what the end goal is, you can start to quantify how close you are to that goal”. Great advice as this too points to the need for a phased approach to implementing portfolio management.
What are your lessons learned with your portfolio management implementations?
Portfolio management maturity models can be a great enabler in making process improvements. Higher maturity often translates into a greater realization of the benefits of PPM. PPM maturity models are very useful for assessing the current state of the portfolio processes and how to arrive at a higher level of maturity.
However, identifying the end state (also known as ‘future state’) portfolio processes has rarely been discussed in portfolio management literature. The common thought has been to endeavor to reach the highest level of maturity possible. Until now, there has been little acknowledgement that some organizations may not need to strive for level 5 (“optimized”) maturity (an elusive state of maturity to say the least). However, organizations can make a general determination of where their portfolio processes need to be in order to have a legitimate competitive advantage. Having this understanding gives the organization a realistic target for improvement. Such a determination can be made based on two important factors: project criticality (the importance of projects to the organization) and portfolio size (probably in dollars, but for the time being will not be quantified). In order to make a realistic assessment of future state portfolio processes, the portfolio management team first needs to make an honest assessment of project criticality (not an easy task).
The chart below is a conceptual chart depicting an organization’s portfolio maturity goal dependent on the portfolio’s criticality (along the x-axis) and the portfolio magnitude (along the y-axis). The target for many organizations would probably fall into the “Defined” stage (level 3). For some organizations with either short project lifecycles or low criticality, a level 2 (“Developing” stage) process may be sufficient. For the few organizations where projects may actually lead to increased revenue, a level 4 (“Proactive” stage) process may be necessary.
Portfolio Maturity Based on Magnitude and Criticality