Prioritize Projects With A Scoring Model

October 11, 2018May 27, 2020 Tim Washington 1 Comment

EVALUATE YOUR PROJECTS

In the previous post, we covered the detailed steps for building a prioritization scoring model. In this post we will cover the steps for how to prioritize projects with a scoring model.

Step 4 – prioritize projects using scoring values

The last part of the scoring model is to determine the scoring values. For each criterion in the scoring model, there needs to be some evaluation of a low, medium, or high score to drive a numerical score for that criterion. In practice we can expand this to “none”, “low”, “medium”, and “high” to give the decision makers a slightly wider range of options. We do not want too many options as this can slow down the scoring process, but we want enough options to help distinguish project evaluations. A common scoring paradigm would include 0 for none, 1 for low, 2 for medium, and 4 for high. If companies want to put even more emphasis on high value, they could use a 0, 1, 3, 9 scoring paradigm.

In the example below, each of the four qualitative values has a corresponding quantitative value. For each value, there is a definition to help determine to which value the project best aligns. If the governance team determined that a particular project has a medium alignment to strategic objective #1, it would score a 2. This is an important step for decision makers to prioritize projects and we will explore all of the calculations at the end of this post.

Prioritize Projects Using Anchor Scores for Strategic Criteria

The definitions alone can significantly improve the quality of the project scores. The example below is relatively generic and requires the governance team to possess enough knowledge about the project and the strategic objective to properly determine the degree of strategic alignment. This can be problematic if the governance team “feels” that a given project has moderate to high alignment and therefore warrants a higher score. The scoring process is intended to make the prioritization process as objective as possible. This can be accomplished by more specifically defining the underlying criteria of “none”, “low”, “medium”, and “high”. In example #2, an IT department of a large Fortune 500 firm has a specific strategic objective to reduce the number of legacy computing systems. The criteria for each of the four options is far clearer with little need for interpretation. If the project actually decommissions a system, it scores “high”. As companies mature their evaluation process, the specific criteria for none through high can be enhanced.

Prioritize Projects - Detailed Strategic Scoring — Prioritize Projects – Detailed Strategic Scoring

For financial evaluations, it is important to set financial thresholds that will really set the winners apart. If the bar is set too low, too many projects will get “high” scores and the scoring model won’t be of much help to distinguish the highest value work.

For the riskiness evaluations, we need to flip the quantitative values so that the highest number corresponds to the lowest risk qualitative value. By doing so, we are giving more value to less risky projects. This makes sense especially if we have two projects which may be nearly identical in value but one is far riskier than the other; under normal circumstances if we could only choose one over the other we should choose the less risky option (i.e. better risk-adjusted value).

Scoring Model - Risk Scores — Scoring Model – Risk Scores

For a free copy of these slides, sign up now.

Step 5 – collect all necessary information in order to prioritize projects

In order to evaluate projects or new proposals, some amount of information is needed to understand the scope, importance, alignment, cost, benefit, and risks of the project. Based on the actual scoring model you built and your current organizational processes, you may already have all the information you need to use the scoring model. In other cases, new information needs to be collected. Let’s break it out in further detail.

Strategic information: at some point during the intake/proposal phase, the project initiator for each proposal should have provided some rationale for how the proposed project aligns with one or more strategic objectives. Simply stating that alignment exists is not useful; there should be an explanation of how the project supports the organization’s strategic objectives. This information should be captured in a document for each governance team member to review. If this information does not exist (or does not exist for all current projects), the Project Sponsor can help explain strategic alignment when the governance team evaluates the projects. If most current projects are missing this information, the Project Management Office (or other team facilitating the prioritization process) should update organizational processes to include this information going forward.

Financial information: assuming that your scoring model utilizes quantitative financial information, it is very important to ensure that this information is available for all projects (a common challenge for many organizations). The best approach is to work with the Finance department to do some level of financial analysis to determine financial benefits for each project. This will lead to consistent financial information across the portfolio. Without Finance’s help, most organizations struggle to get meaningful financial benefits. The best alternative to getting Finance’s help is to provide spreadsheet templates to each project team to assist with calculating the financial metrics used in the scoring model.

Riskiness information: information about the inherent riskiness of the project is useful for conducting this evaluation. It is important to keep in mind that this assessment is not about the specific project risks (although this could be useful), but about the inherent risk nature of the project. Answering questions such as: “how much organizational disruption will this project cause?” “How much experience do we have with this type of project?” “Do we have a sufficient skill set internally to deliver this project?” are important to evaluate the riskiness of the project. Ideally, some of this information is contained in the business case, project proposal form, or even in a project charter.

Step 6 – Educate stakeholders on how to prioritize projects

Everyone involved with evaluating and scoring projects must understand the relevant details of all projects in the portfolio in order to successfully prioritize projects. If your scoring model includes a strategic component, financial component, and riskiness component, we recommend the following roles for scoring projects:

Strategy: The Governance Team, representing the senior leaders of the organization is best equipped to evaluate the strategic value of each project
Financials: this is easily derived from the financial analysis and no additional evaluation is needed. A Portfolio Analyst or PMO Administrator should be available to compile this information and capture it in a single data repository.
Riskiness: The Project Manager or PMO Director will be in the best position to evaluate the riskiness of a project. The PMO Director could be the singular person to evaluate riskiness across all projects in the portfolio, which streamlines this evaluation. Alternatively, each Project Manager could evaluate and score individual and share the results with the Portfolio Analyst or PMO Administrator. A group of Project Managers could come together and work together to score their projects as individual Project Managers may interpret criteria a little differently.

In most cases, this requires a certain level of education so each project is fairly evaluated. This is not a small effort. Although the Governance Team may be sponsoring some of the current projects in the portfolio, they won’t be familiar with all of them. That will affect the team’s ability to evaluate and score the projects. Therefore, some level of education is needed to bring everyone up to speed. There are a few ways to accomplish this:

Prepare summary information of each project for the Governance Team to read on their own before the meeting (examples include: business case, charter, etc.). The advantage of this approach is to expedite the scoring exercise.
Prepare summary information of each project for the Governance Team to review during the meeting. Using this approach, the Governance Team would likely review a batch of projects and score them in the same session. The advantage of using this approach is that everyone will have dedicated time to review project information and discuss as a group. This option also takes longer than option 1.
Conduct mini project reviews in a team meeting with the Project Managers in attendance to present and answer questions. The advantage of this approach is that it gives the Governance Team an opportunity to ask important follow-up questions as well as get to know the Project Managers better. This option takes longer than the previous two options.

Step 7 – Evaluate and Score

Tips for handling the evaluation and scoring:

Pilot the scoring with a handful of projects: pick a few representative projects (projects known to be high priority and lower priority, across various strategic objectives)
Conduct the pilot with the governance team in person: even after prioritizing criteria and discussing it as a team, there will still be questions about specific projects and how they align to the strategic objectives.
Build in adequate time for discussion: there will be different degrees of understanding of the projects and some projects will require more discussion than others. The value is in the discussion, so build in adequate time for discussion so participants do not feel rushed.
After the pilot, get agreement on how the governance team wants to finish scoring projects. Some teams will want to continue scoring as a group; others may want to do their scoring in advance and come prepared to discuss.
Take time to review and validate consistency of scores: some governance team members may be surprised about the scores of certain projects. It is not unusual for some medium and high scoring to be inconsistently applied. The group may have determined that a project was a “high” in one case but after review, it should really be a “medium” or vice versa.

In the next post we will cover the relationship between priorities and resources and the need to reinforce over-communicating priorities and how this affects resource allocation.

A Guide to Building a Project Prioritization Scoring Model

September 4, 2018May 27, 2020 Tim Washington 9 Comments

PRIORITIZATION IN THE CONTEXT OF PROJECT PORTFOLIO MANAGEMENT

Portfolio management is about maximizing organizational value delivery through programs and projects. In order to maximize value delivery, the governance teams that approve work and prioritize projects need to share a common view of “value” in order to use a scoring model to select the most valuable work and assign the right resources to that work. Understanding the relative “value” of each program and project in the portfolio is at the heart of portfolio management and determines what work is selected, how it is prioritized, where resources are allocated, etc. In order to select a winning portfolio, every governance team needs to share a common understanding of value; without it, you’ll fail to realize the benefits of your portfolio.

However, the definition of “value” will differ at every company because every company has different strategic goals, places varying emphasis on financial metrics, and has different levels of risk tolerance. Furthermore, even within a company, each department may interpret the strategic goals uniquely for their organization. Hence, “value” is not clear cut or simple to define. Any organization that manages a portfolio of projects needs to define and communicate what kinds of project work is of highest value.

The Scoring Model is the Prioritization Tool

The tool for assessing project value is a scoring model, which includes the criteria in the model, the weight (importance) of each criterion, and a way of assessing a low, medium, or high score for each criterion in the model. A good scoring model will align the governance team on the highest value work and measure the risk and value of the portfolio. A poor scoring model will not adequately differentiate projects and can give the governance team a false precision in measuring project value.

In the context of the portfolio lifecycle, assessing project value is particularly important in the first two phases: Define Portfolio Value and Optimize the Portfolio. When evaluating new projects for inclusion in the portfolio, the governance team must understand the relative value of the proposed project in relation to the rest of the projects in the portfolio; this will help inform the governance team’s decision to approve, deny, or postpone the project. Once there is an established portfolio, the same value scores can be used to prioritize work within the portfolio. For many organizations, the process of selecting projects and prioritizing projects is merged together to develop a rank order list of projects where the governance team “draws the line” where budget or resources run out is an acceptable way to define the portfolio. Unfortunately, this approach does not result in an optimal portfolio, but is acceptable for lower maturity organizations. Strictly speaking, we should distinguish portfolio selection from project prioritization and for the purposes of this post we will focus strictly on using the scoring model for prioritization.

Gaylord Wahl of Point B Consulting says that priorities create a ‘true north’ which establishes a common understanding of what is important. Without a clear and shared picture of what matters most, lower-value projects can move forward at the expense of high-value projects. Even though experienced leaders understand the need to focus on a select group of projects, in practice it becomes very difficult. Good companies violate the principal of focus ALL THE TIME and frequently try to squeeze in “just one more project.”

Prioritization is About Focus

Prioritization is about focus—where to assign resources and when to start the work. It enables the governance team to navigate critical resource constraints and make the best use of company resources. Higher priority projects need the best resources available to complete the work on time and on quality. Resources that work on multiple projects need to understand where to focus their time. When competing demands require individuals to make choices about where to spend their time, the relative priorities need to be obvious so that high-value work is not slowed down due to resources working on lower-value work. You have to be sure that your most important people are working on the most important projects so that you can get the most important work done within existing capacity constraints.

Furthermore, when resources are not available to staff all of the approved projects, lower priority projects should be started later once enough resources are freed up to begin the work. However, not all projects can be initiated immediately. Understanding relative priorities can help direct the timing and sequencing of projects. In some cases, high priority projects may have other dependencies or resource constraints that require a start date in the future. In other cases, lower priority projects get pushed out into the future. In both cases, schedule priority helps answer the question “when can we start project work?” Remember, prioritization is about focus—WHERE to assign resources and WHEN to start the work.

BUILDING THE SCORING MODEL

Assessing project value is at the heart of portfolio management, and the scoring model is the tool to help assess value. Therefore, building a good scoring model is integral to prioritizing work. In fact, as we will see, prioritizing the criteria in the scoring model is a major component of the prioritization exercise.

Step 1 – Define the Scoring Criteria

The first step in building the scoring model is to identify and define the criteria in the model. In the past, expected financial benefits would be a singular way of measuring project value. Although this is a tangible and quantitative way to measure value, experience shows that merely selecting and prioritizing work based on financial benefits fails to yield optimal strategic results. In high performing organizations, value can include intangible (qualitative) factors such as the degree of strategic accomplishment, customer impact, and organizational benefits. Therefore, we recommend a combination of quantitative and qualitative criteria. At the very least, your scoring model should include three categories of criteria: strategic alignment, financial benefit, and risk. This is your highest tier of scoring criteria (what we will refer to as “tier 1” criteria). Within each of these categories are the sub-criteria that will actually be used to evaluate projects (referred to as “tier 2” criteria). Why are two tiers of criteria needed? Let’s look at the strategic category. Some organizations simply want to evaluate the degree of strategic alignment, but since nearly all organizations have two or more strategic objectives, for the purposes of assessing project value, you should evaluate the degree of alignment across all of your strategic objectives. Projects that positively impact multiple strategic objectives are generally more valuable. In addition, having a discrete understanding of which strategic objectives each project supports will further enable the governance team to prioritize work. Each strategic objective is a sub-criterion (tier 2) used to assess project value. The three recommended scoring model categories are defined below.

Strategic Criteria: Portfolio management is focused on strategic execution, so measuring strategic alignment as part of your scoring model is important. This would include your organization’s strategic objectives.

Financial Criteria: All for-profit companies should incorporate financial benefit into the scoring model such as net present value (NPV), return on investment (ROI), payback, earnings before interest and taxes (EBIT), etc.

Risk Criteria: Finally, a good scoring model takes into account the risk factors of the project. These are not individual projects risks, but a measure of the “riskiness” of the project. Just like a stock portfolio, each investment carries a different level of risk. Remember, if you could only choose one of two investments that each have the same return, you will always go with the least risky option.

Step 2 – Prioritize the Criteria

The second step is to prioritize the criteria from step 1. This is best done using pair-wise evaluations, a simple method of comparing two criteria against each other (also known as the Analytic Hierarchy Process, or AHP). This is easily accomplished in one on one sessions with each decision maker. This evaluation is perhaps the most important step in the entire process because it will not only determine the weighting of your scoring model, but even more it will test and ultimately align the governance team’s understanding of the organizational strategies. For example, when evaluating strategic criteria, the governance team will be asked to compare strategic objective 1 against strategy objective 2. On paper, everyone probably understands why each strategic objective is important but has probably not considered the relative importance of one strategic objective compared to another. Using AHP will force each person to really consider whether the two strategic objectives are equally important or whether one is truly more important than another (when making comparisons using AHP, a criterion can be equal to, twice as important, three times as important, four times as important, etc. to the other criterion). In the example below, Decision Maker #1 believes that strategic objective #1 is three times more important than strategic objective #2.

Step 3 – Review and Validate

The third step is to review the evaluations as a team. This exercise really highlights what is most important to each member of the governance team and affords a way for the governance team to have a common understanding of value. It is necessary to highlight where the biggest gaps are between the members of the governance team and discuss why each person holds their view. In this discussion, no one’s evaluation is right or wrong, but in the discussions as a governance team new information may come to light that helps everyone align to a common understanding of each strategic objective, financial criteria, and risk criteria. Based on experience, the strategic alignment discussion is of critical importance and can surface divergent views that would never have come to light without the pair-wise exercise. Without going through this exercise, it is impossible to determine if the scoring model truly represents the governance team’s understanding of value.

An example of a single comparison is shown below. In this example Decision Makers 1 and 2 believe that the first strategic objective is three times as important as the second objective. However, Decision Makers 3 and 4 believe that the second objective is four times and two times as important respectively. This governance team needs to come back together to discuss the discrepancies. The true benefit of this exercise is in the discussion that helps align the team to a common view of strategic value.

WARNING: It is tempting to skip these steps by arbitrarily picking scoring weights in order to quickly score and prioritize projects. One large company wasted hours in weekly steering committee meeting debating the weighting of each criterion. In the end, the excessive discussion wore down the committee; it did not produce the right discussion. Rather, focus on the relative value of each criterion compared to other criteria; the weighting will be a mathematical output of the pair-wise comparisons. Additionally, if the governance team does not share a common understanding of value, the benefits of going through any prioritization exercise are greatly diminished and can cause more churn in the long-run. Based on experience with Fortune 500 companies, the pair-wise discussions are not only more effective but also more efficient (a single person can complete their evaluations in 15 minutes). The benefit is in the discussion among the governance team members to align on the criteria for evaluating project value.

By this point, the result of prioritizing the criteria is:

A governance team that has been calibrated around how to define value
A scoring model with criteria and weighting that has been validated by the governance team and can be accurately used to assess project value.

The simple example below shows what the weighting could look like after prioritizing the criteria.

In this post we have covered how to purposefully build a scoring model. The next post will cover the remaining steps in order to evaluate and score projects.

What is a prioritization scoring model?

The scoring model is a tool for evaluating project value. It is composed of multiple criteria to assess project value. The final score represents the numeric value of the project and can be used to compare against other projects within the portfolio.

Portfolio Optimization—Data and Constraints

September 27, 2016May 27, 2020 Tim Washington 1 Comment

In our hyper-accelerated business world, data analysis and data visualization are exceptionally important. In the realm of project portfolio management (PPM) and PMO’s, organizations need robust data analysis to strengthen decision making and improve strategic execution. The key is to have the right processes in place to collect the right data and ensure that the data is of good quality. As I have said before, data collection is not free; any data that is collected but not actively used is a waste of organizational resources. Knowing what information is needed to drive better decision making will help ensure that only important data is collected. Therefore, organizations should wisely consider what metrics, analytics, and reports are most important to senior leaders and then develop or improve the processes that drive the collection of that data. The power of having good portfolio data is to conduct strong portfolio optimization.

3 LEVELS OF ANALYSIS

Once organizations have a stable foundation for PMO/PPM data collection, they can embark on the data analysis journey. The graphic below highlights three levels of data analysis:

Descriptive analysis—this helps answer the basic “what has happened?” This level of analysis is the most basic as it is fact-based and is required for developing key performance indicators and dashboards.
Predictive analysis—this helps answer a more important question, “what will happen?” With sufficient data, organizations can begin to predict outcomes, especially related to project risk and project performance and the impact to project delivery as well as the portfolio as a whole.
Prescriptive analysis—this helps answer a more difficult question “what should we do?” This requires more detailed and advanced analysis to determine the optimal path against a set of potential choices. Prescriptive analysis of the portfolio provides significant benefits by enabling organizations to choose the highest value portfolio and choose a group of projects with a higher likelihood of success.

PORTFOLIO OPTIMIZATION

Portfolio optimization is major part of the prescriptive analysis described above. Organizations should endeavor to get to this point because it delivers substantial value and significantly improve strategic execution. In order to optimize any part of the portfolio, organizations must understand the constraints that exist (e.g. budgetary, resource availability, etc.). These constraints are the limiting factors that enable optimal scenarios to be produced. There are four basic types of portfolio optimization described below:

Cost-Value Optimization: this is the most popular type of portfolio optimization and utilizes efficient frontier analysis. The basic constraint of cost-value optimization is the portfolio budget.
Resource Optimization: this is another popular way of optimizing the portfolio, and utilizes capacity management analysis. The basic constraint of resource optimization is human resource availability.
Schedule Optimization: this type of optimization is associated with project sequencing, which relates to project interdependencies. The basic constraints of schedule optimization are project timing and project dependencies.
Work Type Optimization: this is a lesser known way of optimizing the portfolio, but corresponds to a more common term, portfolio balancing. The basic constraints of work-type optimization are categorical designations.

APPROACH

The following diagram summarizes the above points and highlights how having the right data inputs combined with constraints and other strategic criteria can produce optimial outputs across four dimensions of portfolio optimization.

Point B Consulting’s 5-step methodology for conducting PMO analytics enables organizations to realize the full potential of their analytic processes.

Define: Determine the performance criteria for measuring PMO/PPM success and develop a set of questions / hypotheses for further modeling and investigation
Transform: Gather and transform all available resource, project, business data for further visualization and analysis
Visualize: Inventory all projects with related resources and highlight key trends/insights based on project and business data
Evaluate: Develop analytic framework to test, adjust, and optimize against tradeoffs between project sequencing, resource allocation, and portfolio value
Recommend: Develop a final set of project prioritization recommendations for desired future state

In summary, portfolio optimization delivers significant strategic benefits to any organization, but getting the right processes in place to collect good data is not easy. Having the right data can enable your organization to know what is happening to the portfolio (descriptive analysis), what could happen (predictive analysis), and what senior leaders should do (prescriptive analysis).

Improve Portfolio Health By Avoiding Two Portfolio Management Extremes

March 30, 2016November 19, 2018 Tim Washington Leave a comment

Two Simple Questions

You can measure your general portfolio health with two simple questions:

1) Do you approve all or almost all of your projects?

2) Are you approving so few projects that people would say you are “cutting to the bone”?

These are two portfolio management extremes that we will examine in this post.

Approving Everything is Bad

Question number one highlights a common trap for many companies, approving all or almost all projects that get reviewed. This indicates that the project selection process is not working well. When governance councils have a project approval over 90%, it means very few projects are getting screened out and some poor projects are probably getting approved. Approving nearly all projects also means that significant diminishing returns kick in for this group of projects and executing this work likely requires unnecessary multi-tasking and exceeding the resource capacity of critical resources. While it is theoretically possible for an organization to do an outstanding job of selecting the best possible project candidates upfront and still have a high approval rate, I doubt this occurs very often. More likely, organizations operate in a reactive mode and approve projects as they get proposed; since most projects look good by themselves and almost always have a good reason for getting initiated, the project gets approved and funded. Therefore, one of the best portfolio governance council metrics to measure portfolio health is the project approval rate. We can illustrate these concepts with the graphic below.

Portfolio Cumulative Frontier - Extreme 1 — **Portfolio Cumulative Frontier – Extreme 1**

Here we have a bounded curve of possible portfolios (in this case we can apply the cumulative frontier, which is the cumulative portfolio value based on the rank order of projects in the portfolio, not to be confused with the efficient frontier which is based on portfolio optimization). At the upper far right is the problem area in question. If organizations are approving most projects it means there is little to no discrimination among projects which is a symptom of not having enough project candidates to review and stems from poor ideation, work intake, and weak phase-gate processes.. When organizations have more project candidates than they can reasonably take on, the governance council is pushed to do a better job of selecting projects. Organizations can still do a poor job of selecting projects (or may simply ignore resource capacity and continue approving everything) even when they have more than they can take on, but the emphasis here is on increasing the project pipeline so that the governance council will become less reactive and more proactive and say no to projects that really should be screened out. Creating a strategic roadmap to identify important projects (top-down approach) combined with an employee ideation (process bottom-up approach) will help build up the pipeline of projects and increase the decision making rigor by the governance council.

Don’t Cut to the Bone

We can also evaluate portfolio health by looking at the other extreme where an organization is cutting costs so much that any further cuts will hurt the organization’s day to day operations (aka “cut to the bone”). In one place I worked, the cost-cutting measures had been in place for years and a number of good project candidates were hardly under consideration because funds simply were not available and a buildup of project requests was accumulating. A few high value projects got approved, but “money” was left on the table as a result of not taking action on those good project candidates. In some cases, the rigor to do a good cost-benefit analysis is absent and makes it difficult to communicate how much ‘value’ is being ignored by not taking on additional projects due to strong cost cutting measures. Such extreme cost cutting also has the negative residual effect of discouraging innovation among employees. We can also illustrate this with the same graphic.

Portfolio Cumulative Frontier - Extreme 2 — **Portfolio Cumulative Frontier – Extreme 2**

Summary

In short, asking simple questions about the approval rate of projects and the cost-cutting measures of an organization can highlight general portfolio health. In both cases, organizations should be pushing toward the middle. Adding more project candidates will help ensure that only the most valuable projects get approved. In the case of extreme cost-cutting, companies should improve their ability to measure project value in order to communicate the ‘value’ left on the table. This is best accomplished when a company is doing reasonably well and not when the company is truly in dire straits. Cutting costs “to the bone” is never a good way to stimulate innovation, therefore careful attention is needed when companies are cutting costs too much and not investing in the future.

Portfolio Review Meetings

November 24, 2014May 27, 2020 Tim Washington 3 Comments

Portfolio Review Meetings

Portfolio review meetings are a great way to review and assess the entire project portfolio with the governance team. Unfortunately in practice, these meetings can be overwhelming, time consuming, and unproductive. There are many ways to conduct a portfolio review meeting, but one of the key questions of the governance team is “what do they want to accomplish at the end of the portfolio review”? For some organizations, portfolio review meetings are about getting project status of every project in the portfolio. For other organizations, portfolio review meetings are designed to evaluate each project in the portfolio with the intention of updating priorities.

Options for Portfolio Review Meetings

With this background in mind, we can look at four options for conducting portfolio review meetings:

OPTION 1: A review of all in-flight projects, current status, relative priority, business value, etc. Some projects may be cancelled, but the primary purpose is to inform the LT of the current in-flight projects.
OPTION 2: A partial review of projects in the portfolio consisting of high-value/high-risk projects. This provides more in-depth information of critical initiatives and may result in a possible change of priority of certain projects.
OPTION 3: A high-level review of all projects in the portfolio with the intention of updating project priorities for every project in the portfolio.
OPTION 4: A review of portfolio scenarios that meet current business needs followed by a selection of a recommended portfolio

Option 4 comes courtesy of Jac Gourden of FLIGHTMAP in a 2012 blog post and is the best approach I have seen for conducting portfolio review meetings. I also have sat through long sessions (although not all-day sessions) of reviewing all the projects in the portfolio and it can be painstakingly tiring. Moreover, these types of portfolio review meetings wear out governance team members and do not yield much value. While there is certainly a time and a place for review the status of all projects or conducting a lengthy review for the purpose of re-prioritizing projects in the portfolio, taking a strategic view is the way to go. Rather than merely focusing on individual projects, a portfolio team can compile a few portfolio scenarios that should be reviewed by the governance team. In many instances, there is significant overlap between the portfolio scenarios, but the emphasis is on the business goals of the portfolio and how a portfolio scenario supports a certain goal. Some examples of portfolio scenarios include:

Revenue Growth Scenario
Customer Growth Scenario
Market Growth Scenario
Reduced R&D Spend Scenario
Balanced Portfolio Scenario

These scenarios are easier to produce when efficient frontier analysis is applied. Even after a portfolio recommendation is accepted, there is further work to screen out the projects not included in the portfolio, and in some cases to make worthy exceptions for some projects that would have otherwise been removed from the portfolio.

What do you think? Have you tried this approach before? How successful was it? Let me know.

Five Uses of a Prioritization Scoring Model

May 24, 2014October 11, 2018 Tim Washington 1 Comment

Project prioritization is one of the most common topics in portfolio management literature. Within the context of project prioritization is the matter of scoring models because scoring models are the most widely used approach to prioritize projects. Although there are a lot of opinions on the effectiveness of common scoring models, they are nonetheless the most common method for prioritizing projects. However, most people may not realize the many uses of a scoring model and how it drives better decision making beyond project prioritization. In this post, we will look at five uses of a scoring model.

1) Project prioritization is the most common reason for using scoring models. As we saw in a previous post, project prioritization is for resource allocation. Since portfolio management is about delivering the most business value through projects, it is logical to ensure that resources are spent on the most important work. Ranking projects helps provide a common understanding of what is most important in the organization and scoring models are one of the easiest ways of establishing a rank order. For more information on using prioritization scoring models to rank order projects, please see Mastering Project Portfolio Management.

2) A prioritization scoring model is also used for project selection. The idea is to rank projects from highest value to lowest value and select projects until resources run out. This approach has merits over other approaches that do not sufficiently take account of strategic drivers. However, it can be shown that even simple optimization techniques can yield a higher value portfolio at the same cost. For organizations that do not employ portfolio optimization techniques, using a scoring model to rank order projects and fund projects until resources runs out is a reasonable way to go.

3) Portfolio optimization is very useful for identifying higher-value portfolios than merely using scoring models as discussed in the previous paragraph. The scores for each project can represent a “utility score” which can then be used as the input for the optimization calculations. In this way, projects are optimized based on all the scoring inputs, not merely on net present value or some other financial estimate. For more information about this technique, please refer to Richard Bayney’s book Enterprise Project Portfolio Management: Building Competencies for R&D and IT Investment Success.

4) A prioritization scoring model can also be used to make go/no-go decisions at gate review meetings. There are at least two ways to accomplish this:

A) Organizations can predetermine a threshold score that projects must exceed in order to be considered for inclusion in the portfolio (known as a scoring hurdle).

B) An alternative approach is to use a scoring range to provide better input to the decision makers. In other words, if the scoring range were from 0 to 100, scores below 30 might represent high-risk/low-value investments that should otherwise be rejected, but may only get approved if there were other intangible factors not considered by the scoring model. Projects in the middle range of scores might be approved with more scrutiny, and projects in the upper range would likely get approved. The prerequisite to taking this approach is to have an adequate number of historical scores from past projects to compare against. Statistical analysis would further help refine this approach. Another assumption is that the scoring model would have to remain fairly consistent over time with few changes. Otherwise, historical scores could not be used to determine the correct range unless special adjustments are made to the scores.

5) Finally, scoring models provide the input to build risk-value bubble charts, which provide great visual information to senior leaders. The scoring model needs to contain both value elements and risk elements as inputs for the diagram. Normally, these scores are summed to become a single number, but with the risk-value bubble chart, we need to break out the total value score and the total risk score in order to correctly plot the data on a chart. With further data elements such as strategic alignment and expected cost (or return), more information can be displayed on the bubble charts (see example below).

The Purpose and Goal of Prioritization

April 21, 2014October 11, 2018 Tim Washington 1 Comment

Prioritization is firstly about focus—where to assign resources and when to start the work. It is not primarily about scoring methods and ranking mechanisms. Without defining project priorities, it is difficult to effectively distribute personnel to carry out the highest valued projects. Project priorities enable management to assign their employees to the most important projects. Gaylord Wahl of Point B says that priorities create a ‘true north’ which establishes a common understanding of what is important. Prioritizing projects enables organizations to make the best use of company resources. Without a clear and shared picture of what matters most, lower-value projects can move forward at the expense of high-value projects. Again, prioritization is about focus—WHERE to assign resources and WHEN to start the work. Prioritization and resource allocation go hand in hand.

In the diagram above we see that prioritization relates to resource priority and schedule priority. Resource priority drives the question, “where are we going to invest our resources now?” The fundamental resources are money and people. Since there is often more work to be done than there are resources available, senior leadership needs to provide guidance of where to investment money and where to allocate human resources. This requires an understanding of how to get the most important work done within existing capacity constraints. However, not all projects can be initiated immediately. Prioritization can also help direct the timing and sequencing of projects. In some cases, high priority projects may have other dependencies or resource constraints that require a start date in the future. In other cases, lower priority projects get pushed out into the future. In both cases, schedule priority helps answer the question “when can we start project work?” Having the right human resources available to do project work is a critical success factor. High priority projects have a higher likelihood of success due to adequate staffing. Lower priority projects may face more resource contention and have a higher risk of project delays due to inadequate resource time. Lower priority projects that get pushed out into the future have an even lower likelihood of success since these projects face challenges around project initiation and higher resource contention.

The purpose of prioritization is to allocate resources to the most important work. Prioritization provides focus—WHERE to assign resources and WHEN to start the work. The goal of prioritization is to accomplish the most important work to deliver maximum business value. Although prioritization is a critical need in many organizations, in the next post we will highlight cases where prioritization is a waste of time.

Prioritization Matrix

November 3, 2012May 27, 2020 Tim Washington 2 Comments

In a recent LinkedIn discussion, questions were asked about the short-comings of prioritization matrices. I would like to highlight the strengths and weaknesses of using such a tool for portfolio management. Firstly, a priority matrix differs from a more traditional scoring approach in that it offers a limited number of priority selections. The most simplistic prioritization matrix has three choices, low, medium, and high. Of course, to be effective, every choice should have some predefined criteria. Otherwise, the matrix is of little value because decision makers can have wildly different views for what is of high importance versus low importance.

Strengths

Prioritization matrices have three primary strengths: simplicity, speed, and applicability to all types of work. Prioritization matrices are easy to understand and simple to use. Calculations are not required for determining the relative priority of a project. Basic criteria should be developed for each part of the matrix, but once complete, decision makers can apply the criteria to various types of work. Because of its simplicity, prioritization becomes a much faster exercise and allows decision makers to quickly distinguish important projects from less important projects. In addition, various kinds of work can be prioritized using a prioritization matrix. With a traditional scoring model, it is difficult to evaluate “keep the lights on” type of work, but with a prioritization matrix it is easier to compare priorities for project and non-project work.

Weaknesses

Prioritization matrices are unable to produce a rank ordered list of projects in a portfolio. At best, such a matrix can provide a categorical ranking of projects in the portfolio, but this won’t help prioritize projects within the same category. Prioritization matrices cannot do a good job of evaluating projects based on multiple criteria, and therefore cannot do a thorough job of distinguishing important projects from less important projects. When evaluating multiple large projects, a scoring system will provide a more accurate analysis over a prioritization matrix.

When Should a Prioritization Matrix Be Used?

Prioritization matrices are good for organizations new to the portfolio management process. Due to the simplicity, organizations can quickly get the benefit of prioritization without spending the time to do a thorough scoring of each project. Even in organizations where projects are scored and ranked, prioritization matrices can be used for “pre-screening” purposes to do a preliminary prioritization. This would be commonly used in a stage-gate process before a formal business case has been developed. A governance team could quickly determine a categorical priority for the project at an early gate review. Prioritization matrices can also be used to triage large volumes of project requests to focus the organization on the hottest projects. I have seen this approach used in an organization that received a high volume of small project requests. In this case, scoring would be an over-kill; the organization just needed to determine the most important work at that time.