Qualitative risk assessment

In many circumstances, and increasingly on major projects and programmes it is desirable to quantify the risks to a project in terms of cost and time. The reasons for quantitative analysis may include:

• to give an additional level of confidence in the project’s ability to manage risk;
• to inform the setting of  project contingency or float in a schedule;
• where clients or stakeholders request it – it is becoming increasingly normal for public sector projects where the project forms part of a larger program of projects;
• to focus attention on risk management and the demonstration of the effectiveness of risk management;
• where projects are considered to be complex or risks high
• where client procedures require it or where funders require visibility on the management of risk;
• where the management of contingency is key to the; and
• where it builds confidence of funders or other third parties.

Quantitative analysis is generally used to inform the effect on cost or schedule. Quantitative cost risk analysis (QCRA) is usually based on the cost plan or cost report and quantitative schedule risk analysis (QSRA) is based on the schedule respectively. 

Quantitative cost risk analysis (QCRA)

Project contingency is often derived as a percentage of the capital cost of a project i.e. 5%. This approach produces a contingency figure which is subjective and takes no account of the particular circumstances of the project in hand.  There may circumstances where projects are being repeated as part of a continuing programme and past is a measure of future performance, or where the project is deemed to be relatively simple and similar enough to previous projects to set the contingency as a percentage of capital cost.  The method is neither efficient nor accurate and is largely at the choice of the client as to whether to invest in a more traceable method that, at the very least, gets the project team to focus on managing risk if nothing else.

Using QCRA, it is common to report at the 80% confidence level. This represents an 80% probability (based on the output of the analysis) that the figure will not be exceeded. This is commonly referred to as the P80. There is no particular significance to the P80 and its choice is rather arbitrary, depending more on the governance requirements of the organisation than anything else. Note that the amount of contingency and the analysis of risk are not necessarily linked as contingency is often used to drive performance. Software tools that carry out the analysis, sometimes called Monte Carlo will give a range of confidence levels to enable a choice to be madeto specific needs.

The inputs for a quantified risk analysis are typically the probability values for each risk and an assessment of their impact by either cost or time, i.e. minimum cost impact of £10,000, most likely cost impact of £15,000 and maximum cost impact of £30,000.

In order for this to be effective it is essential that the register reflects actual risks relevant to the project and does not merely represent a list of generic risks that could affect any project. The data used to inform a quantified risk analysis must be of high quality (no gaps, no overlaps). The assessment of risks qualitatively for probability and cost should align with a quantitative approach.

It is recommended during workshops that the risk manager ensures risks are assessed against the appetite ranges previously agreed. To ensure accuracy of results the cost and risk manager need to work closely to ensure the respective models are aligned.

The graphs below show an example of graphs generated by computer based analysis software such as @Risk.

Figure 6a: Cumulative probability distribution.

Figure 6b: Cumulative probability distribution.

Quantitative schedule risk analysis (QSRA)

QSRA is similar to QCRA but is carried out to determine the likelihood of delivering a key milestone or completion, taking into account the risks and uncertainty surrounding the work. The output for a QSRA is similar cumulative probability curve as produced for cost. Sensitivity analysis is particularly important for QSRA as it highlights the areas of the schedule that are critical or near critical and what is driving the outcome.  It is nearly always the case in QSRA that the sensitivity analysis is of more use than the actual output of the analysis as it highlights the areas that need urgent attention.

In order to run a QSRA a logic-linked network must have been developed and form the basis of the schedule. It is sometimes the case that the quality of the logic in the network is insufficient to carry out the risk analysis so amendments and alterations to the network and schedule must often be made. The first exercise in carrying out a QSRA is to check and amend the logic, which often results in an improvement in the scheduling activity.

Quantitative schedule risk analysis is usually carried out to:

• test the validity of the logic network that drives the schedule;
• identify the key drivers for risk and uncertainty in the schedule;
• identify the critical and subcritical activities in the schedule;
• enhance the mitigation strategies for risk and uncertainty;
• assist in setting program duration;
• improve the confidence in achieving the key milestones and completion date; and
• monitor the change in delivery confidence following significant updates to the project baseline.

Like a quantitative cost risk analysis QSRA gives a confidence level as an output. However, in this case it will give varying confidence levels for different end dates, for example, 80% confidence that the project will be completed on or before 18 May.

The most important output from a QSRA is the sensitivity analysis that can be carried out and presented in a number of ways. Various analytical methods have been developed to show the key risk and activities including:

• a tornado diagram that shows the risks or activities that are most highly correlated to the outcome;
• a schedule of the activities that are most often on the critical path including those that are near critical;
• a schedule showing the contribution of activities to the outcome;
• spider diagrams showing the impact on the output from changes in specific activity uncertainties or risks.

An example risk ranking tornado diagram is shown below:

Figure 7: Tornado diagram.