EURIM Modernising Government Group Occasional Paper September 2003
		Checklist for Programme Review
0	Background
	EURIM has published a series of briefings on Modernising Government over the five years from 1998 to 2003 and a central theme of these has been the challenge of and variable performance in managing complex programmes of change. The Government has recognised that improvement in this area is key to changing the way that government is delivered to exploit new technology and place the citizen, rather than the Department, at the centre of the process. Major progress has been made through organisations such as the Office of Government Commerce, National Audit Office, Office of Public Services Reform and the Office of the e Envoy to improve skills and put in place mechanisms to increase success. The OGC has also put in place, and continues to develop, a process of Gateway Reviews to help Departments assess and address issues on delivery programmes. In support of those initiatives, EURIM identified a need to brief Parliamentarians on how they might explore, in talking to Departments or participating in select committees, whether these guidelines are being actively applied in practice to deliver results. This paper is by way of a checklist of questions that might help Parliamentarians to probe the reasons for failure, whether in projects to deliver new systems or programmes to achieve policy outcomes.
1	Starting point
	We take as founding principle the guidelines, endorsed by the Cabinet in December 2002, to ensure better management of IT-enabled change programmes:
1.	1.1	Establish Project/Programme Management (PPM) Centres of Excellence in each Department with a remit including its Agencies and Non-Departmental Public Bodies. To be established by June 2003.
2.
3.	1.2	Accounting Officers to provide assurance on existing (pre go-live) and new major projects that they are not based on common causes of failure identified by the National Audit Office.
4.
5.	1.3	Mandate no big-bang implementations and developments (i.e. modular, incremental developments and implementations) unless approved by central scrutiny group (e.g. Chief Secretary to the Treasury, Minister for e-Transformation, Office of the eEnvoy, OGC).
6.
7.	1.4	No government initiative (including legislation) dependent on new IT to be announced before analysis of risks and implementation options has been undertaken.
8.
9.	1.5	Force prioritisation of all existing and new projects as Mission Critical, Highly Desirable and Desirable.
10.
11.	1.6	All high risk and mission critical projects to have clearly identified (i) responsible Minister (ii) SRO and Project Manager with good relevant track records.
	The following questions expand on these and address some of the identified common causes of failure.
2	Programme objectives
	How do the key parameters of the programme – costs, timescale and forecast benefits – compare today (or at the end of the programme) with those when the contract was let?
	2.1	What was the scale of the changes made (if any) post contract award? Were these consistent with expectations of both parties and the nature of the contract?
	2.2	Was the programme specified (in the contract) in terms of business benefits / policy outcomes or was it specified on the basis of developing a system? What were the objectives?
	2.3	Given that the success of the policy objectives might not be measurable for a number of years, what interim measures of success were specified? How is the programme performing against those?
	2.4	Will the programme deliver the planned policy outcomes? (If it will, we may still judge the programme to be a success even if it is off track at present, provided there is a plan to get it there)
3	Marketplace and suppliers
	3.1	Was there an effective competition?
		3.1.2	How much interest was there from suppliers? (There could be a number of reasons for lack of interest, such as suppliers not believing the customer had the money or capacity for what he was asking for, did not really know what he wanted, or that an incumbent supplier was too well entrenched.)
	3.2	Was the supplier chosen on the basis of the cheapest compliant bid or genuinely on the best value for money (and quality) over the life of the programme?
		3.2.1	How did the quoted effort estimate and price of the chosen supplier compare with other bids?
		3.2.2	Was there a higher cost proposal that might have been lower risk / more successful?
		3.2.3	Were the risks associated with the chosen supplier’s approach assessed and were they greater or less than other proposed approaches?
4	Supplier and customer capability
	Did the experience, track-record and quality processes of both the supplier and the customer organisation match the importance and risks of the programme?
	4.1	On the customer side: had the SRO and Programme Manager experience of managing programmes of a similar scale and complexity before?
	4.2	If not, (how) were they given support from coaches or mentors, independent of the solution suppliers, who had experience of this type, scale and complexity of programme?
	4.3	Have there been any significant changes on the customer side (organisational change, change of Ministers or key managers, changes to Government policy) over the life of the programme? How well were these accommodated by customer and supplier? Did they have a major effect on the outcome?
	4.4	On the supplier side: who were the SIE and Programme Manager; did they change during the programme and how often? Had the SIE and Programme Manager experience of managing programmes of a similar scale and complexity before?
	4.5	Did the supplier commit a team of appropriately skilled and experienced practitioners and what assurances did they give about making this team available for the work?
5	Customer-Supplier Relationship
	5.1	What was the nature of the customer-supplier relationship – traditional, or more of a modern partnership?
	5.2	Was this appropriate for the capability/services that were being procured?
	5.3	Would you characterise the relationship between customer and supplier as one of trust and openness? If not, why not? If it broke down, what were the reasons for this?
6	Risk profile
	6.1	Was there a single, common risk register?
	6.2	At the start of the programme, what were the top 5 risks identified and what plans were put in place to mitigate them?
	6.3	Did these risks materialise and what action was taken as a result?
	6.4	What are now the top 5 risks that might prevent the programme from delivering its policy outcomes and what action is being taken to mitigate each?
7	Technology
	7.1	Had all the elements of the chosen technology been proven in other programmes that had been delivered on time and on budget?
	7.2	If not, was the feasibility of the chosen technology demonstrated before committing to using it in a “production” solution?
	7.3	Was the programme incremental in nature or big bang?
		7.3.1	Was the programme broken down into projects, any one of which could have been cancelled (if it was not delivering) and replaced with an alternative solution without prejudice to the overall programme?
		7.3.2	If not, was this considered and why was it rejected?
		7.3.3	What would have been the impact on time and cost of cancelling the largest project in the programme and adopting an alternative solution?
		7.3.4	At what level had the big bang approach been endorsed?
8	Management and control
	8.1	Was the programme subjected to independent Gateway reviews prior to all key decision points? Were the key findings implemented in a timely way?
	8.2	Were any major milestones missed? If so, what management actions were taken to review the causes and impacts and avoid recurrence of the problems?
What is EURIM? EURIM is an independent, UK-based, all-party Parliament-Industry group funded by its members. It is a fast-track conduit through which ICT stakeholders can communicate their concerns to policy makers in government. This might involve arguing the case for a reduction in regulation, for a review of current legislation, or for new initiatives by government. Alternatively it might be a case of alerting parliamentarians and officials to potential barriers to achieving policy objectives, and where possible, identifying solutions. EURIM is proactively consulted by Government at the pre-legislative stage of Bills and European Directives, and has an excellent track record of success in achieving change at all stages of the consultation process. www.EURIM.org
© Copyright EURIM 2003. All Rights Reserved. For written permission to reproduce any part of this publication please contact the Administrative Secretary, EURIM (email: admin@eurim.org, Fax 01984 618 383). This will normally be given provided EURIM is fully credited. Whilst EURIM has tried to ensure the accuracy of this publication, it cannot accept responsibility for any errors, omissions, mis-statements or mistakes.