Algemene Rekenkamer NCA

The first force pushing projects past what they can carry is enthusiasm. 'Political decision-makers tend to believe in ICT as a miracle cure for all manner of policy issues: the ideal solution to any problem. Yet managers often do not fully understand what ICT can do and, more important, what it cannot' (p.14). The consequence runs in two directions at once: they overestimate the technical capabilities and underestimate the time, money and people implementation will take. If the minister then decides without adequate advice, the project becomes organisationally and technically unfeasible and its goals stop being specific, measurable, agreed, realistic, time-bound and consistent - 'without a realistic business case, the project can be a source of great disappointment'. Nor is the enthusiasm corrected from outside: vendors 'for commercial reasons tend not to be overly critical', and because they deal directly with political managers, 'the latter quickly come to think that an ICT solution is within reach'.

• Function: IT, Strategy
• Value: Domain knowledge
• Stakeholders: Policy setter, Supplier
• Quality: Adequate resources and competences Appropriate use of technology and automation

'Deadlines are often the outcome not of reasoned and realistic plans but of political considerations' (p.14). The House expects quick solutions to urgent problems; ministers time delivery to their own term of office. When other conditions are fixed in advance too - how many organisations take part, what functionality is required - the project becomes incapable of being completed on time. The report's illustration is the capacity-for-work legislation: the implementing bill went to the Council of State for advice at the end of March 2005 with introduction proposed for 1 January 2006, and the implementing bodies said they could manage it only if the bill passed unamended, with systems operational by 1 September 2005 at the latest. At that point nobody knew when the bill would become law. The second half of the finding is quieter and just as damaging: under time pressure, 'there is also a risk of not enough time being taken to define the project's goals and requirements'.

• Function: Planning, Governance
• Value: Regulatory system
• Stakeholders: Lawmaker, Policy setter
• Quality: Clear objectives and goal-setting Sound risk management

Changes to a project's terms of reference should prompt a re-examination of the assumptions under it. 'This, however, is more the exception than the rule. As a result, the original development path becomes entrenched and project management considerably more complex' (p.15). The same is true when warning signs appear in planning, staffing, budget or required functionality: it rarely happens, 'chiefly because political decisions are difficult to reverse once they have been taken' - and because 'the individual signs are too small to be worth considering whereas all the signs together are not'. The audit office's illustration is a shared service centre for personnel administration, where a second-opinion committee made several critical comments, concrete action was taken on one of them, and the action plan was never taken up again in subsequent meetings. The chapter closes on the difference from private business: 'a project muddles on instead of being stopped when it should be. The private sector must obey the rules of the market.'

• Function: Project methodology, Governance
• Value: Finance
• Stakeholders: Management, Oversight
• Quality: Sound risk management Cost control and value for money

'Government ICT projects are often organisationally complex because they involve several organisations. The autonomy enjoyed by the organisations means there is little if any central steering or commitment' (p.16). And below the structural point sits a behavioural one: 'in practice, organisations act primarily in their own interests. Their contribution to and acceptance of ICT solutions often depend on the extent to which the solutions serve their own interests or solve their own problems.' The illustration is a residence-permit programme launched in September 2002 by three ministers, implemented jointly by a chain that included the foreign ministry, the police, the border service, the immigration service and the municipalities, each responsible for its own project. An audit in 2005 found 'the chain did not have any shared goals. The individual organisations pursued their own interests' - not necessarily contrary to the responsible minister's goals, but not serving them either - and that the participants 'did not always look upon each other as partners, did not always think they were appreciated as partners and did not always accept directions'.

• Function: Governance, Organizational design
• Value: Community
• Stakeholders: Delivery partner, Mandating authority
• Quality: Clear objectives and goal-setting Streamlined, standardized processes

'ICT projects generally lead to organisational change and, conversely, organisational change can have a significant impact on the ICT infrastructure. The pursuit of a political goal often overlooks the relationship between ICT and the organisation' (p.17). Transferring tasks from one body to another changes business processes, and because processes and systems are tightly bound, a change in one has serious consequences for the other - 'the impact that this can have is frequently underestimated'. The illustration: preparations to move tasks from one service to the immigration service began in 2002, and once it became clear new systems would be needed the service studied what would have to change; it found the planned completion date unrealistic and the implementation date had to be put back six months. The reverse direction is the same failure seen from the other side: 'ICT systems are frequently developed without considering the consequences for the organisations that have to use them', with the system operating sub-optimally because it does not fit working practices or because users were inadequately prepared and trained.

• Function: Organizational design, Work process design
• Value: Human capital
• Stakeholders: User, Management
• Quality: User-centred design and usability Streamlined, standardized processes

'ICT projects are different from other projects because ICT systems are relatively inflexible. Many political and organisational processes, however, are dynamic and flexible' (p.18). Development benefits from a stable environment and from early, precise definition of goals and requirements; without them 'the requirements the system must satisfy will not be known', and where an external provider builds it, 'there is then a significant risk of it delivering something the government did not intend', because the provider 'forms an incomplete and possibly incorrect picture of what the client expects'. The subsequent changes then cost time and money. The illustration inverts the proper order entirely: under great time pressure in 2003, the immigration service had to develop new systems while some of its processes were still undefined - 'the process designers were only one step ahead of the system designers. At one point the ICT department was actually leading the development of the business processes.'

• Function: IT, Product/service purchase
• Value: Domain knowledge
• Stakeholders: Supplier, Mandating authority
• Quality: Clear objectives and goal-setting Adequate resources and competences

'A complicating factor in the development of ICT systems is that the systems themselves often do not work in isolation but have to be connected to other systems already in operation' (p.19). Compatibility inside one organisation is already demanding; across several there are further organisational and technical complications, and data from two bodies 'must be collated even if they do not agree with each other'. The illustration is a stock-records audit: in 2004 there was a 17 percent variation between recorded and actual stock at one military logistics centre and 28 percent at the clothing and equipment agency, the warehouse records at both being unreliable. The causes are named exactly - 'old and polluted systems being connected to each other', and old batch-processing systems connected to modern real-time systems, 'which created discrepancies in the files'. The audit office concluded that reliable stock records would remain a problem for as long as the old systems were running. The chapter adds a third technical factor: ICT advances so quickly that expertise and know-how become obsolete during the build.

• Function: IT, Infrastructure
• Value: Assets
• Stakeholders: Operator, Management
• Quality: Interoperability that spares users redundant effort Reliable, integrated information base

The report states its own central conclusion twice, and the second time it explains itself. The remedies are not obscure: 'the motto is: start small and proceed in small steps', limit the number of organisations, choose standard software, split the project into controllable parts, and where complexity is genuinely unavoidable adapt the completion time instead. 'All the recipes given above are known. But they are often not applied even though those involved know that projects are doomed to failure if they are too ambitious or too complex. Why is this so?' (p.20). The answer is that three parties each have a legitimate reason to want the project big. The House 'expects the government to solve complex problems, preferably as quickly as possible'. Ministers 'like to show they are decisive', and 'announcing a feasibility study or a small-scale pilot scheme is not usually seen as decisive action'. Providers 'need contracts, preferably big ones'. None of them checks the others: 'the parties entrap each other in the spiral and inevitably agree upon a project that is too complex but has the status of political fact from which there is no elegant way back.'

• Function: Governance, Strategy
• Value: Community, Regulatory system
• Stakeholders: Lawmaker, Policy setter, Supplier
• Quality: Cost control and value for money Functioning oversight and governance

Without accurately formulated goals, 'it will not be possible during implementation to determine whether the project is still on course and the minister will subsequently not know whether he has achieved his goals and whether the money was spent as agreed' (p.22). The illustration runs the length of a decade. Neither the House nor the ministry doubted that something had to be done about computers in schools, and from the late 1990s a policy was built covering infrastructure, equipment, teacher training and digital teaching aids. 'At the beginning of the project, the House repeatedly requested operationalised policy goals, but never received them.' What it got instead were formulations the report quotes without comment: 'taking an integrated approach to embed the use of ICT into education', 'taking a lead on neighbouring countries in the effective use of ICT', 'full integration of ICT in education', 'preparing children and young people for the knowledge-based society'. None of them can be failed.

• Function: Planning
• Value: Domain knowledge
• Stakeholders: Lawmaker, Policy setter
• Quality: Clear objectives and goal-setting Reliable, integrated information base

Many ICT projects carry the characteristics that make a project 'large' under the parliamentary regulation - non-routine, large-scale, new technology, organisationally complex implementation - and so should be reported on periodically whether or not they are formally classified as such (p.23). Two failures are set out. In 2000 a parliamentary motion asked for a single police information system and for annual progress reports; in 2003 the audit office found the second half of the motion had simply not been acted on, and the minister then undertook to report annually from 2004 - four years late. And on the emergency communications network, the audit office concluded in June 2003 that 'on two thirds of the checkpoints, information did not satisfy the requirements of the Large Projects Procedural Regulations. Some information, including information on delays, was available at the ministry but was not provided to the House. Other information, for example financial information, was not available to the state secretary.' Over the six years to February 2003 the House 'was unable to form a reliable picture of the financial, substantive and planning aspects' of that project.

• Function: Governance, Regulations
• Value: Regulatory system
• Stakeholders: Lawmaker, Oversight
• Quality: Functioning oversight and governance Reliable, integrated information base

Information can be delivered on time and still leave the reader worse informed. On the shared service centre for personnel administration, the information given to the House until mid-2005 'was at times too optimistic and unclear' (p.25). The specific instance is a progress report of 30 June 2004 which 'did not clearly state that the government's approval of the report would in effect be a government decision to go ahead with the implementation'. Conditions had been set for that approval; the minister knew when the decision was taken that there were risks and that the conditions might not be met; and 'the picture the minister presented to the House on 30 June 2004 did not clarify these risks and shortcomings. The House had to suffice with comments made by the Second Opinion Committee in an appendix.' On the same project the minister was later reluctant to report operational disagreements while there was still hope of success, arguing that commercial interests had to be respected - to which the audit office answered that the House could have been informed in confidence.

• Function: Governance
• Value: Finance
• Stakeholders: Lawmaker, Management
• Quality: Reliable, integrated information base Functioning oversight and governance