State funds spent on development, operation and using of data centres services Read full summary in English
The aim of the audit was to scrutinise the management of funds spent on building and operating the national data centre (hereinafter “STC1 data centre”), including the expenditure of selected organisational units of the state on buying hosting, server-housing and other related services. The audited period was between 2010 and 2014; where relevant, the preceding period was also scrutinised. Audited entities: Ministry of the Interior (“MoI”); Ministry of Finance (“MoF”); STÁTNÍ TISKÁRNA CENIN, state firm (state banknote printing firm, hereinafter “STC” or “the state firm”). The audit was conducted from June to December 2014. At its 5th session held on 30 March 2015 the SAO Board issued resolution no. 12/V/2015 approving the audit conclusion.
- From the time it took over responsibility (in June 2007) for the coordination of information and communication technologies (“ICT”) on the basis of the amendment of the Competences Act3 to the completion of the audit, the MoI did not propose a strategy governing the use of data centres at national level for approval by government resolution. Even the adoption of a document called Strategic Framework for the Development of the Czech Republic’s Public Administration for the Years 2014-2020 (“the Strategic Framework”), which the government approved by resolution no. 680 of 27 August 20144, did nothing to change this fact. At present, therefore, no system for the development of the sustainability of data centres’ shared services has been put in place; the coordination of the building and development of data centres is undefined; where existing and new information systems should be operated has not been specified; and there are no rules for public administration’s switch to data centres operated by Czech Post or STC.
- The adoption of the Strategic Framework divided ICT competences between the MoI and MoF, whereby the MoI is the coordinator of information systems serving the general public (e.g. the IS of data centres, the basic registers, CzechPoint) and arranges the backbone infrastructure for them5. The MoF is in charge of IS in the areas of financial management, tax, the state treasury and other ICT support in the context of management processes at the level of state administration and the backbone infrastructure for services used by bodies of state administration.
- The Strategic Framework does not specifically define how data centres for the operation of public administration information systems are to be coordinated or any rules for their use.
- Data centres were defined as elements of critical infrastructure on the basis of government resolution no. 432/2010 Coll6. Drawing up the list of these elements is the responsibility of the MoI7. The SAO audit found that this list approved by government resolution no. 934 of 14 December 20118 does not give a full and reliable overview of data centres falling within critical infrastructure, as it does not include the STC data centre where, among other things, the basic registers and state treasury are operated. In the event of an extraordinary event, in which a state of danger, a state of emergency or a state of threat to the state is declared, the incompleteness and unreliability of the information on this list could have a negative impact on the handling of crisis situations. For example, the availability of essential services the state provides for citizens (registers, CzechPoint etc.) could be fundamentally compromised.
- An obligation to inform the government about how selected ICT services used by state administration are secured was only imposed on the MoI9 and only with regard to services provided by Czech Post, even though STC, a firm established by the Ministry of Finance, has also provided key services for state administration bodies since 2009 (see Section 2.2). The government is thus unable to get hold of complete information about the securing of ICT services in state administration.
- Although the MoF’s first intention was that the first data centre would be completed in 2005, it was not until July 2005 that the MoF informed STC of the decision to build a data centre inside the complex of this state firm. STC put the part of the data centre necessary for the operation of the Integrated Information System of the State Treasury (“state treasury operation”) into operation in the existing building in 2009. STC wanted to build other premises as a new building in its complex. The delays in the preparation of the data centre were accompanied by insufficient cooperation between the MoF and STC. Despite being repeatedly urged by STC, the MoF did not sign a cooperation agreement with the state firm covering the building of the data centre and the MoF did not define the technical requirements and present them to STC until the middle of 2007.
- In September 2005 the MoF signed a contract with an external contractor for an initial study dealing with the construction of the data centre. STC received this study from the MoF and assigned the elaboration of project documentation for zoning proceedings to the same contractor. STC then commissioned the drawing up of an opinion from the Czech Chamber of Certified Engineers and Technicians Active in Construction with the conclusion that the author’s invention was incorporated into the study and project documentation for zoning proceedings, so the author has the right to use copyright. Based on this opinion, STC awarded the elaboration of project documentation for building permission and construction work in non-public procedure with reference to copyright. The general project designer of the data centre construction project was consequently the company that drew up the initial study and project documentation for zoning proceedings.
- STC removed the elaboration of project documentation, which covered selected parts of the data centre technologies (including UPS back-up sources12) from the scope of the aforementioned contract for the elaboration of project documentation for building permission and construction work. In the contract documentation STC specified as the supplier of this part of the project documentation the company that had been advisor for construction and technology since the initial phases of the data centre preparation and subsequently won the contract for the building contractor. To STC’s order this company drew up the said part of the project documentation for building permission.
- STC concluded seven addenda to the contract with the general project engineer. One of them covered the elaboration of project documentation for the construction of the technological part. STC was in breach of the Act on Public Procurement, as it awarded the elaboration of this documentation in a procedure without publication, even though the conditions for this type of public contract were not in place. In the audited case the elaboration of construction documentation for the technological part was a necessary component of the project work, so the need for it could not have been caused by circumstances that were objectively unforeseeable13. What is more, the obligation to draw up this documentation was set out in the contract documentation and contract for the elaboration of project documentation for the construction of the data centre.
- STC conducted the public contract for the construction of the data centre in the form of an open procedure.
- In the contract documentation STC required candidates to supply such technological equipment (including UPS back-up sources) as satisfied precisely defined technical parameters or, where appropriate, to offer such equipment as displayed better parameters than those defined.
- Three candidates submitted bids. Two of these candidates’ bids, which were approximately CZK 50 million cheaper, were excluded on the basis of expert findings on the grounds that these bids did not meet the contract conditions, particularly in the technological part comprising UPS sources. In other words, the only candidate not excluded during the procurement procedure was the supplier of the project documentation for building permission concerning this technology. That candidate’s bid corresponded exactly to the parameters in the contract documentation that arose out of the project documentation created by the candidate.
- STC paid a total of CZK 386 million, including VAT, for the preparation and execution of the construction of the data centre.
- STC has provided the MoF with data centre hosting services since 2009 for the state treasury operation, simultaneously providing it with related Service Desk services (see Section 2.3.1 for more). In 2011 STC started to provide data centre services to two other entities. In the middle of 2012 STC put a new data centre into pilot operation and signed a contract with the Basic Registers Administration for the operation of the basic registers. Pilot operation in the newly built premises commenced in the second half of 2012. After standard operation was launched at the start of 2013, STC signed three more contracts for the provision of data centre services.
- According to the STC business plan that was based on the MoF department’s ICT strategy14, hosting, especially for the MoF department, should be the basic data centre service provided. With the exception of contracts concluded with the MoF and dealing with the provision of hosting services, at the time when the SAO’s audit finished all the other contracts dealt with housing. Under most contracts, therefore, STC provided services with lower added value.
- At present a private company can buy data centre services from STC and sell them on to another state administration body. The SAO already drew attention to this situation in the audit conclusion of audit no. 12/35, which stated that the information systems for the pay-out of Ministry of Labour and Social Affairs benefits are operated through a private company in the STC data centre. The SAO has repeatedly flagged up the possible risk of uneconomical spending of state funds in cases where state administration bodies using data centre services are sold services bought from a state firm through an intermediary that is a private company
- From the start of live operation to the end of this audit, the occupancy rate of the newly built data centre was around 50%. At the time of the completion of the audit, approximately half of this data centre’s capacity had thus been unused for almost two years.
- In the SAO’s opinion, the following are the main reasons for the low occupancy rate of the new STC data centre: (1) insufficient coordination by the MoI and the absence of any regulation defining which IS are to be operated in data centres under state control; (2) the higher price of the services provided, especially at the start of operation of the new STC data centre, compared to data centres run by private entities, as STC initially based the prices of its services on costs linked to the current occupancy rate of the data centre.
- When the Service Desk service was being introduced, the MoF signed a contract to support its introduction by an external entity worth CZK 2.2 million including VAT. Approximately 1.5 months after this contract expired, the MoF signed another contract with the equivalent content as the previous one. Approximately 1.5 months after the second contract expired, the MoF concluded a third contract. The total value of the contracts was CZK 5.5 million. The MoF concluded the contracts on the basis of separate procurement procedures in the form of smallscale contracts and by contacting three candidates, or five in the case of the last contract. All three contracts were won by the same candidate. The MoF violated the provision of Section 13 (3) of the Act on Public Procurement by awarding the second and third contracts as smallscale public contracts. In doing so, the MoF split the subject of the public contract, thus reducing the expected value below the financial limit specified in the Act on Public Procurement.
- The MoF pays STC a monthly flat rate for the use of Service Desk services. From 2010 to 2014 the amount changed from CZK 1.7 million including VAT to CZK 2.2 million including VAT. The MoF paid STC a total of CZK 125.2 million including VAT from January 2010 to September 2014. In the same period STC paid the subcontractor that authored the classified document on whose basis the contract was awarded in classified mode CZK 56.4 million including VAT, which is 45% of the amount received from the MoF.
- The MoF pays STC a monthly flat rate for the state treasury operation. Over the years the amount has increased from CZK 17.2 million including VAT in 2009 to CZK 17.8 million including VAT in 2014. During the implementation of the contract, i.e. from 2009 to 2014, the state treasury operation cost CZK 1.031 billion including VAT. STC hired a subcontractor to provide selected services under the contract in question. That subcontractor was the company supplying the Integrated Information System of the State Treasury for the MoF. In the same period STC paid this company CZK 703.5 million, which is almost 70% of the amount STC received from the MoF.
- As the MoF awarded both contracts to STC in classified mode, STC had to follow the same procedure when awarding the contract to the subcontractor supplying the relevant services. It follows from the above that these subcontracted services had a major role in the services STC provided to the MoF.
- STC did not have an effective pricing directive either when preparing the bid or at the time of signing of the contract for the state treasury operation and for the provision of Service Desk services. The main pricing difference between printing services and ICT services was that indirect overheads were assigned to individual contracts and a different approach was taken to defining the level of profit from the contracts. STC thus had to define prices and profit levels individually for the two contracts in question. The profit actually achieved by STC from the two contracts was significantly different.
- In the case of Service Desk services, the profit level was defined directly in the contract and amounted to approx. CZK 6 million, i.e. 6% of the total price for the five-year period. The actual profit of CZK 11.75 million that STC achieved a year before it stopped providing the service exceeds the profit as defined in the contract by 78%. For STC to achieve the agreed profit level, in 2014 it had to achieve 38% lower costs than the average annual costs for the period of service provision.
- Under mandate contracts STC provided the MoF ICT services from the year 2000. According to the MoF, the ministry needed these to ensure the proper operation of the information systems in its charge.
- . STC did not provide the MoF with specific services and invoice the MoF for these specific services - it invoiced the MoF for the work performance of its employ
- Over the last ten years, 32 STC employees in total carried out contractual activities for the MoF; 12 of these employees were original employees of the MoF. As part of the implementation of the contract, the MoF paid a flat rate of CZK 95,000 not including VAT (CZK 82,000 not including VAT up to 2002) per STC employee. This amount is not consistent with these STC employees’ pay, however, as the SAO audit found that their pay, including payments to the state required by law, was less than 50% of the amount paid by the MoF.
- The MoF justified concluding a mandate contract instead of using its own employees by the need to satisfy the requirements of a number of government materials on reductions in the workforce of ministries and also by economic advantages. The MoF’s arguments indicate that the MoF transferred its employees to STC in order to fulfil government regulations and was not, according to the MoF, capable of securing the standard operation of the ministry without them. The course of action chosen by the MoF was uneconomical, as the MoF paid these employees double the amount they were paid by STC (and previously by the ministry itself).
- Assigning ordinary tasks to another employee was permitted by the Act on Employment up to the end of 2006. By not using its own employees from 2007 the MoF obviated the Act on Employment and the Labour Code. In connection with the implementation of the mandate contracts STC repeatedly signed work contracts for a period of one calendar year, i.e. for a fixed period, with most of the employees working for the MoF. STC thus repeatedly violated the Labour Code, which does not permit the manner of concluding fixed-term work agreements described above.
- Based on the mandate contract, STC invoiced the MoF a flat rate per employee on the basis of attendance sheets, which were part of the invoicing, without specifying the work done by the employee. The MoF was therefore hiring STC employees.
- The MoF paid the full flat rate for STC employees even in periods when they were not doing any work for the MoF, e.g. when they were on leave or maternity leave, when they were incapacitated for work, visiting a doctor or on non-training work trips. The MoF paid at least CZK 4,611,034 including VAT for this work absence.
- The MoF paid a total of CZK 253,894,984 including VAT for the implementation of the mandate contract from 2004 to September 2014. By not using state budget funds of CZK 4.6 million in line with substantive performance and not proceeding in the most economical manner when carrying out certain tasks, the MoF was in breach of budgetary discipline26 and violated the Act on the Property of the Czech Republic
- In 2002 the MoF intended to build a second data centre for its department by June 2009 in collaboration with STC. Minutes of a session of the STC supervisory committee attended by MoF representatives revealed that the STC management only informed the supervisory committee of the building of the back-up data centre in October 2011, i.e. at a time when the MoI Strategic Framework had not been approved. In the years 2012 to 2014 STC commissioned the elaboration of all parts of the project documentation and commenced preparatory demolition work in Zeleneč on land it had bought from the MoI. STC paid a total of CZK 63.3 million including VAT for the preparation of the back-up data centre in Zeleneč and for the land. The construction of the back-up data centre was expected to take place in three stages. Its total area was to be three times bigger than that of the existing STC data centre.
- The occupancy rate of the existing STC data centre in September 2014 was around 50%. The presented documentation does not justify the parameters of the intended project for the data centre in Zeleneč (e.g. the selected location, the size of the data centre), in particular given the state of affairs whereby the use of data centres by state administration bodies is not effectively coordinated and governed by regulations.
- In this context, the SAO audit found that documents related to the MoF’s checks whether STC’s business activities were being provided efficiently and economically for the state’s requirements under the Act on State Firms did not contain any conclusive record that these kind of checks had been performed.