Redefining PUE through the holistic management of data centre infrastructure
26th November 2019
The Impossible made possible
Since 2017, EcoCooling has been involved in a ground-breaking pan-European research project with partners; H1 Systems, Fraunhofer IOSB, RISE SICS North and Boden Business Agency. The objective of the project is ‘to build and research the world’s most energy and data centre’.
Two years since the start of the project, there have been some exciting results achieved, with more expected as the project continues.
Introducing the Project
The Boden Type DC One project (BTDC-1), as it is named, is based in Sweden, where there is an abundant supply of renewable and clean hydro-electricity and a cold climate. BTDC-1 has a design capacity of 500kW of IT load and is made up of three separate research ‘Pods’ of Open Compute (conventional) IT, HPC (High Performance Computing) and ASIC (Application Specific Integrated Circuit) equipment. The EU’s target was to design a data centre with a PUE of less than 1.1 across all of these technologies. So far, the project has demonstrated pPUEs of below 0.02, which has surpassed expectations.
The highly innovative modular building and cooling system were devised to be suitable for any size data centre. In adopting these construction, cooling and operation techniques, smaller-scale operators will be able to achieve or better the cost and energy efficiencies of hyperscale data centres.
Although we all recognise that PUE has limitations as a metric, for this article and dissemination we will use PUE as it is a comparative measure.
Exciting First Results
At BTDC-1, one of the main economic features is the use of EcoCooling’s direct adiabatic (evaporative) and free cooling technology, which produces the cooling effect without requiring an expensive conventional refrigeration plant.
On very hot or very cold, dry days, the ‘single box approach’ of EcoCoolers can switch to the adiabatic mode and provide as much cooling or humidification as necessary to ensure the IT equipment is kept within the ASHRAE standards, 100% of the time. With this cooling and humidification approach, the project was able to produce some very exciting results.
Instead of the commercial data centre norm of a PUE of 1.8 or 80% extra energy use for cooling, BTCD-1 has been achieving a PUE of less than 1.05 with General Purpose OCP servers. This is particularly impressive as it is lower than the published values of some data centre operators that use ‘single-purpose’ servers. The same PUE has also been achieved in BTDC-1 using high-density ASIC servers favoured for bitcoin and blockchain.
This is an amazing development for the potential cost and carbon footprint reduction of data centres. If you apply the economic findings to a typical 100kW medium-size data centre, the energy cost for cooling drops from £80,000 to just £5,000. That’s a £75,000 per year saving for an average 100kW medium-size commercial data centre.
Smashing a PUE of 1.05
What was achieved next has had some truly phenomenal results and presents a wake-up call to conventional server manufacturers if they want to get serious about the total cost of ownership and global data centre energy usage.
You may know that within every server, there are multiple temperature sensors that feed into algorithms to control the internal fans. Mainstream servers don’t yet make this temperature information available outside the server.
As part of the project, one of the three BTDC-1 Pods is kitted out with around 140kW of Open Compute servers. One of the strengths of these servers is that the temperature measurements are made accessible to the cooling system.
The EcoCooling system uses the temperature information from the server to alter the process controllers. Normally, the processing of the cooling system is separate, with inefficient time-lags and wasted energy, however here they have been close-coupled and can react to load changes in milliseconds rather than minutes. What’s more, this is all achieved without the need for extra hardware.
As a result, BTDC-1 Pod 1 operates with a PUE of 1.03!
The BTDC-1 project has demonstrated a robust repeatable strategy for reducing the energy cost of cooling a data centre from £80,000 to £3,000, per 100kW of IT load. This represents a saving of £77,000 a year for a typical 100kW data centre. If you consider the cost and environmental implication of this on the hundreds of new data centres anticipated to be rolled out to support 5G and ‘edge’ deployment, the potential savings are quite remarkable.
Planning for the Future
An integrated and dynamic approach to DC management is going to be essential as energy-use patterns change.
What does this mean? At present most current-generation data centres and indeed the servers within them have a fairly constant energy load. That is because the typical server’s energy use only reduces from 100% when it is at maximum capacity to 75% when it is dormant. There are two huge changes anticipated that will alter the way that data centres operate and the BTDC-1 project has been designed with these in mind.
Firstly, the next generations of servers will use far less energy when not busy, with the 75% quiescent energy usage expected to fall to 25%. This means the cooling system must continue to deliver 1.003 pPUE at very low loads.
Secondly, the implementation of Smart City technology will require data centres to deal with massive processing loads created by things such as driverless cars as well as wildly varying loads depending on the time of day. In BTDC-1 Pod 1, project partners, RISE SICS, are emulating a complete SMART CITY and testing the reaction of the servers as well as the power and cooling loads requirements.
This is just the beginning
The opportunity for EcoCooling to work with leading research institute RISE SICS and German research institute Fraunhofer has allowed the independent analysis and validation of what can be achieved using direct fresh air cooling.
The initial results are incredibly promising and considering we are only halfway through the project we are excited to see what additional efficiencies can be achieved.
You can keep up to date in the progress of the project online at www.bodentypedc.eu.
EU code of conduct the average PUE is 1.8, see https://www.mdpi.com/1996-1073/10/10/1470/pdf page 7 of 18
*** Boden Type Data Centre One is A pan-European consortium consisting of data centre engineering specialist H1 Systems, cooling provider EcoCooling, research institute Fraunhofer IOSB, research institute RISE SICS North and infrastructure developers Boden Business Agency, all of whom have joined forces aiming to design and validate a future-proof concept.