Energy Efficient High Performance Computing Working Group SC14 Presentations

SC15 Working Group Birds of Feather Presentations

Identifying a Few, High-Leverage Energy Efficiency Metrics?


How do we bring speed, clarity and objectivity to energy management in HPC centers? What are usage models and key metrics for current and envisioned energy information systems? What are metering requirements (what sensors and meters at which points and at what measurement frequency)? These are timely and critical questions for System Administrators, Operations and Facilities Managers and End-users.  The purpose of this BoF is to review and provide feedback on a collaborative effort to identify requirements for HPC Data Center energy information systems.  The BoF will host a panel of seasoned administrators and managers, followed by lively audience discussion.

Panelist Presentations:

Website Survey Questions: 

We are still soliciting responses to the survey to provide feedback on proposed energy efficiency metrics: One for the HPC Center Operations Manager, another for the HPC Center Facilities Manager and the last for the HPC Center System Administration Manager.  Rate each of the metrics as to its priority (high, medium or low). 

Please complete the survey at

Dynamic Liquid Cooling, Telemetry and Controls; Opportunity for Improved TCO


Today’s practice for liquid cooling is to use CDUs with constant flow-rate and temperature. Tomorrow’s products could be designed for variable flow-rate and temperature based on actual heat removal requirements, but there would have to be more and finer grained telemetry and controls. How much savings can be gained in energy savings compared to the incremental capital and operational costs? Application load differences can cause rack power variation, but nodes within a rack can also vary. Where is the sweet spot for implementation- at the rack, node or even component level? This BoF will address these and other related questions.

Panelist Presentations

  • Nic Dube from HP
  • Dave Martinez from SandiaNL
  • Thomas Durbin from NCSA
  • Wade Doll from Cray
  • Chris Marroquin from IBM
    • Introduction
    • Can we reduce operational expenditures and improve energy efficiency by optimizing liquid cooling systems with dynamic controls?
    • Where are liquid cooling controls best implemented – in the HPC system or in the building or both?  Where is the line between best cost vs control?
    • What feedback (telemetry) is needed from the system to optimize dynamic cooling?  Experiences?

Moderated Panel Discussion

Primary  Discussion Questions:

    1. What are your new and upcoming technologies that would provide liquid cooling controls to promote energy efficiencies? 
    2. What controls systems are necessary from the perspective of the build infrastructure?
    3. Can liquid cooling control systems improve efficiencies with higher inlet water temperatures?  Where should inlet flows and temperature differential go from here?
    4. Can we reduce operational expenditures and improve energy efficiency by optimizing liquid cooling systems with more dynamic controls?  Where is the balance between capital expenditures and controls and payback?
    5. Where are liquid cooling controls best implemented- in the HPC system or in the building or both? What telemetry is required for optimized dynamic liquid cooling?

Backup Discussion Questions:

    1. The chemistry and quality of the water is a concern.  What needs to be done for monitoring and maintaining the right chemistry?  How far do controls go?  Is it just cooling, the total environment for providing the cooling?  Can we automate those controls?
    2. How many sites have an opportunity for greenfield - rebuild their infrastructure and support - verses an evolving infrastructure? 
    3. How can infrastructure and controls stretch the lifecycle of the cooling equipment and grow the efficiency overtime? 
    4. Can utilization of advanced controls increase the lifecycle of equipment and facilities?
    5. HPC loads can vary by multiple megawatts], what are your or your customer’s site experiences? Is there a wide difference in site experiences?
    6. Environmental conditions also vary, which allows for water to be cooled with greater or lesser amounts of energy and/or for the water to be cooled to varying temperatures. What are your site experiences? Is there a wide difference in site experiences?
    7. What other factors might drive for more dynamic controls- e.g., water conservation?
    8. What other strategies could be implemented to minimize power load, such as power managed job scheduling.
    9. What are the challenges to implementing dynamic controls?

Website Survey Questions: (Please email your survey answers to EE HPC WG BoF Survey)

    1. Did you find this BoF valuable? A lot, somewhat, very little
    2. What would have made it more valuable?
    3. Would you like to join the EE HPC WG? Name, affiliation, and email
    4. Would you be willing to do a case study on dynamic liquid cooling controls?  Yes or No
    5. What can we learn from you in regards to dynamic liquid cooling controls?
    6. Other Comments?
Acrobat documents
  Lawrence Livermore National Laboratory
7000 East Avenue • Livermore, CA 94550
  Operated by Lawrence Livermore National Security, LLC, for the
Department of Energy's National Nuclear Security Administration
LLNL-WEB-670983  |  Privacy & Legal Notice