This project has analyzed the business potential of urgent HPC simulations of earthquakes. The current technology developments seem well aligned with market needs, especially for insurance and international agencies. Nevertheless, in order to fit the needs of these sectors, further developments are needed. The sector related to agencies is likely to provide funding to further develop the technology whereas the insurance sector might be the best end user in terms of market volume. Current bottlenecks could be solved with a strong partnership with HPC providers and building confidence in the earthquake risk market.

Our society is profoundly influenced by advanced materials and technologies. Materials have helped to raise our standard of living, but new challenges steadily emerge and novel materials are demanded, representing a key element of the commercial success of tomorrow's industrial products and competitiveness of the industry. It is thus not surprising that the European Commission considers Material Research as “Key Enabling Technology”1.

Up to now, new materials are characterized and designed mainly using experimental techniques. However, this approach has limitations, as for instance safety issues, the necessity to synthesize a material before characterizing it, a lot of trial-and-error, elevated costs, and large time-to-market. Another way to design new materials is by using simulations, which allow to determine the characteristics of matter on the computer. In particular, we focus on atomistic simulations, which take into account the fundamental constituents of matter and therefore yield an outstanding precision and universal applicability.

On the one hand, atomistic simulations have many advantages over experimental techniques: e.g. it is possible to investigate the behaviour of matter under conditions that are hard to reach experimentally (e.g. temperature, pressure) or to analyse materials that have not been synthesized yet. Moreover simulations are often cheaper and faster than experimental techniques and allow automated workflows.

On the other hand, atomistic simulations are not easy to conduct and require a lot of background knowledge and computational power, which are often missing in the industry and therefore hinder their usage.

In this EuroLab4HPC Business Prototyping Project we have explored ways to overcome these barriers, developing various business hypotheses to make atomistic simulations accessible to industry.

We hereby describe the outcomes of the PREDICT project, a EuroLab4HPC Business Prototyping Project (BPP) to develop a business plan for BSC’s cost-aware failure prediction model for HPC systems. In HPC systems, a system failure leads to the termination of an entire HPC job and the loss a potentially large number of server hours. There exist various approaches to mitigate this cost, such as checkpoint–restart, but the overhead is considerable, so they are cost-effective only when the probability of system failure is high. For this reason, it is necessary to estimate the probability of a system failure, which requires an accurate failure prediction algorithm such as PREDICT. All relevant strategic research agendas in Europe recognize the importance and challenges of HPC system resiliency.

In our initial business model, we anticipated that the typical end users of our technology would be medium and large HPC service providers, such as the operators of Tier-0 or Tier-1 PRACE systems in Europe, National supercomputers in the US or industrial HPC systems. We recognized that commercialization of PREDICT would also require an analysis of system integrators or OEMs (original equipment manufacturers), such as IBM, Lenovo, Bull/Atos, Oracle, Cray, Dell, or HP. OEMs provide a list of the software that is certified to work on a given system, and we knew that HPC system administrators are reluctant to install any system software that has not been certified by the OEM that deployed the HPC cluster.

The first step in the PREDICT BPP project was to estimate the true cost of HPC system failures, which are caused by uncorrected DRAM errors. Since there is no suitable public data, we analysed information from the MareNostrum supercomputer (hosted by BSC) and held discussions with the operators of Tier-0 PRACE HPC systems. From a detailed analysis of production logs from MareNostrum, obtained from the BSC operations department, we found that a small proportion (below 0.001%) of the overall server hours is lost due to uncorrected DRAM errors. This was much lower than we had expected. Nevertheless, some production jobs are extremely large and these jobs have to be protected with advanced resiliency mechanisms.

We also approached three Tier-0 HPC system operators in Europe (anonymized to protect the interested parties). All three institutions have a record in HPC system resiliency research. All the supercomputing centres that we approached agreed that it is important to analyse the cost of system failures and the cost-efficiency of any (advanced) resiliency approach that would be deployed in the field. Up to this point, however, we found that the sharing of quantitative data requires non-disclosure agreements and that this data is typically protected by service agreements between the HPC centre and the system integrator. Progress on data sharing has therefore been very delicate and slow. We also found that there is a need for our PREDICT technology to be validated by the scientific community through the publication of a scientific paper in a highly ranked conference or journal.

Based on our analysis, we determined that PREDICT should not target overall system resiliency, as anticipated in the initial business plan, but rather it should focus on the resiliency of very large jobs. The current objective is therefore to focus on HPC users with very large jobs, the developers of HPC applications that (often) require very large jobs, and the developers of support libraries such as checkpoint–restart libraries. This is in contrast to the initial business plan, which targeted HPC system providers and system integrators.

In conclusion, the EuroLab4HPC BPP has helped us to test various hypothesis in the initial business model, leading to an evolution of the business model during the project. This process, however, has led to new hypotheses that are to be tested: (1) A small proportion (below 0.001%) of the overall server hours is lost due to uncorrected DRAM errors. This data is obtained for the MareNostrum supercomputer; it should be confirmed for similar HPC systems. (2) Some of HPC jobs required tens of thousands of server hours. This data is obtained for the MareNostrum supercomputer; it should be confirmed for similar HPC systems. (3) The large HPC jobs have to be protected with advanced resiliency mechanisms. The most obvious current choice would be the checkpoint–restart approach. (4) PREDICT can be used to improve resiliency approaches targeting large HPC jobs, e.g., it can be used to impact the checkpoint interval based on the failure probability. We plan to discuss this approach with researchers developing checkpoint–restart libraries in Europe. (5) Scientific publication of the PREDICT technology would increase the confidence of potential early adopters.