Understanding model uncertainty key to fabricating future materials

Tackling modern technological challenges sometimes necessitates discovering materials with novel properties like ionic conductivity for high-capacity batteries or high thermal conductivity for silicon chips.

Designing and experimenting with new materials can be costly, so machine learning (ML) models are used to predict material properties. However, simulation predictions are affected by various sources of uncertainties. Understanding the uncertainty is crucial for evaluating the model’s output. Varivoda et al. compared the uncertainty quantification (UQ) of different graph neural network-based ML models that screen properties to guide material manufacturing and testing.

“UQ, in its most basic sense, is a measure of how confident a model is for a given prediction,” said author Jianjun Hu. “In machine learning, it is beneficial for a model to not only provide relatively accurate predictions but also understand where these predictions may be wrong. This allows for model researchers, especially in material discovery applications where model predictions are used to direct experimental design, to estimate model imprecision and avoid wasting valuable time and resources on uncertain outputs.”

The authors compared four representative approaches used in UQ for graph neural networks and compared their uncertainty performance across multiple metrics. They found that, depending on the operational constraints, each UQ method had its own merits.

“There is no ‘best’ method that can very efficiently estimate model prediction for every dataset and every underlying model,” said Hu. “Each of these classes of models has strengths and weaknesses and practitioners should test multiple models to achieve the best performance.”

Materials science researchers can save time and money by using the authors’ open-source UQ code to screen material properties.

Source: “Materials property prediction with uncertainty quantification: A benchmark study,” by Daniel Varivoda, Rongzhi Dong, Sadman Sadeed Omee, and Jianjun Hu, Applied Physics Reviews (2023). The article can be accessed at https://doi.org/10.1063/5.0133528 .