Introduction
The use of Machine Learning (ML) is now wide spread across may aspects of society and scientific domains are certainly no exception. The explosion of ML has brought many new ideas to these scientific domains. In the chemical sciences ML has been applied for many years to predict properties of molecules and screen datasets. More recently, ML has entered as an augmentation to the physics based calculations used in chemistry. This provides new approximations to expensive computational calculations in a fraction of the time.
In this article we will have a look a few examples of these methods and the code needed to run one method. Let's dive in!
What is a Machine Learning Potential?
In general we can consider this to be any ML model which attempts to replicate a potential energy function for a calculation. This definition can extend beyond the chemical domain, but for this article we will focus of potential energy functions related to chemical applications.
For many years, to accurately determine the energy and structure of a chemical system required expensive static quantum chemistry calculations. However, such calculations provide no information on the dynamics of the system unless very computationally intensive ab initio molecular dynamics could be performed. This leads to the following hierarchy of methods. The ideal case is the top left of the plot i.e. low cost high accuracy, but this region is bare.
The diagram is showing that to be as accurate as we can be costs more (money, energy and time). So, what could be done differently to try to remedy this? Enter the ML potential!
ML potentials can be evaluated rapidly at relatively low computational cost. This discounts the fact that training an ML potential in the first place is computationally intensive, but this can be, in some cases, a one off initial cost. ML potentials in principle can be trained against any of these underlying physics methods, but are commonly trained to quantum chemical data. This means that we can approximate quantum chemical accuracy (at least within a suitable chemical domain of the potential is trained for) at the cost of a much cheaper method. We can also apply this to simulations to gain dynamics.
Types of Machine Learning Potential
Many different research groups and companies have developed their own forms of ML potentials. There are far too many to comment on and provide examples for here. In this section we provide a brief overview of two kinds of ML methods and some features which have been used for these potentials, but refer the interested reader to the following references for more information. [1, 2, 3]
Some common ML models which are used for these potentials include Gaussian Processes (GP) and Neural Networks (NN).
Gaussian processes (GP) are stochastic processes. The model assumes that any sample of random variables from the process has a multivariate normal distribution. In practice this means that a GP represents a distribution over functions, where the random variables are the functions themselves and finite sample are evaluations of these functions. Gaussian processes have been used as ML potentials by several research teams. [4, 5, 6, 7] These models have been used widely in chemistry for both property prediction and as ML potentials. GP models have been successfully trained using SOAP descriptors and relative atomic position descriptors.
Neural networks are parametric models which have shown great utility for ML potentials. These models come in a variety of shapes and sizes. In the simplist form they comprise of stacks of neurons in layers which are fully connected acorss layers. Each neuron has activation function and each connection a weight.
One example of a potential of this form are the Behler-Parrinello potentials[8]. Another is the ANI family of potentials[9, 10]. There are also more complex architectures such as SchNet which utilize convolutional network architectures. One example of features which have been used successfully with the Behler-Parrinello and ANI Neural network potentials are symmetry functions.
Training Your Own Potentials
Many of the packages come with the ability to train and build your own potentials. Choosing the right potential can be a complex choice. The following article provides an overview of the differences, strengths and weaknesses of the models [1] and could be a useful place to start.
Two different examples you can look at from the neural network potentials are:
- SchNet Pack: This is a PyTorch implementation which incorporates approximate physics into the network
- ANI: This is also a PyTorch implementation which has an ensemble of DNN models.
Both of the above allow users to train their own potentials, though others are available.
Using a Neural Potential
A simple way to try out an ML potential is using some of the interfaces to larger programs such as LAMMPS, OpenMM or the ASE (Atomic Simulation Environment). Here is an example to utilize a pre-trained ANI ensemble model through the ASE (check out the official pages for more information).
You will need to install the following assuming you are using Conda to manage your python environments:
pip install --upgrade --user ase
conda install -c pytorch pytorch
conda install -c conda-forge torchani
# potentially the following may be required
# https://github.com/isayev/ASE_ANIWhen all are installed you can start by importing:
import torch
import torchani
import ase
from ase.optimize import BFGS
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")There are a choice of ANI models available ANI1x, ANI1ccx and ANI2x. You can select from any of these pre-trained models
#ANI1x
model = torchani.models.ANI1x(periodic_table_index=True,
model_index=None).to(device)
#ANIccx
model = torchani.models.ANI1ccx(periodic_table_index=True,
model_index=None).to(device)
#ANI2xb
model = torchani.models.ANI2x(periodic_table_index=True,
model_index=None).to(device)Note that you can use the model_index to select just one model from the ensemble e.g:
model = torchani.models.ANI1x(periodic_table_index=True,
model_index=1).to(device)There are 8 models in each ensemble to the best of my knowledge.
This means that you can compute the energy prediction for each model separately giving you access to the mean predicted energy (as would come from the model used as an ensemble) and the other moments of the distribution of predictions. This can be useful in assessing the models.
Once you have selected your model you can proceed to use it through the ASE. Below is an example for optimising Benzene.
molecule = ase.Atoms(["C", "C", "C", "C", "C", "C",
"H", "H", "H", "H", "H", "H", ],
positions=[
[-1.21099146e+00, -6.87241173e-01, 1.87502061e-05],
[-1.20062173e+00, 7.05208001e-01, 5.26593618e-05],
[-1.02652346e-02, -1.39235668e+00, -1.98984942e-05],
[ 1.03282322e-02, 1.39240814e+00, 2.27946892e-05],
[ 1.20069770e+00, -7.05153193e-01, 1.92150687e-05],
[ 1.21105704e+00, 6.87304535e-01, 6.48032992e-06],
[-2.15681312e+00, -1.22393749e+00, 1.37568185e-06],
[-2.13842880e+00, 1.25589904e+00, 1.02995211e-04],
[-1.83828666e-02, -2.47988187e+00, -8.20728367e-05],
[ 1.84284654e-02, 2.47994166e+00, 3.71134816e-06],
[ 2.13851946e+00, -1.25581634e+00, 7.55204642e-05],
[ 2.15687232e+00, 1.22402540e+00, -1.53103206e-06]],
calculator=model.ase())
opt = BFGS(molecule)
opt.run(fmax=1e-4)That's it! This is a very simple way to get to try out an ML potential for the first time. If after this you want to use a physics method to refine the structure you can easily do this with the ASE. Assuming you have quantum chemistry package installed that the ASE has a calculator for you can just swap the calculator over. An example using Psi4 (see a previous blog on Psi4 and the official Psi4 docs for more details on this.) is below:
from ase.calculators.psi4 import Psi4
psi4_calculator = Psi4(
method = "wb97x",
memory = "1GB",
basis = "6-31g*"
)
molecule.calc = psi4_calculator
opt = BFGS(molecule)
opt.run(fmax=1e-4)Summary
That's it for this introductory blog on ML potentials. This is a rapidly changing field with a range of improvements and new methods arriving all the time. Check out the reference list for some further reading on this area.
Hopefully you have found this useful. These are just some possibilities for getting started on trying these new methods. Make sure to have a look for new and updated verions.
As usual you will find a disclaimer for this article on my profile page https://medium.com/@james.l.mcdonagh/about
References
1 Choosing the right molecular machine learning potential. DOI: https://doi.org/10.1039/D1SC03564A (Edge Article) Chem. Sci., 2021, 12, 14396–14413
2, Perspective: Machine learning potentials for atomistic simulations. https://doi.org/10.1063/1.4966192 J. Chem. Phys. 145, 170901 (2016)
3 Machine Learning Force Fields. https://doi.org/10.1021/acs.chemrev.0c01111 Chem. Rev. 2021, 121, 16, 10142–10186
4 Gaussian Approximation Potentials: The Accuracy of Quantum Mechanics, without the Electrons. https://doi.org/10.1103/PhysRevLett.104.136403 Phys. Rev. Lett. 104, 136403
5 Machine Learning of Dynamic Electron Correlation Energies from Topological Atoms. https://doi.org/10.1021/acs.jctc.7b01157 J. Chem. Theory Comput. 2018, 14, 1, 216–224
6 Gaussian Process Regression Models for Predicting Atomic Energies and Multipole Moments https://doi.org/10.1021/acs.jctc.2c00731 J. Chem. Theory Comput. 2023, 19, 4, 1370–1380
7 Gaussian Process Regression for Materials and Molecules. https://doi.org/10.1021/acs.chemrev.1c00022 Chem. Rev. 2021, 121, 16, 10073–10141
8. Generalized Neural-Network Representation of High-Dimensional Potential-Energy Surfaces https://doi.org/10.1103/PhysRevLett.98.146401 Phys. Rev. Lett . 98, 146401
9. ANI-1: an extensible neural network potential with DFT accuracy at force field computational cost. Chemical science 8 (4), 3192-3203, 2017. https://doi.org/10.1039/C6SC05720A
10. Less is more: Sampling chemical space with active learning J. Chem. Phys. 148, 241733 (2018)