Lowering the force and stress mae/mrse

[gly123456-phy]

Hello,
I am training a model for 2x2x2 BaTiO3 system. My result shows that energy_mae and energy_mrse are fine (< 1 meV/ atom); However, the force_mae/force_mrse are a bit higher (0.06 eV/A and 0.09 eV/A); the stress_mae/stress_mrse are also higher (0.006 eV/A^3 and 0.009 eV/A^3); any recommendations that I can lower force and stress errors?

Thanks a lot in advance!

[Linux-cpp-lisp]

Just as a general comment, we have repeatedly heard about cases where people's underlying DFT labels for stress were not sufficiently converged to fit on successfully, even if their energy and forces were OK. It's quite important when trying to train on or get good stress predictions to double check the quality of the underlying labels.

[gly123456-phy]

Thanks. What are suggested parameters to tune so that I can get better forces and stress?

[apoletayev]

if using VASP for the underlying DFT, convergence to 1e-6 eV or better is a must, and prec=accurate.
For other DFT engines, I imagine something comparable, but have not tested in detail.

[gly123456-phy]

if using VASP for the underlying DFT, convergence to 1e-6 eV or better is a must, and prec=accurate. For other DFT engines, I imagine something comparable, but have not tested in detail.

Thanks! Actually my DFT is pretty accurate. I used Quantum Espresso, and the electronic step is converged to 1e-8 Ry; so that is well below 1e-6 eV. My training data is collected as 400 consecutive frames from an AIMD trajectory, Not sure if the data too "correlated."

[apoletayev]

Whether your training data is "too" correlated or "not too" correlated depends, in one sentence, on whether the dynamics of interest are captured in the training dataset. However, you cannot tell whether your training dataset is "too" correlated from the validation or testing errors sampling the same distribution of data as the training set itself. Here we touch the thorny question of whether validation / test errors in training are representative of errors on properly unseen data that may be ever so slightly mismatched versus the distribution of the data encountered in training (and from which the validation / test sets are sampled). To retroactively benchmark this, you can run a trajectory with your machine learning potential, then pick frames out of that trajectory, compute them as single points with DFT and look at those errors. In most cases I've encountered, these retroactive errors were passable but substantially higher than validation or test errors from training. In my personal view, this is the real error of your machine learning potential. Here is a paper that discusses this problem in detail: https://pubs.rsc.org/en/content/articlelanding/2020/cc/d0cc03512b If I had to make a guess, in one sentence, consecutive frames from one AIMD trajectory at one temperature are probably too correlated.

[cw-tan]

For model hyperparameters, see https://nequip.readthedocs.io/en/latest/api/nequip_model.html#nequip.model.NequIPGNNModel. Generally, l_max and num_features are good to tune for better accuracies. Also good to read https://nequip.readthedocs.io/en/latest/guide/configuration/model.html.

But the answer to your question is extremely open-ended. Maybe it's about training dynamics, maybe it's about model hyperparameters, maybe it's about your data quality, maybe errors of that magnitude are intrinsic to your dataset if it contains various far-from-equilibrium configurations that have high energies. It is unlikely for us to be able to provide any meaningfully useful advice that can help for your specific situation without any concrete information of your system/dataset, model hyperparameters, training hyperparameters, logged training curves (loss, train metric, validation metric), etc. But if there's something specific you can share about your set up, or some specific aspect of training you're unsure about, we'll try our best to answer and update our docs for others that may have similar questions. We're always trying to improve our documentation and the default parameters we provide.

[gly123456-phy]

Thanks a lot! That's very helpful. Let me first play with l_max and num_features and work on your recommended guide to see if there will be any improvements. Will update you soon!

[gly123456-phy]

Hello @cw-tan
Sorry for the very late update. I played with many training parameters: I tried increasing cutoff_radius from 5.0 Angstrom to 7.0 Angstrom, increasing l_max from 1 to 3, increasing num_features from 32 to 48, num_layers from 4 to 5, radial_mlp_depth from 2 to 3; I even tried tuning the loss of forces from 1.0 to 5.0 (still keep 1.0 for energies) in the training_module. However, none of them worked: the forces_mae and forces_rmse are still around 50 meV/A and 70 meV/A.

I doubt whether my training data is good enough. I constructed a 2x2x2 BTO supercell for the tetragonal phase and run an AIMD calculation with NVT thermostats at 100 K (Note that I haven't included virial in the current training). Then I just took 400 structures consecutively from the AIMD trajectory. I am sure my AIMD calculation is accurate enough: electronic step convergence is far less than 1e-6 eV, and my k-point mesh is 4x4x4 dense enough, and my cutoff energy is 50 Ry which is good for my norm conserving PP.

Do you think if there is any problem with my training data? Let me know if any other detail about my data preparation and training is needed to better help diagnose the problem. Thanks a lot for your assistance in advance!