Optimization acceleration techniques such as momentum play a key role in state-of-the-art machine learning algorithms. Recently. generic vector sequence extrapolation techniques. such as regularized nonlinear acceleration (RNA) of Scieur et al. https://www.roneverhart.com/M-M-s-Peanut-Bulk-25LB-Case/