Learning from less: SINDy Surrogates in RL

This paper presents a method to create simpler versions of reinforcement learning (RL) environments using the Sparse Identification of Nonlinear Dynamics (SINDy) algorithm. The method was tested in OpenAI Gym's Mountain Car and Lunar Lander environments. Results show that these SINDy-based models can closely match the real environment's behavior while cutting down computational cost by 20–35%. With just 75 data points for Mountain Car and 1000 for Lunar Lander, the models achieved over 0.997 state-wise correlation, with very low prediction errors (as low as 3.11 × 10⁻⁶ for Mountain Car velocity and 1.42 × 10⁻⁶ for Lunar Lander position). RL agents trained in these simplified environments needed fewer steps to learn (65,075 vs. 100,000 for Mountain Car and 801,000 vs. 1,000,000 for Lunar Lander) and performed similarly to those trained in the full environments. This work offers a fast and accurate way to create interpretable models for use in model-based RL.