Nathaniel Keplinger is a Graduate Research Assistant at Scope Lab developing theoretical foundations and practical tools for non-stationary Markov decision processes (MDPs). He is lead developer of the NS-Gym toolkit, which provides a standardized environment for benchmarking and testing algorithms designed for dynamic, changing systems.
Non-stationary MDPs capture the reality that many decision problems change over time—traffic patterns evolve, system parameters shift, and the optimal policy adapts. Nathaniel’s work on NS-Gym enables the broader research community to develop and evaluate algorithms that can handle these non-stationary challenges, advancing the state of the art in adaptive decision-making.