Nathaniel is a Vanderbilt University Computer Science Department graduate student and a research assistant at the Institute for Software Integrated Systems. He received a bachelor’s degree in physics and Chinese language from Trinity University in San Antonio, TX, in May 2021. He later completed an MS in data science at the Colorado School of Mines. His research interests lie at the intersection of online planning, decision-making under uncertainty, and cyber-physical systems.

Nathaniel Keplinger Publications

1. N. S. Keplinger, B. Luo, I. Bektas, Y. Zhang, K. H. Wray, A. Laszka, A. Dubey, and A. Mukhopadhyay, NS-Gym: Open-Source Simulation Environments and Benchmarks for Non-Stationary Markov Decision Processes. 2025.
   Bibtex Abstract

   @misc{keplinger2025nsgymopensourcesimulationenvironments,
     author = {Keplinger, Nathaniel S. and Luo, Baiting and Bektas, Iliyas and Zhang, Yunuo and Wray, Kyle Hollins and Laszka, Aron and Dubey, Abhishek and Mukhopadhyay, Ayan},
     title = {NS-Gym: Open-Source Simulation Environments and Benchmarks for Non-Stationary Markov Decision Processes},
     year = {2025},
     archiveprefix = {arXiv},
     contribution = {colab},
     eprint = {2501.09646},
     primaryclass = {cs.AI},
     url = {https://arxiv.org/abs/2501.09646}
   }
   

   In many real-world applications, agents must make sequential decisions in environments where conditions are subject to change due to various exogenous factors. These non-stationary environments pose significant challenges to traditional decision-making models, which typically assume stationary dynamics. Non-stationary Markov decision processes (NS-MDPs) offer a framework to model and solve decision problems under such changing conditions. However, the lack of standardized benchmarks and simulation tools has hindered systematic evaluation and advance in this field. We present NS-Gym, the first simulation toolkit designed explicitly for NS-MDPs, integrated within the popular Gymnasium framework. In NS-Gym, we segregate the evolution of the environmental parameters that characterize non-stationarity from the agent’s decision-making module, allowing for modular and flexible adaptations to dynamic environments. We review prior work in this domain and present a toolkit encapsulating key problem characteristics and types in NS-MDPs. This toolkit is the first effort to develop a set of standardized interfaces and benchmark problems to enable consistent and reproducible evaluation of algorithms under non-stationary conditions. We also benchmark six algorithmic approaches from prior work on NS-MDPs using NS-Gym. Our vision is that NS-Gym will enable researchers to assess the adaptability and robustness of their decision-making algorithms to non-stationary conditions.