Baiting Luo is a graduate student in the Department of Electrical Engineering and Computer Science at Vanderbilt University and works as a research assistant at the Institute for Software Integrated Systems. He received his M.S. degree in Computer Engineering from Northwestern University in June 2021 and his Bachelor’s degree in Computer Engineering and Computer Science from Rensselaer Polytechnic Institute in 2019.

Baiting Luo Publications

A. Pettet, Y. Zhang, B. Luo, K. Wray, H. Baier, A. Laszka, A. Dubey, and A. Mukhopadhyay, Decision Making in Non-Stationary Environments with Policy-Augmented Search, Proceedings of the 23rd Conference on Autonomous Agents and MultiAgent Systems, AAMAS 2024, Auckland, New Zealand. 2024.
```
@misc{pettet2024decision,
  title = {Decision Making in Non-Stationary Environments with Policy-Augmented Search},
  author = {Pettet, Ava and Zhang, Yunuo and Luo, Baiting and Wray, Kyle and Baier, Hendrik and Laszka, Aron and Dubey, Abhishek and Mukhopadhyay, Ayan},
  year = {2024},
  booktitle = {Proceedings of the 23rd Conference on Autonomous Agents and MultiAgent Systems, {AAMAS} 2024, Auckland, New Zealand},
  location = {Auckland, New Zealand},
  numpages = {9}
}
```
Sequential decision-making under uncertainty is present in many important problems. Two popular approaches for tackling such problems are reinforcement learning and online search (e.g., Monte Carlo tree search). While the former learns a policy by interacting with the environment (typically done before execution), the latter uses a generative model of the environment to sample promising action trajectories at decision time. Decision-making is particularly challenging in non-stationary environments, where the environment in which an agent operates can change over time. Both approaches have shortcomings in such settings – on the one hand, policies learned before execution become stale when the environment changes and relearning takes both time and computational effort. Online search, on the other hand, can return sub-optimal actions when there are limitations on allowed runtime. In this paper, we introduce \textitPolicy-Augmented Monte Carlo tree search (PA-MCTS), which combines action-value estimates from an out-of-date policy with an online search using an up-to-date model of the environment. We prove theoretical results showing conditions under which PA-MCTS selects the one-step optimal action and also bound the error accrued while following PA-MCTS as a policy. We compare and contrast our approach with AlphaZero, another hybrid planning approach, and Deep Q Learning on several OpenAI Gym environments. Through extensive experiments, we show that under non-stationary settings with limited time constraints, PA-MCTS outperforms these baselines.
B. Luo, Y. Zhang, A. Mukhopadhyay, and A. Dubey, Act as You Learn: Adaptive Decision-Making in Non-Stationary Markov Decision Processes, in Proceedings of the 23rd Conference on Autonomous Agents and MultiAgent Systems, AAMAS 2024, Auckland, New Zealand, 2024.
```
@inproceedings{baiting2024AAMAS,
  title = {Act as You Learn: Adaptive Decision-Making in Non-Stationary Markov Decision Processes},
  author = {Luo, Baiting and Zhang, Yunuo and Mukhopadhyay, Ayan and Dubey, Abhishek},
  year = {2024},
  booktitle = {Proceedings of the 23rd Conference on Autonomous Agents and MultiAgent Systems, {AAMAS} 2024, Auckland, New Zealand},
  location = {Auckland, New Zealand},
  numpages = {9}
}
```
A fundamental (and largely open) challenge in sequential decision- making is dealing with non-stationary environments, where exoge- nous environmental conditions change over time. Such problems are traditionally modeled as non-stationary Markov decision pro- cesses (NSMDP), which can account for a non-stationary environ- mental distribution during planning. However, existing approaches for decision-making in NSMDPs have two major shortcomings: first, they assume that the updated environmental dynamics at the current time are known (although future dynamics can change); and second, planning is largely pessimistic, i.e., the agent acts “safely” to account for the non-stationary evolution of the environment. We argue that both these assumptions are invalid in practice—updated environmental conditions are rarely known, and as the agent inter- acts with the environment, it can learn about the updated dynamics and avoid being pessimistic, at least in states whose dynamics it is confident about. We present a heuristic search algorithm called Adaptive Monte Carlo Tree Search (ADA-MCTS) that addresses these challenges. We show that the agent can learn the updated dynamics of the environment over time and then act as it learns, i.e., if the agent is in a region of the state space about which it has updated knowledge, it can avoid being pessimistic. To quantify “updated knowledge,” we disintegrate the aleatoric and epistemic uncertainty in the agent’s updated belief and show how the agent can use these estimates for decision-making. We compare the proposed approach with the multiple state-of-the-art approaches in decision-making across multiple well-established open-source problems and empirically show that our approach is faster and highly adaptive without sacrificing safety.
B. Luo, S. Ramakrishna, A. Pettet, C. Kuhn, G. Karsai, and A. Mukhopadhyay, Dynamic Simplex: Balancing Safety and Performance in Autonomous Cyber Physical Systems, in Proceedings of the ACM/IEEE 14th International Conference on Cyber-Physical Systems (with CPS-IoT Week 2023), New York, NY, USA, 2023, pp. 177–186.
```
@inproceedings{baiting2023iccps,
  title = {Dynamic Simplex: Balancing Safety and Performance in Autonomous Cyber Physical Systems},
  author = {Luo, Baiting and Ramakrishna, Shreyas and Pettet, Ava and Kuhn, Christopher and Karsai, Gabor and Mukhopadhyay, Ayan},
  year = {2023},
  booktitle = {Proceedings of the ACM/IEEE 14th International Conference on Cyber-Physical Systems (with CPS-IoT Week 2023)},
  location = {San Antonio, TX, USA},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  series = {ICCPS '23},
  pages = {177--186},
  doi = {10.1145/3576841.3585934},
  isbn = {9798400700361},
  url = {https://doi.org/10.1145/3576841.3585934},
  numpages = {10}
}
```
Learning Enabled Components (LEC) have greatly assisted cyber-physical systems in achieving higher levels of autonomy. However, LEC’s susceptibility to dynamic and uncertain operating conditions is a critical challenge for the safety of these systems. Redundant controller architectures have been widely adopted for safety assurance in such contexts. These architectures augment LEC "performant" controllers that are difficult to verify with "safety" controllers and the decision logic to switch between them. While these architectures ensure safety, we point out two limitations. First, they are trained offline to learn a conservative policy of always selecting a controller that maintains the system’s safety, which limits the system’s adaptability to dynamic and non-stationary environments. Second, they do not support reverse switching from the safety controller to the performant controller, even when the threat to safety is no longer present. To address these limitations, we propose a dynamic simplex strategy with an online controller switching logic that allows two-way switching. We consider switching as a sequential decision-making problem and model it as a semi-Markov decision process. We leverage a combination of a myopic selector using surrogate models (for the forward switch) and a non-myopic planner (for the reverse switch) to balance safety and performance. We evaluate this approach using an autonomous vehicle case study in the CARLA simulator using different driving conditions, locations, and component failures. We show that the proposed approach results in fewer collisions and higher performance than state-of-the-art alternatives.
S. Ramakrishna, B. Luo, C. B. Kuhn, G. Karsai, and A. Dubey, ANTI-CARLA: An Adversarial Testing Framework for Autonomous Vehicles in CARLA, in 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC), 2022, pp. 2620–2627.
```
@inproceedings{ramakrishna2022anticarla,
  author = {Ramakrishna, Shreyas and Luo, Baiting and Kuhn, Christopher B. and Karsai, Gabor and Dubey, Abhishek},
  booktitle = {2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC)},
  title = {ANTI-CARLA: An Adversarial Testing Framework for Autonomous Vehicles in CARLA},
  year = {2022},
  volume = {},
  number = {},
  pages = {2620-2627},
  doi = {10.1109/ITSC55140.2022.9921776}
}
```
Despite recent advances in autonomous driving systems, accidents such as the fatal Uber crash in 2018 show these systems are still susceptible to edge cases. Such systems need to be thoroughly tested and validated before being deployed in the real world to avoid such events. Testing in open-world scenarios can be difficult, time-consuming, and expensive. These challenges can be addressed by using driving simulators such as CARLA instead. A key part of such tests is adversarial testing, in which the goal is to find scenarios that lead to failures of the given system. While several independent efforts in adversarial testing have been made, a well-established testing framework that enables adaptive stress testing has yet to be made available for CARLA. We therefore propose ANTI-CARLA, an adversarial testing framework in CARLA. The operating conditions in which a given system should be tested are specified in a scenario description language. The framework offers an adversarial search mechanism that searches for operating conditions that will fail the tested system. In this way, ANTI-CARLA extends the CARLA simulator with the capability of performing adversarial testing on any given driving pipeline. We use ANTI-CARLA to test the driving pipeline trained with Learning By Cheating (LBC) approach. The simulation results demonstrate that ANTI-CARLA can effectively and automatically find a range of failure cases despite LBC reaching an accuracy of 100% in the CARLA benchmark.
S. Ramakrishna, B. Luo, Y. Barve, G. Karsai, and A. Dubey, Risk-Aware Scene Sampling for Dynamic Assurance of Autonomous Systems, in 2022 IEEE International Conference on Assured Autonomy (ICAA) (ICAA’22), virtual, Puerto Rico, 2022.
```
@inproceedings{ICAA2022,
  author = {Ramakrishna, Shreyas and Luo, Baiting and Barve, Yogesh and Karsai, Gabor and Dubey, Abhishek},
  title = {{Risk-Aware} Scene Sampling for Dynamic Assurance of Autonomous Systems},
  booktitle = {2022 IEEE International Conference on Assured Autonomy (ICAA) (ICAA'22)},
  address = {virtual, Puerto Rico},
  days = {22},
  month = mar,
  tag = {ai4cps},
  year = {2022},
  keywords = {Cyber-Physical Systems; Dynamic Assurance; Dynamic Risk; High-Risk Scenes;
                 Bow-Tie Diagram; Hazards}
}
```
Autonomous Cyber-Physical Systems must often operate under uncertainties like sensor degradation and distribution shifts in the operating environment, thus increasing operational risk. Dynamic Assurance of these systems requires augmenting runtime safety components like out-of-distribution detectors and risk estimators. Designing these safety components requires labeled data from failure conditions and risky corner cases that fail the system. However, collecting real-world data of these high-risk scenes can be expensive and sometimes not possible. To address this, there are several scenario description languages with sampling capability for generating synthetic data from simulators to replicate the scenes that are not possible in the real world. Most often, simple search-based techniques like random search and grid search are used as samplers. But we point out three limitations in using these techniques. First, they are passive samplers, which do not use the feedback of previous results in the sampling process. Second, the variables to be sampled may have constraints that need to be applied. Third, they do not balance the tradeoff between exploration and exploitation, which we hypothesize is needed for better coverage of the search space. We present a scene generation workflow with two samplers called Random Neighborhood Search (RNS) and Guided Bayesian Optimization (GBO). These samplers extend the conventional random search and Bayesian Optimization search with the limitation points. We demonstrate our approach using an Autonomous Vehicle case study in CARLA simulation. To evaluate our samplers, we compared them against the baselines of random search, grid search, and Halton sequence search.

ScopeLab - Design, Operation and Optimization of Smart Cyber-Physical Systems

Baiting Luo Publications

ScopeLab
- Design, Operation and Optimization of Smart Cyber-Physical Systems