Baiting Luo is a graduate student in the Department of Electrical Engineering and Computer Science at Vanderbilt University and works as a research assistant at the Institute for Software Integrated Systems. He received his M.S. degree in Computer Engineering from Northwestern University in June 2021 and his Bachelor’s degree in Computer Engineering and Computer Science from Rensselaer Polytechnic Institute in 2019.

Baiting Luo Publications

1. S. Ramakrishna, B. Luo, C. B. Kuhn, G. Karsai, and A. Dubey, ANTI-CARLA: An Adversarial Testing Framework for Autonomous Vehicles in CARLA, in 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC), 2022, pp. 2620–2627.
   Bibtex Abstract DOI PDF

   @inproceedings{ramakrishna2022anticarla,
     author = {Ramakrishna, Shreyas and Luo, Baiting and Kuhn, Christopher B. and Karsai, Gabor and Dubey, Abhishek},
     booktitle = {2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC)},
     title = {ANTI-CARLA: An Adversarial Testing Framework for Autonomous Vehicles in CARLA},
     year = {2022},
     volume = {},
     number = {},
     pages = {2620-2627},
     doi = {10.1109/ITSC55140.2022.9921776}
   }
   

   Despite recent advances in autonomous driving systems, accidents such as the fatal Uber crash in 2018 show these systems are still susceptible to edge cases. Such systems need to be thoroughly tested and validated before being deployed in the real world to avoid such events. Testing in open-world scenarios can be difficult, time-consuming, and expensive. These challenges can be addressed by using driving simulators such as CARLA instead. A key part of such tests is adversarial testing, in which the goal is to find scenarios that lead to failures of the given system. While several independent efforts in adversarial testing have been made, a well-established testing framework that enables adaptive stress testing has yet to be made available for CARLA. We therefore propose ANTI-CARLA, an adversarial testing framework in CARLA. The operating conditions in which a given system should be tested are specified in a scenario description language. The framework offers an adversarial search mechanism that searches for operating conditions that will fail the tested system. In this way, ANTI-CARLA extends the CARLA simulator with the capability of performing adversarial testing on any given driving pipeline. We use ANTI-CARLA to test the driving pipeline trained with Learning By Cheating (LBC) approach. The simulation results demonstrate that ANTI-CARLA can effectively and automatically find a range of failure cases despite LBC reaching an accuracy of 100% in the CARLA benchmark.

2. S. Ramakrishna, B. Luo, Y. Barve, G. Karsai, and A. Dubey, Risk-Aware Scene Sampling for Dynamic Assurance of Autonomous Systems, in 2022 IEEE International Conference on Assured Autonomy (ICAA) (ICAA’22), virtual, Puerto Rico, 2022.
   Bibtex Abstract PDF

   @inproceedings{ICAA2022,
     author = {Ramakrishna, Shreyas and Luo, Baiting and Barve, Yogesh and Karsai, Gabor and Dubey, Abhishek},
     title = {{Risk-Aware} Scene Sampling for Dynamic Assurance of Autonomous Systems},
     booktitle = {2022 IEEE International Conference on Assured Autonomy (ICAA) (ICAA'22)},
     address = {virtual, Puerto Rico},
     days = {22},
     month = mar,
     tag = {ai4cps},
     year = {2022},
     keywords = {Cyber-Physical Systems; Dynamic Assurance; Dynamic Risk; High-Risk Scenes;
                    Bow-Tie Diagram; Hazards}
   }
   

   Autonomous Cyber-Physical Systems must often operate under uncertainties like sensor degradation and distribution shifts in the operating environment, thus increasing operational risk. Dynamic Assurance of these systems requires augmenting runtime safety components like out-of-distribution detectors and risk estimators. Designing these safety components requires labeled data from failure conditions and risky corner cases that fail the system. However, collecting real-world data of these high-risk scenes can be expensive and sometimes not possible. To address this, there are several scenario description languages with sampling capability for generating synthetic data from simulators to replicate the scenes that are not possible in the real world. Most often, simple search-based techniques like random search and grid search are used as samplers. But we point out three limitations in using these techniques. First, they are passive samplers, which do not use the feedback of previous results in the sampling process. Second, the variables to be sampled may have constraints that need to be applied. Third, they do not balance the tradeoff between exploration and exploitation, which we hypothesize is needed for better coverage of the search space. We present a scene generation workflow with two samplers called Random Neighborhood Search (RNS) and Guided Bayesian Optimization (GBO). These samplers extend the conventional random search and Bayesian Optimization search with the limitation points. We demonstrate our approach using an Autonomous Vehicle case study in CARLA simulation. To evaluate our samplers, we compared them against the baselines of random search, grid search, and Halton sequence search.