Dynamic Simplex: Balancing Safety and Performance in Autonomous Cyber Physical Systems

Baiting Luo, Shreyas Ramakrishna, Ava Pettet, Christopher Kuhn, abhishek dubey, Gabor Karsai, Ayan Mukhopadhyay

Proceedings of the ACM/IEEE 14th International Conference on Cyber-Physical Systems (with CPS-IoT Week 2023) 2023 ICCPS '23

PDF DOI

Why This Matters

Cyber-physical systems often operate in environments that change unpredictably due to weather, equipment failures, or other factors. Approaches that depend entirely on pre-trained policies fail to adapt, while pure online planning is computationally expensive. This work is innovative because it provides a principled framework for deciding when and how to augment learned policies with online planning, enabling systems to maintain safe and efficient operation despite environmental changes.

What We Did

This paper addresses the problem of learning and adapting decision-making policies in cyber-physical systems when the operating environment changes. The approach develops a framework for determining when to update from a learned policy to online planning, and how to combine historical knowledge with new information about changed conditions. The work includes theoretical analysis of how policy performance degrades with environmental changes and provides algorithms for adaptation with bounded error.

Key Results

The theoretical analysis shows conditions under which a learned policy remains sufficiently accurate despite environmental changes, and quantifies the error incurred when policy updates are needed. Experimental validation demonstrates the approach's ability to maintain system performance when operating conditions shift, outperforming both pure offline learning and pure online planning in uncertain environments. The work provides a practical methodology for policy adaptation in cyber-physical systems.

Full Abstract

Cite This Paper

@inproceedings{baiting2023iccps,
  author = {Luo, Baiting and Ramakrishna, Shreyas and Pettet, Ava and Kuhn, Christopher and dubey, abhishek and Karsai, Gabor and Mukhopadhyay, Ayan},
  booktitle = {Proceedings of the ACM/IEEE 14th International Conference on Cyber-Physical Systems (with CPS-IoT Week 2023)},
  title = {Dynamic Simplex: Balancing Safety and Performance in Autonomous Cyber Physical Systems},
  year = {2023},
  address = {New York, NY, USA},
  pages = {177--186},
  publisher = {Association for Computing Machinery},
  series = {ICCPS '23},
  acceptance = {25.6},
  abstract = {Learning Enabled Components (LEC) have greatly assisted cyber-physical systems in achieving higher levels of autonomy. However, LEC's susceptibility to dynamic and uncertain operating conditions is a critical challenge for the safety of these systems. Redundant controller architectures have been widely adopted for safety assurance in such contexts. These architectures augment LEC "performant" controllers that are difficult to verify with "safety" controllers and the decision logic to switch between them. While these architectures ensure safety, we point out two limitations. First, they are trained offline to learn a conservative policy of always selecting a controller that maintains the system's safety, which limits the system's adaptability to dynamic and non-stationary environments. Second, they do not support reverse switching from the safety controller to the performant controller, even when the threat to safety is no longer present. To address these limitations, we propose a dynamic simplex strategy with an online controller switching logic that allows two-way switching. We consider switching as a sequential decision-making problem and model it as a semi-Markov decision process. We leverage a combination of a myopic selector using surrogate models (for the forward switch) and a non-myopic planner (for the reverse switch) to balance safety and performance. We evaluate this approach using an autonomous vehicle case study in the CARLA simulator using different driving conditions, locations, and component failures. We show that the proposed approach results in fewer collisions and higher performance than state-of-the-art alternatives.},
  contribution = {colab},
  doi = {10.1145/3576841.3585934},
  isbn = {9798400700361},
  location = {San Antonio, TX, USA},
  numpages = {10},
  url = {https://doi.org/10.1145/3576841.3585934},
  keywords = {cyber-physical systems, policy learning, online planning, adaptive control, environmental changes, decision-making, learned models, system resilience}
}
Quick Info
Year 2023
Series ICCPS '23
DOI 10.1145/3576841.3585934
Keywords
cyber-physical systems policy learning online planning adaptive control environmental changes decision-making learned models system resilience
Research Areas
CPS ML for CPS POMDP
Search Tags

Dynamic, Simplex, Balancing, Safety, Performance, Autonomous, Cyber, Physical, Systems, cyber-physical systems, policy learning, online planning, adaptive control, environmental changes, decision-making, learned models, system resilience, CPS, ML for CPS, POMDP, 2023, Luo, Ramakrishna, Pettet, Kuhn, dubey, Karsai, Mukhopadhyay, ICCPS23