Assuring Cyber-Physical Systems with Learning Enabled Components

In recent years, AI based components are being heavily used in CPS. Despite their impressive capability, using them in safety critical applications is challenging because they learn from training data, and subtle changes in the images during testing could cause these components to predict erroneously, In addition, testing and verifying these components is complex and sometimes not possible and as a result safety and assurance case development of systems using these components is complicated. The group in collaboration with Prof. Gabor Karsai, Prof. Taylor Johnson, Prof. Xenofon Koutsoukos, Prof. Ted Bapty and Prof. Janos Sztipanovits have been focusing on methods to identify anomalies and recover from failures as well as develop system level safety assurance arguments. Till now, the SCOPE-Lab research group have developed a methodology to use a class of variational autoencoder called Beta-VAE in combination with dissimilarity metrics like Kullback-Leibler divergence to perform anomaly detection on the input data streams. Once an anomaly is detected we use a weighted simplex strategy to transition to a safe controller. Instead of using only a single control output (as in Simplex Architecture), we designed a weighted ensemble of the two control outputs. The weights are computed dynamically to improve the balance of safety versus performance of the system. We are also working on a methodology to semi-automate the generation of assurance cases for CPS with AI components. We have also built a test-bed called Deep NN-Car for experimentation and validation of these approaches.


  1. S. Ramakrishna, C. Hartsell, M. P. Burruss, G. Karsai, and A. Dubey, Dynamic-Weighted Simplex Strategy for Learning Enabled Cyber Physical Systems, Special Issue on the 2019 IEEE Symposium on Real-time Computing ISORC, 2020.
  2. V. K. Sundar, S. Ramakrishna, Z. Rahiminasab, A. Easwaran, and A. Dubey, Out-of-Distribution Detection in Multi-Label Datasets using Latent Space of β-VAE, in 2020 Workshop on Assured Autonomous Systems (WAAS), 2020.
  3. C. Hartsell, N. Mahadevan, H. Nine, T. Bapty, A. Dubey, and G. Karsai, Workflow Automation for Cyber Physical System Development Processes, in 2020 IEEE Workshop on Design Automation for CPS and IoT (DESTION), 2020.
  4. S. Ramakrishna, C. Hartsell, A. Dubey, P. Pal, and G. Karsai, A Methodology for Automating Assurance Case Generation, in Thirteenth International Tools and Methods of Competitive Engineering Symposium (TMCE 2020), 2020.
  5. C. Hartsell et al., Model-based design for CPS with learning-enabled components, in Proceedings of the Workshop on Design Automation for CPS and IoT, DESTION@CPSIoTWeek 2019, Montreal, QC, Canada, 2019, pp. 1–9.
  6. C. Hartsell, N. Mahadevan, S. Ramakrishna, A. Dubey, T. Bapty, and G. Karsai, A CPS toolchain for learning-based systems: demo abstract, in Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems, ICCPS 2019, Montreal, QC, Canada, 2019, pp. 342–343.
  7. M. P. Burruss, S. Ramakrishna, G. Karsai, and A. Dubey, DeepNNCar: A Testbed for Deploying and Testing Middleware Frameworks for Autonomous Robots, in IEEE 22nd International Symposium on Real-Time Distributed Computing, ISORC 2019, Valencia, Spain, May 7-9, 2019, 2019, pp. 87–88.
  8. S. Ramakrishna et al., Augmenting Learning Components for Safety in Resource Constrained Autonomous Robots, in IEEE 22nd International Symposium on Real-Time Distributed Computing, ISORC 2019, Valencia, Spain, May 7-9, 2019, 2019, pp. 108–117.
  9. C. Hartsell et al., CPS Design with Learning-Enabled Components: A Case Study, in Proceedings of the 30th International Workshop on Rapid System Prototyping, RSP 2019, New York, NY, USA, October 17-18, 2019, 2019, pp. 57–63.