AI for Cyber-Physical Systems

Context: The fundamental advantage of AI methods is their ability to handle high-dimensional state-space and learn decision procedures/control algorithms from data rather than models. This is critical as real-world state spaces are complex, dynamic, and hard to model. Our goal is to bridge this gap and learn an abstract representation of these state spaces and develop decision procedures for use in the different research verticals we investigate - proactive emergency response systems, transit management systems, and electric power grids. However, the challenge is that AI components learn from training data, which may not necessarily cover the real-world distribution. Further, testing and verifying these components is complex and sometimes not possible. Our work in this area, focuses on the development of the decision procedures to be used in the system along with runtime monitors, and assurance cases to show that the system at runtime will be safe if there is a fault in the component of the system (software, hardware, or AI). We deploy these procedures in the research domains we work in.

Innovation and Research Products:

  1. ReSonAte - We have designed a dynamic assurance approach called ReSonAte that computes the likelihood of unsafe conditions or system failures considering the safety requirements, assumptions made at design time, past failures in a given operating context, and the likelihood of system component failures. The system has been demonstrated in simulations using two separate autonomous system simulations: CARLA and an unmanned underwater vehicle. The system was evaluated across 600 separate simulation scenes where we tested scenes with distribution shifts, component failures, and a high likelihood of collisions (based on past observations). Through the tests, we were able to show that our methodology has a precision of 73% and a recall of 79%. We are currently working on methods to dynamically estimate and learn the conditional probabilities and improve the precision values. On average, the framework takes 0.3 milliseconds for the computation of risk scores.

  2. Assurance Monitors - While ReSonAte monitors the safety assurance at the system-level, we need monitors at component levels to detect its anomalous behavior. Monitors to detect anomalies like data validity, pre-condition, and post-condition failures, user-code failure have been designed for conventional CPS, but a LEC based CPS would require complex monitors or assurance monitors to detect OOD. For this, we have designed an assurance monitor using a 𝛃-Variational Autoencoder, which can detect if an input (e.g. image) to a LEC is OOD or not. Besides, it can also diagnose the precise changes (e.g. brightness, blurriness, occlusion, weather change, etc.) in the input that caused the input to be OOD. Conceptually, we generate an interpretable latent representation for inputs using the 𝛃-VAE and then perform a correspondence between the latent units and generative factors to perform a latent space-based OOD detection and diagnosis. The disentanglement based diagnosis capability of the 𝛃-VAE monitor is the key innovation of this work, and our analysis shows it can be utilized as a multi-class classifier for multi-label datasets.

  3. Runtime Recovery Procedures - We have also developed runtime recovery procedures that manage the health of the system by using the system design information including the system information flow, requirement models and function decomposition models, temporal failure propagation graphs at runtime to identify the problems and recover the system by solving a dynamic constraint problem at runtime. The goal of the runtime problem is to identify the optimal component configuration that can provide the lowest risk of subsequent failures given the currently available resources and environment information.

  4. DeepNNCar - to test the efficacy of our methods on embedded systems, we have developed a low-cost research testbed that was designed in our lab. It is built upon the chassis of Traxxas Slash 2WD 1/10 Scale RC car, and is mounted with a USB forward-looking camera, IR- optocoupler, and a 2D LIDAR. The speed and steer for the robot are controlled using pulse-width modulation (PWM), by varying the duty cycle. Further, for autonomous driving, the robot uses a modified NVIDIA DAVE-II CNN model that takes in the front camera image and speed to predict the steering. More information about the car is available in this Medium Article. Videos of DeepNNCar with different controllers is available here.

  5. Finally, one of the interesting aspects of this work is the focus on the design of the decision procedures and generative predictive models that help us design the decision procedures. We have been applying this across the three research verticals of public transit, emergency response and power systems

Follow ups : Further information is available at following links.

  1. DeepNNCar: A Testbed for Autonomous Algorithms
  2. Deep NN Car Repository
  3. ReSonAte: A Runtime Risk Assessment Framework for Autonomous Systems

Publications in this area

  1. G. Pettet, A. Mukhopadhyay, M. J. Kochenderfer, and A. Dubey, Hierarchical Planning for Dynamic Resource Allocation in Smart and Connected Communities, ACM Trans. Cyber-Phys. Syst., 2021.
  2. Y. Senarath, A. Mukhopadhyay, S. Vazirizade, hemant Purohit, S. Nannapaneni, and A. Dubey, Practitioner-Centric Approach for Early Incident Detection Using Crowdsourced Data for Emergency Services, in 21st IEEE International Conference on Data Mining (ICDM 2021), 2021.
  3. S. Singla, A. Mukhopadhyay, M. Wilbur, T. Diao, V. Gajjewar, A. Eldawy, M. Kochenderfer, R. Shachter, and A. Dubey, WildfireDB: An Open-Source Dataset ConnectingWildfire Spread with Relevant Determinants, in 35th Conference on Neural Information Processing Systems (NeurIPS 2021) Track on Datasets and Benchmarks, 2021.
  4. M. Wilbur, A. Mukhopadhyay, S. Vazirizade, P. Pugliese, A. Laszka, and A. Dubey, Energy and Emission Prediction for Mixed-Vehicle Transit Fleets Using Multi-Task and Inductive Transfer Learning, in Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 2021.
  5. S. M. Vazirizade, A. Mukhopadhyay, G. Pettet, S. E. Said, H. Baroud, and A. Dubey, Learning Incident Prediction Models Over Large Geographical Areas for Emergency Response Systems, 2021.
  6. A. Sivagnanam, A. Ayman, M. Wilbur, P. Pugliese, A. Dubey, and A. Laszka, Minimizing Energy Use of Mixed-Fleet Public Transit for Fixed-Route Service, in Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI-21), 2021.
  7. S. Basak, S. Sengupta, S.-J. Wen, and A. Dubey, Spatio-temporal AI inference engine for estimating hard disk reliability, Pervasive and Mobile Computing, vol. 70, p. 101283, 2021.
  8. M. Wilbur, P. Pugliese, A. Laszka, and A. Dubey, Efficient Data Management for Intelligent Urban Mobility Systems, in Proceedings of the Workshop on AI for Urban Mobility at the 35th AAAI Conference on Artificial Intelligence (AAAI-21), 2021.
  9. G. Pettet, A. Mukhopadhyay, M. Kochenderfer, and A. Dubey, Hierarchical Planning for Resource Allocation in Emergency Response Systems, in Proceedings of the 12th ACM/IEEE International Conference on Cyber-Physical Systems, ICCPS 2021, Nashville, TN, USA, 2021.
  10. S. Ramakrishna, Z. RahimiNasab, G. Karsai, A. Easwaran, and A. Dubey, Efficient Out-of-Distribution Detection Using Latent Space of Ξ²-VAE for Cyber-Physical Systems, ACM Trans. Cyber-Phys. Syst., 2021.
  11. C. Hartsell, S. Ramakrishna, A. Dubey, D. Stojcsics, N. Mahadevan, and G. Karsai, ReSonAte: A Runtime Risk Assessment Framework for Autonomous Systems, in 16th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, SEAMS 2021, 2021.
  12. S. Ramakrishna, C. Hartsell, A. Dubey, P. Pal, and G. Karsai, A Methodology for Automating Assurance Case Generation, in Thirteenth International Tools and Methods of Competitive Engineering Symposium (TMCE 2020), 2020.
  13. A. Ayman, M. Wilbur, A. Sivagnanam, P. Pugliese, A. Dubey, and A. Laszka, Data-Driven Prediction of Route-Level Energy Use for Mixed-Vehicle Transit Fleets, in 2020 IEEE International Conference on Smart Computing (SMARTCOMP) (SMARTCOMP 2020), Bologna, Italy, 2020.
  14. C. Hartsell, N. Mahadevan, H. Nine, T. Bapty, A. Dubey, and G. Karsai, Workflow Automation for Cyber Physical System Development Processes, in 2020 IEEE Workshop on Design Automation for CPS and IoT (DESTION), 2020.
  15. H. Tu, S. Lukic, A. Dubey, and G. Karsai, An LSTM-Based Online Prediction Method for Building Electric Load During COVID-19, in Annual Conference of the PHM Society, 2020.
  16. G. Pettet, M. Ghosal, S. Mahserejian, S. Davis, S. Sridhar, A. Dubey, and M. Meyer, A Decision Support Framework for Grid-Aware Electric Bus Charge Scheduling, in 2020 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT), 2020.
  17. A. Ayman, A. Sivagnanam, M. Wilbur, P. Pugliese, A. Dubey, and A. Laszka, Data-Driven Prediction and Optimization of Energy Use for Transit Fleets of Electric and ICE Vehicles, ACM Transations of Internet Technology, 2020.
  18. M. Wilbur, A. Ayman, A. Ouyang, V. Poon, R. Kabir, A. Vadali, P. Pugliese, D. Freudberg, A. Laszka, and A. Dubey, Impact of COVID-19 on Public Transit Accessibility and Ridership, in Preprint at Arxiv, 2020.
  19. V. Sundar, S. Ramakrishna, Z. Rahiminasab, A. Easwaran, and A. Dubey, Out-of-Distribution Detection in Multi-Label Datasets using Latent Space of Ξ²-VAE, in 2020 IEEE Security and Privacy Workshops (SPW), Los Alamitos, CA, USA, 2020, pp. 250–255.
  20. G. Pettet, A. Mukhopadhyay, M. Kochenderfer, Y. Vorobeychik, and A. Dubey, On Algorithmic Decision Procedures in Emergency Response Systems in Smart and Connected Communities, in Proceedings of the 19th Conference on Autonomous Agents and MultiAgent Systems, AAMAS 2020, Auckland, New Zealand, 2020.
  21. C. Hartsell, N. Mahadevan, S. Ramakrishna, A. Dubey, T. Bapty, T. T. Johnson, X. D. Koutsoukos, J. Sztipanovits, and G. Karsai, Model-based design for CPS with learning-enabled components, in Proceedings of the Workshop on Design Automation for CPS and IoT, DESTION@CPSIoTWeek 2019, Montreal, QC, Canada, 2019, pp. 1–9.
  22. S. Basak, A. Dubey, and B. P. Leao, Analyzing the Cascading Effect of Traffic Congestion Using LSTM Networks, in IEEE Big Data, Los Angeles, Ca, 2019.
  23. S. Basak, F. Sun, S. Sengupta, and A. Dubey, Data-Driven Optimization of Public Transit Schedule, in Big Data Analytics - 7th International Conference, BDA 2019, Ahmedabad, India, 2019, pp. 265–284.
  24. A. Mukhopadhyay, G. Pettet, C. Samal, A. Dubey, and Y. Vorobeychik, An online decision-theoretic pipeline for responder dispatch, in Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems, ICCPS 2019, Montreal, QC, Canada, 2019, pp. 185–196.
  25. C. Hartsell, N. Mahadevan, S. Ramakrishna, A. Dubey, T. Bapty, and G. Karsai, A CPS toolchain for learning-based systems: demo abstract, in Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems, ICCPS 2019, Montreal, QC, Canada, 2019, pp. 342–343.
  26. M. P. Burruss, S. Ramakrishna, G. Karsai, and A. Dubey, DeepNNCar: A Testbed for Deploying and Testing Middleware Frameworks for Autonomous Robots, in IEEE 22nd International Symposium on Real-Time Distributed Computing, ISORC 2019, Valencia, Spain, May 7-9, 2019, 2019, pp. 87–88.
  27. C. Hartsell, N. Mahadevan, S. Ramakrishna, A. Dubey, T. Bapty, T. T. Johnson, X. D. Koutsoukos, J. Sztipanovits, and G. Karsai, CPS Design with Learning-Enabled Components: A Case Study, in Proceedings of the 30th International Workshop on Rapid System Prototyping, RSP 2019, New York, NY, USA, October 17-18, 2019, 2019, pp. 57–63.
  28. S. Ramakrishna, A. Dubey, M. P. Burruss, C. Hartsell, N. Mahadevan, S. Nannapaneni, A. Laszka, and G. Karsai, Augmenting Learning Components for Safety in Resource Constrained Autonomous Robots, in IEEE 22nd International Symposium on Real-Time Distributed Computing, ISORC 2019, Valencia, Spain, May 7-9, 2019, 2019, pp. 108–117.
  29. M. Wilbur, A. Dubey, B. LeΓ£o, and S. Bhattacharjee, A Decentralized Approach for Real Time Anomaly Detection in Transportation Networks, in IEEE International Conference on Smart Computing, SMARTCOMP 2019, Washington, DC, USA, 2019, pp. 274–282.
  30. S. Basak, S. Sengupta, and A. Dubey, Mechanisms for Integrated Feature Normalization and Remaining Useful Life Estimation Using LSTMs Applied to Hard-Disks, in IEEE International Conference on Smart Computing, SMARTCOMP 2019, Washington, DC, USA, 2019, pp. 208–216.
  31. S. Basak, A. Aman, A. Laszka, A. Dubey, and B. Leao, Data-Driven Detection of Anomalies and Cascading Failures in Traffic Networks, in Proceedings of the 11th Annual Conference of the Prognostics and Health Management Society (PHM), 2019.
  32. F. Sun, A. Dubey, C. Samal, H. Baroud, and C. Kulkarni, Short-Term Transit Decision Support System Using Multi-task Deep Neural Networks, in 2018 IEEE International Conference on Smart Computing, SMARTCOMP 2018, Taormina, Sicily, Italy, June 18-20, 2018, 2018, pp. 155–162.
  33. S. Pradhan, A. Dubey, S. Khare, S. Nannapaneni, A. S. Gokhale, S. Mahadevan, D. C. Schmidt, and M. Lehofer, CHARIOT: Goal-Driven Orchestration Middleware for Resilient IoT Systems, TCPS, vol. 2, no. 3, pp. 16:1–16:37, 2018.
  34. S. Basak, S. Sengupta, and A. Dubey, A Data-driven Prognostic Architecture for Online Monitoring of Hard Disks Using Deep LSTM Networks, CoRR, vol. abs/1810.08985, 2018.
  35. F. Sun, A. Dubey, and J. White, DxNAT - Deep neural networks for explaining non-recurring traffic congestion, in 2017 IEEE International Conference on Big Data, BigData 2017, Boston, MA, USA, December 11-14, 2017, 2017, pp. 2141–2150.
  36. A. Ghafouri, A. Laszka, A. Dubey, and X. D. Koutsoukos, Optimal detection of faulty traffic sensors used in route planning, in Proceedings of the 2nd International Workshop on Science of Smart City Operations and Platforms Engineering, SCOPE@CPSWeek 2017, Pittsburgh, PA, USA, April 21, 2017, 2017, pp. 1–6.
  37. C. Samal, F. Sun, and A. Dubey, SpeedPro: A Predictive Multi-Model Approach for Urban Traffic Speed Estimation, in 2017 IEEE International Conference on Smart Computing, SMARTCOMP 2017, Hong Kong, China, May 29-31, 2017, 2017, pp. 1–6.
  38. F. Sun, C. Samal, J. White, and A. Dubey, Unsupervised Mechanisms for Optimizing On-Time Performance of Fixed Schedule Transit Vehicles, in 2017 IEEE International Conference on Smart Computing, SMARTCOMP 2017, Hong Kong, China, May 29-31, 2017, 2017, pp. 1–8.
  39. G. Pettet, S. Nannapaneni, B. Stadnick, A. Dubey, and G. Biswas, Incident analysis and prediction using clustering and Bayesian network, in 2017 IEEE SmartWorld, 2017, pp. 1–8.
  40. A. Mukhopadhyay, Y. Vorobeychik, A. Dubey, and G. Biswas, Prioritized Allocation of Emergency Responders based on a Continuous-Time Incident Prediction Model, in Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems, AAMAS 2017, SΓ£o Paulo, Brazil, May 8-12, 2017, 2017, pp. 168–177.
  41. G. Biswas, H. Khorasgani, G. Stanje, A. Dubey, S. Deb, and S. Ghoshal, An application of data driven anomaly identification to spacecraft telemetry data, in Prognostics and Health Management Conference, 2016.