Scientific Computing Autonomic Reliability Framework

Computing clusters in scientific computing environments suffer from both transient and persistent failures that impact application performance and result in significant economic losses. The innovation of this work is applying model-based design to cluster reliability, enabling specification of mitigation policies that can be verified for correctness. This systematic approach provides a bridge between autonomous fault management concepts and practical implementation on distributed systems.

This paper presents a model-based autonomic reliability framework for computing clusters that enables periodic monitoring of vital health parameters and provides automated fault mitigation. The work develops techniques for fault prediction by analyzing correlations between system parameters like CPU utilization and temperature, enabling proactive mitigation before failures occur. The framework includes both discrete event-based scheduling and feedback control mechanisms for managing sensor execution across cluster nodes.

The framework demonstrates model-based reliability engineering for LQCD clusters with 127-600 computing nodes across multiple systems. Monitoring and mitigation components show the ability to detect and respond to various failure modes including power outages, hardware failures, and non-responsive jobs. Experimental data illustrates the effectiveness of the approach in maintaining cluster productivity despite hardware and software faults.